diff --git a/README.md b/README.md index c557dae9e84251d2f2fa25e445fe20a3aa438107..45fa45524f2422a4afd780bbfdda2d623035ad82 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,24 @@ # hotkey +![输入图片说明](https://images.gitee.com/uploads/images/2020/0616/105737_e5b876cd_303698.png "redis热key探测及缓存到JVM (1).png") -正在京东APP后台灰度了几千台机器，等618海量并发检验后再推广使用，暂时不要下载使用它。 +京东APP后台热数据探测框架，历经多次高压压测和2020年京东618大促考验。在上线运行的这段时间内，每天探测的key数量数十亿计，精准捕获了大量爬虫、刷子用户，另准确探测大量热门商品并毫秒级推送到各个服务端内存，大幅降低了热数据对数据层的查询压力，提升了应用性能。 -该框架历经多次压测，8核单机worker端每秒可接收处理16万个key探测任务，16核单机至少每秒20万，实际压测达到30万以上，CPU平稳支撑，框架无异常。测试详情可去我[CSDN博客](https://blog.csdn.net/tianyaleixiaowu)查看。 +该框架历经多次压测，8核单机worker端每秒可接收处理16万个key探测任务，16核单机至少每秒平稳处理20万以上，实际压测达到30万以上，CPU平稳支撑，框架无异常。在真实业务场景中，可用1：1000的比例，即1台worker支撑1000台服务端Tomcat的key探测任务，即可带来极大的数据存储资源节省（如对redis集群的扩充）。测试详情可去我[CSDN博客](https://blog.csdn.net/tianyaleixiaowu)查看。 + +#### 架构设计 + **见京东零售公众号** +https://mp.weixin.qq.com/s/xOzEj5HtCeh_ezHDPHw6Jw + +#### 参与开发 +该框架的开发得到了来自于： + +京东数科-生活服务研发组-李云峰的大力支持，完成了整个dashboard控制台的研发。 + +京东零售-平台业务中心-APP后台架构师 杜晓宇的大力支持，参与并帮助解决了诸多技术难题。 + +另有京东集团多个部门提供的意见、建议，和对框架的见解、优化方案。 + +在此一起感谢。 #### 介绍 对任意突发性的无法预先感知的热点请求，包括并不限于热点数据（如突发大量请求同一个商品）、热用户（如爬虫、刷子）、热接口（突发海量请求同一个接口）等，进行毫秒级精准探测到。 @@ -31,56 +47,160 @@ #### 尚未完成 -控制台功能缺失如下： +文档有待完善。 -1.rule的定时保存，当etcd更换时需要能一键导入原有配置 -2.对/jd/count/cartsoa/ 目录下信息进行展示，代表的是每个worker连接的client数量（已完成） +该开源项目战略意义重大，经历百万级并发，参与京东开源中间件项目建设，一直在等你。 -3.对/jd/caffeineSize/ 目录进行展示，里面是每个worker内caffeine缓存的数量 +#### worker端强悍的性能表现 +每10秒打印一行，totalDealCount代表处理过的key总量，可以看到每10秒处理量在270万-310万之间，对应每秒30万左右QPS。 -4.筛选功能，对记录表里做筛选，按时间范围筛选key出现次数大于xx次的数据 +仅需要很少的机器，即可完成海量key的实时探测计算推送任务。比扩容redis集群规模成本低太多。 +![输入图片说明](https://images.gitee.com/uploads/images/2020/0611/152336_78597937_303698.png "屏幕截图.png") +![输入图片说明](https://images.gitee.com/uploads/images/2020/0611/152249_4ac01178_303698.png "屏幕截图.png") -5.导出功能，将key排重后导出的功能，按时间范围筛选 -6.还有很多 +### 界面效果 +![输入图片说明](https://images.gitee.com/uploads/images/2020/0622/163805_0aa68d4b_303698.png "屏幕截图.png") +### 加我微信拉入群 +![输入图片说明](https://images.gitee.com/uploads/images/2020/0723/165039_67a40b9f_303698.jpeg "WechatIMG77.jpeg") -该开源项目战略意义重大，要经历百万级并发，参与京东开源中间件项目建设，一直在等你。 +#### 安装教程 -#### worker端强悍的性能表现 -每10秒打印一行，totalDealCount代表处理过的key总量，可以看到每10秒处理量在270万-310万之间，对应每秒30万左右QPS。 +1. 安装etcd -仅需要很少的机器，即可完成海量key的实时探测计算推送任务。比扩容redis集群规模成本低太多了。 -![输入图片说明](https://images.gitee.com/uploads/images/2020/0611/152336_78597937_303698.png "屏幕截图.png") -![输入图片说明](https://images.gitee.com/uploads/images/2020/0611/152249_4ac01178_303698.png "屏幕截图.png") + 在etcd下载页面下载对应操作系统的etcd，https://github.com/etcd-io/etcd/releases 使用3.4.x以上。相关搭建细节，及常见问题会发布到CSDN博客内。 +2. 启动worker（集群） + 下载并编译好代码，将worker打包为jar，启动即可。如： -#### 安装教程 + **` java -jar $JAVA_OPTS worker-0.0.1-SNAPSHOT.jar --etcd.server=${etcdServer}`** + + worker可供配置项如下： +![输入图片说明](https://images.gitee.com/uploads/images/2020/0622/164514_c57d740a_303698.png "屏幕截图.png") + etcdServer为etcd集群的地址，用逗号分隔 + + JAVA_OPTS是配置的JVM相关，可根据实际情况配置 + + threadCount为处理key的线程数，不指定时由程序来计算。 + + workerPath代表该worker为哪个应用提供计算服务，譬如不同的应用appName需要用不同的worker进行隔离，以避免资源竞争。 + +3. 启动控制台 + + 下载并编译好dashboard项目，创建数据库并导入resource下db.sql文件。 配置一下application.yml里的数据库相关和etcdServer地址。 + + 启动dashboard项目，访问ip:8081，即可看到界面。 + + 其中节点信息里，即是当前已启动的worker列表。 + + 规则配置就是为各app设置规则的地方，初次使用时需要先添加APP。在用户管理菜单中，添加一个新用户，设置他的APP名字，如sample。之后新添加的这个用户就可以登录dashboard给自己的APP设置规则了，登录密码默认123456。 +![输入图片说明](https://images.gitee.com/uploads/images/2020/0622/175255_e1b05b4c_303698.png "屏幕截图.png") + + 如图就是一组规则，譬如其中as__开头的热key的规则就是interval-2秒内出现了threshold-10次就认为它是热key，它就会被推送到jvm内存中，并缓存60秒，prefix-true代表前缀匹配。那么在应用中，就可以把一组key，都用as__开头，用来探测。 + +4. client端接入使用 + + 引入client的pom依赖。 + + 在应用启动的地方初始化HotKey，譬如 + +``` +@PostConstruct + +public void initHotkey() { + + ClientStarter.Builder builder = new ClientStarter.Builder(); + ClientStarter starter = builder.setAppName("appName").setEtcdServer("http://1.8.8.4:2379,http://1.1.4.4:2379,http://1.1.1.1:2379").build(); + starter.startPipeline(); +} +``` +其中还可以setCaffeineSize(int size)设置本地缓存最大数量，默认5万，setPushPeriod(Long period)设置批量推送key的间隔时间，默认500ms，该值越小，上报热key越频繁，相应越及时，建议根据实际情况调整，如单机每秒qps10个，那么0.5秒上报一次即可，否则是空跑。该值最小为1，即1ms上报一次。 + +注意： + +如果原有项目里使用了guava，需要升级guava为以下版本，否则过低的guava版本可能发生jar包冲突。或者删除自己项目里的guava的maven依赖，guava升级不会影响原有任何逻辑。 + +``` + + com.google.guava + guava + 28.2-jre + compile + +``` + +有时可能项目里没有直接依赖guava，但是引入的某个pom里引了guava，也需要将guava排除掉。 + + + +使用： + +主要有如下4个方法可供使用 + +boolean JdHotKeyStore.isHotKey(String key) + +Object JdHotKeyStore.get(String key) + +void JdHotKeyStore.smartSet(String key, Object value) + +Object JdHotKeyStore.getValue(String key) + + + +1 boolean isHotKey(String key) ，该方法会返回该key是否是热key，如果是返回true，如果不是返回false，并且会将key上报到探测集群进行数量计算。该方法通常用于判断只需要判断key是否热、不需要缓存value的场景，如刷子用户、接口访问频率等。 + +2 Object get(String key)，该方法返回该key本地缓存的value值，可用于判断是热key后，再去获取本地缓存的value值，通常用于redis热key缓存 + +3 void smartSet(String key, Object value)，方法给热key赋值value，如果是热key，该方法才会赋值，非热key，什么也不做 + +4 Object getValue(String key)，该方法是一个整合方法，相当于isHotKey和get两个方法的整合，该方法直接返回本地缓存的value。 +如果是热key，则存在两种情况，1是返回value，2是返回null。返回null是因为尚未给它set真正的value，返回非null说明已经调用过set方法了，本地缓存value有值了。 +如果不是热key，则返回null，并且将key上报到探测集群进行数量探测。 + + +最佳实践： + +1 判断用户是否是刷子 + + if (JdHotKeyStore.isHotKey(“pin__” + thePin)) { + //限流他，do your job + } +2 判断商品id是否是热点 + + + Object skuInfo = JdHotKeyStore.getValue("skuId__" + skuId); + if(skuInfo == null) { + JdHotKeyStore.smartSet("skuId__" + skuId, theSkuInfo); + } else { + //使用缓存好的value即可 + } + + 或者这样： + + + + if (JdHotKeyStore.isHotKey(key)) { + //注意是get，不是getValue。getValue会获取并上报，get是纯粹的本地获取 + Object skuInfo = JdHotKeyStore.get("skuId__" + skuId); + if(skuInfo == null) { + JdHotKeyStore.smartSet("skuId__" + skuId, theSkuInfo); + } else { + //使用缓存好的value即可 + } + + } -1. xxxx -2. xxxx -3. xxxx +### 测试环境 -#### 使用说明 +我司为方便大家快速接入试用、查看hotkey效果，提供了1台公网etcd机器（16c16g）、2台worker机器（8c12g），供快速接入测试。 -1. xxxx -2. xxxx -3. xxxx -#### 参与贡献 +etcd地址为： **http://open-etcd.jd.com:2000** ，可以在下载项目后，在sample项目、dashboard项目里yml文件修改etcd连接地址为该地址，然后进入界面控制台，即可看到当前连接的worker，之后可以在sample项目里，进行hotkey测试。 -1. Fork 本仓库 -2. 新建 Feat_xxx 分支 -3. 提交代码 -4. 新建 Pull Request +控制台公网测试地址：http://hotkey.tianyalei.com:9001/ 账号admin，密码123456 -#### 码云特技 -1. 使用 Readme\_XXX.md 来支持不同的语言，例如 Readme\_en.md, Readme\_zh.md -2. 码云官方博客 [blog.gitee.com](https://blog.gitee.com) -3. 你可以 [https://gitee.com/explore](https://gitee.com/explore) 这个地址来了解码云上的优秀开源项目 -4. [GVP](https://gitee.com/gvp) 全称是码云最有价值开源项目，是码云综合评定出的优秀开源项目 -5. 码云官方提供的使用手册 [https://gitee.com/help](https://gitee.com/help) -6. 码云封面人物是一档用来展示码云会员风采的栏目 [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/) +测试前，注意在控制台新建自己的app规则。 diff --git a/client/src/main/java/com/jd/platform/hotkey/client/core/key/NettyKeyPusher.java b/client/src/main/java/com/jd/platform/hotkey/client/core/key/NettyKeyPusher.java index d5afa72900cffa6c9c37570a83eafd85171b5554..f98548963e506d402941d23793713e0394c114a0 100644 --- a/client/src/main/java/com/jd/platform/hotkey/client/core/key/NettyKeyPusher.java +++ b/client/src/main/java/com/jd/platform/hotkey/client/core/key/NettyKeyPusher.java @@ -44,7 +44,7 @@ public class NettyKeyPusher implements IKeyPusher { for (Channel channel : map.keySet()) { try { List batch = map.get(channel); - channel.writeAndFlush(MsgBuilder.buildByteBuf(new HotKeyMsg(MessageType.REQUEST_NEW_KEY, FastJsonUtils.convertObjectToJSON(batch)))); + channel.writeAndFlush(MsgBuilder.buildByteBuf(new HotKeyMsg(MessageType.REQUEST_NEW_KEY, FastJsonUtils.convertObjectToJSON(batch)))).sync(); } catch (Exception e) { try { InetSocketAddress insocket = (InetSocketAddress) channel.remoteAddress(); @@ -78,7 +78,7 @@ public class NettyKeyPusher implements IKeyPusher { try { List batch = map.get(channel); channel.writeAndFlush(MsgBuilder.buildByteBuf(new HotKeyMsg(Context.APP_NAME, - MessageType.REQUEST_HIT_COUNT, FastJsonUtils.convertObjectToJSON(batch)))); + MessageType.REQUEST_HIT_COUNT, FastJsonUtils.convertObjectToJSON(batch)))).sync(); } catch (Exception e) { try { InetSocketAddress insocket = (InetSocketAddress) channel.remoteAddress(); diff --git a/client/src/main/java/com/jd/platform/hotkey/client/etcd/EtcdStarter.java b/client/src/main/java/com/jd/platform/hotkey/client/etcd/EtcdStarter.java index c129fb155b99d17303e2d2bcad97056cdc992c52..216160a3b7e0d47f9e70db646018f9066ca87f4d 100644 --- a/client/src/main/java/com/jd/platform/hotkey/client/etcd/EtcdStarter.java +++ b/client/src/main/java/com/jd/platform/hotkey/client/etcd/EtcdStarter.java @@ -38,8 +38,6 @@ public class EtcdStarter { fetchRule(); - fetchExistHotKey(); - // startWatchWorker(); startWatchRule(); @@ -243,6 +241,9 @@ public class EtcdStarter { JdLogger.info(getClass(), "trying to connect to etcd and fetch rule info"); boolean success = fetchRuleFromEtcd(); if (success) { + //拉取已存在的热key + fetchExistHotKey(); + scheduledExecutorService.shutdown(); } diff --git a/common/src/main/java/com/jd/platform/hotkey/common/configcenter/etcd/JdEtcdClient.java b/common/src/main/java/com/jd/platform/hotkey/common/configcenter/etcd/JdEtcdClient.java index 06e85e957794fa9b0559ac81836480bdbd997f56..fb64d9e28b9a3217e1824f93b762f147427fd205 100644 --- a/common/src/main/java/com/jd/platform/hotkey/common/configcenter/etcd/JdEtcdClient.java +++ b/common/src/main/java/com/jd/platform/hotkey/common/configcenter/etcd/JdEtcdClient.java @@ -3,7 +3,6 @@ package com.jd.platform.hotkey.common.configcenter.etcd; import cn.hutool.core.collection.CollectionUtil; import com.google.protobuf.ByteString; import com.ibm.etcd.api.KeyValue; -import com.ibm.etcd.api.Kv; import com.ibm.etcd.api.LeaseGrantResponse; import com.ibm.etcd.api.RangeResponse; import com.ibm.etcd.client.KvStoreClient; diff --git a/dashboard/pom.xml b/dashboard/pom.xml index 30eacfa44eebc69b66ed08db05272acf602260ec..41dd9cdbc1ac0b5ad3fd163a91c5ca1b29e27c79 100644 --- a/dashboard/pom.xml +++ b/dashboard/pom.xml @@ -81,11 +81,6 @@ poi-ooxml 4.0.1 - - com.jd.common - sso-uim-spring - 1.2.0-SNAPSHOT - diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/DashboardApplication.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/DashboardApplication.java index 2b361c66ba6119558b49e80da745a2b5d80fb2d7..074e45d6b2dfde723115750dc1d18b8a049bf2fa 100644 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/DashboardApplication.java +++ b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/DashboardApplication.java @@ -1,12 +1,7 @@ package com.jd.platform.hotkey.dashboard; -import com.alibaba.fastjson.JSON; -import com.jd.platform.hotkey.common.configcenter.ConfigConstant; -import com.jd.platform.hotkey.common.configcenter.IConfigCenter; -import com.jd.platform.hotkey.common.tool.IpUtils; import com.jd.platform.hotkey.dashboard.mapper.KeyTimelyMapper; -import com.jd.platform.hotkey.dashboard.mapper.RulesMapper; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.boot.CommandLineRunner; @@ -14,10 +9,7 @@ import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.scheduling.annotation.EnableAsync; import org.springframework.scheduling.annotation.EnableScheduling; - import javax.annotation.Resource; -import java.util.HashMap; -import java.util.Map; @EnableAsync @@ -25,7 +17,6 @@ import java.util.Map; @SpringBootApplication public class DashboardApplication implements CommandLineRunner { - private Logger logger = LoggerFactory.getLogger(getClass()); @Resource @@ -37,11 +28,11 @@ public class DashboardApplication implements CommandLineRunner { }catch (Exception e){ e.printStackTrace(); } - } + @Override - public void run(String... args) { + public void run(String... args) { int row = timelyMapper.clear(); logger.info("clear db timely hotKey, effect row : {}",row); } diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/common/domain/vo/RuleLineChartVo.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/common/domain/vo/RuleLineChartVo.java deleted file mode 100644 index 31ba5305e879aa21bdaa2c339fb639a9f8360584..0000000000000000000000000000000000000000 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/common/domain/vo/RuleLineChartVo.java +++ /dev/null @@ -1,35 +0,0 @@ -package com.jd.platform.hotkey.dashboard.common.domain.vo; - -/** - * @author liyunfeng31 - */ -public class RuleLineChartVo { - - private String rule; - - private Integer count; - - public RuleLineChartVo() { - } - - public RuleLineChartVo(String rule, Integer count) { - this.rule = rule; - this.count = count; - } - - public String getRule() { - return rule; - } - - public void setRule(String rule) { - this.rule = rule; - } - - public Integer getCount() { - return count; - } - - public void setCount(Integer count) { - this.count = count; - } -} diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/controller/UserController.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/controller/UserController.java index bc571ba53252bac31d25651c221387a27ddf0e94..67053667f47241e683edc25ad4a8c32648dee025 100644 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/controller/UserController.java +++ b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/controller/UserController.java @@ -1,14 +1,12 @@ package com.jd.platform.hotkey.dashboard.controller; -import java.util.*; - import com.github.pagehelper.PageInfo; import com.github.pagehelper.util.StringUtil; import com.jd.platform.hotkey.dashboard.common.base.BaseController; import com.jd.platform.hotkey.dashboard.common.domain.Constant; import com.jd.platform.hotkey.dashboard.common.domain.Page; -import com.jd.platform.hotkey.dashboard.common.domain.req.PageReq; import com.jd.platform.hotkey.dashboard.common.domain.Result; +import com.jd.platform.hotkey.dashboard.common.domain.req.PageReq; import com.jd.platform.hotkey.dashboard.common.eunm.ResultEnum; import com.jd.platform.hotkey.dashboard.model.User; import com.jd.platform.hotkey.dashboard.service.UserService; @@ -23,6 +21,8 @@ import javax.annotation.Resource; import javax.servlet.http.Cookie; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; +import java.util.HashMap; +import java.util.Map; @Controller diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpConfiguration.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpConfiguration.java deleted file mode 100644 index 6b12a85d17330d3175b0efe00331b8d757156f31..0000000000000000000000000000000000000000 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpConfiguration.java +++ /dev/null @@ -1,24 +0,0 @@ -package com.jd.platform.hotkey.dashboard.erp; - - -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; - -@Configuration -public class ErpConfiguration { - - @Autowired - ErpProperties erpProperties; - - @Bean - public ErpUimInterceptor erpUimInterceptor() { - ErpUimInterceptor springSSOInterceptor = new ErpUimInterceptor(); - springSSOInterceptor.setSsoAppKey(erpProperties.getSsoAppKey()); - springSSOInterceptor.setSsoAppToken(erpProperties.getSsoAppToken()); - springSSOInterceptor.setLoginUrl(erpProperties.getLoginUrl()); - springSSOInterceptor.setExcludePath(erpProperties.getExcludePath()); - springSSOInterceptor.setSsoAppUrl(erpProperties.getSsoAppUrl()); - return springSSOInterceptor; - } -} diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpProperties.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpProperties.java deleted file mode 100644 index 1d4cf23e1875976e8761598f1e1017408ce758ef..0000000000000000000000000000000000000000 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpProperties.java +++ /dev/null @@ -1,75 +0,0 @@ -package com.jd.platform.hotkey.dashboard.erp; - -import com.jd.platform.hotkey.dashboard.autoconfigure.AbstractProperties; -import org.springframework.boot.context.properties.ConfigurationProperties; -import org.springframework.stereotype.Component; - -/** - * User: fuxueliang - * Date: 16/8/17 - * Email: fuxueliang@jd.com - */ -@Component -@ConfigurationProperties(prefix = "erp") -public class ErpProperties extends AbstractProperties { - - private String excludePath; - - private String loginUrl; - - private String ssoAppUrl; - - private String ssoAppKey; - - private String ssoAppToken; - - private Boolean enabled; - - public String getExcludePath() { - return excludePath; - } - - public void setExcludePath(String excludePath) { - this.excludePath = excludePath; - } - - public String getLoginUrl() { - return loginUrl; - } - - public void setLoginUrl(String loginUrl) { - this.loginUrl = loginUrl; - } - - public String getSsoAppUrl() { - return ssoAppUrl; - } - - public void setSsoAppUrl(String ssoAppUrl) { - this.ssoAppUrl = ssoAppUrl; - } - - public String getSsoAppKey() { - return ssoAppKey; - } - - public void setSsoAppKey(String ssoAppKey) { - this.ssoAppKey = ssoAppKey; - } - - public String getSsoAppToken() { - return ssoAppToken; - } - - public void setSsoAppToken(String ssoAppToken) { - this.ssoAppToken = ssoAppToken; - } - - public Boolean getEnabled() { - return enabled; - } - - public void setEnabled(Boolean enabled) { - this.enabled = enabled; - } -} diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpUimInterceptor.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpUimInterceptor.java deleted file mode 100644 index beabf4e29cf2a1b453357ad18d5b0cc05d95f892..0000000000000000000000000000000000000000 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/erp/ErpUimInterceptor.java +++ /dev/null @@ -1,56 +0,0 @@ -package com.jd.platform.hotkey.dashboard.erp; - -import com.jd.common.springmvc.interceptor.SpringSSOInterceptor; -import com.jd.common.web.LoginContext; -import com.jd.platform.hotkey.dashboard.model.User; -import com.jd.platform.hotkey.dashboard.service.UserService; -import org.springframework.beans.factory.annotation.Autowired; - -import javax.servlet.http.Cookie; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; - - -public class ErpUimInterceptor extends SpringSSOInterceptor { - - @Autowired - private UserService userService; - - public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception { - LoginContext loginContext = LoginContext.getLoginContext(); - if (null == loginContext) { - super.preHandle(request, response, handler); - } - loginContext = LoginContext.getLoginContext(); - if (null == loginContext) { - return true; - } - /*Cookie cookie = new Cookie("erp", loginContext.getNick()); - cookie.setMaxAge(3600*24*7); - cookie.setPath("/"); - response.addCookie(cookie);*/ - String pin = loginContext.getPin(); - String nickName = loginContext.getNick(); - String orgName = loginContext.getOrgName(); - String orgId = loginContext.getOrgId(); - String mobile = loginContext.getMobile(); - String email = loginContext.getEmail(); - User user = new User(); - user.setAppName("test"); - user.setUserName(pin); - user.setNickName(nickName); - user.setPhone(mobile); - user.setRole("APPUSER"); - Cookie cookie = userService.loginErpUser(user); - Cookie[] cookies = request.getCookies(); - if(cookies != null){ - for(Cookie tempCookie : cookies){ - if("token".equals(tempCookie.getName()) && tempCookie.getValue() != null){ - return true; - } - } - } - response.addCookie(cookie); - return true; - } -} diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/JwtInterceptor.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/JwtInterceptor.java index 173a46598708858ad5b59aab24f3a855ca6f7ac8..02bc53e79934459c16291fd31c3dd19ce59e8a2f 100644 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/JwtInterceptor.java +++ b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/JwtInterceptor.java @@ -29,9 +29,8 @@ public class JwtInterceptor extends HandlerInterceptorAdapter{ } String header = request.getHeader("x-requested-with"); if(!StringUtils.isEmpty(header) && "XMLHttpRequest".endsWith(header) && request.getMethod().equals(Constant.POST)){ - final String authHeader = JwtTokenUtil.getAuthHeader(request); - if (StringUtils.isEmpty(authHeader) - || !authHeader.startsWith(JwtTokenUtil.TOKEN_PREFIX)) { + String authHeader = request.getHeader(JwtTokenUtil.AUTH_HEADER_KEY); + if (StringUtils.isEmpty(authHeader) || !authHeader.startsWith(JwtTokenUtil.TOKEN_PREFIX)) { throw new BizException(ResultEnum.NO_LOGIN); } final String token = authHeader.substring(2); diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/MyWebAppConfigurer.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/MyWebAppConfigurer.java index 529e41683a9683b98454a852c53cc84b923dbe76..7360a638195fffb47203f3c5b9909c863e00de07 100644 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/MyWebAppConfigurer.java +++ b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/interceptor/MyWebAppConfigurer.java @@ -1,22 +1,18 @@ package com.jd.platform.hotkey.dashboard.interceptor; -import com.jd.platform.hotkey.dashboard.erp.ErpUimInterceptor; -import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Configuration; import org.springframework.core.Ordered; import org.springframework.web.servlet.config.annotation.*; + @Configuration public class MyWebAppConfigurer implements WebMvcConfigurer { - @Autowired - ErpUimInterceptor erpUimInterceptor; @Override public void addInterceptors(InterceptorRegistry registry) { //拦截路径可自行配置多个 可用 ，分隔开 InterceptorRegistration registration = registry.addInterceptor(new JwtInterceptor()).addPathPatterns("/**"); - registry.addInterceptor(erpUimInterceptor); - registration.excludePathPatterns("/user/login","/user/getUserName","/error","/static/**","/main/**"); + registration.excludePathPatterns("/user/login","/error","/static/**","/main/**"); } diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/UserService.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/UserService.java index 71f57ea705fefcbe37dfdd62b940d6bb303ae83b..67db3d3ca56e439239399d2f498480ee4c10bcf0 100644 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/UserService.java +++ b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/UserService.java @@ -5,7 +5,6 @@ import com.jd.platform.hotkey.dashboard.common.domain.req.PageReq; import com.jd.platform.hotkey.dashboard.common.domain.req.SearchReq; import com.jd.platform.hotkey.dashboard.model.User; -import javax.servlet.http.Cookie; import java.util.List; /** @@ -23,10 +22,6 @@ public interface UserService { int insertUser(User user); - int insertUserByErp(User user); - - Cookie loginErpUser(User user); - int deleteByPrimaryKey(int id); User selectByPrimaryKey(int id); diff --git a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/impl/UserServiceImpl.java b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/impl/UserServiceImpl.java index 5cb83e5d0c0a983ba0a78bc40fdf273c15300c2b..d81f51e56296c0a58723450800295d992e65c6a9 100644 --- a/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/impl/UserServiceImpl.java +++ b/dashboard/src/main/java/com/jd/platform/hotkey/dashboard/service/impl/UserServiceImpl.java @@ -34,8 +34,6 @@ import java.util.Set; @Service public class UserServiceImpl implements UserService { - @Value("${erp.defaultPwd}") - private String defaultPwd; @Resource private UserMapper userMapper; @Resource @@ -68,46 +66,6 @@ public class UserServiceImpl implements UserService { return userMapper.insertSelective(user); } - @Override - public int insertUserByErp(User user) { - User userParam = new User(); - userParam.setUserName(user.getUserName()); - List users = userMapper.selectHkUserList(userParam); - if(users.size() == 0){ - user.setCreateTime(new Date()); - user.setPwd(DigestUtils.md5DigestAsHex(defaultPwd.getBytes())); - int ret = userMapper.insertSelective(user); - System.out.println(user.getId()); - return ret; - } - return 0; - } - - @Override - public Cookie loginErpUser(User user){ - User userParam = new User(); - userParam.setUserName(user.getUserName()); - List users = userMapper.selectHkUserList(userParam); - if(users.size() == 0){ - user.setCreateTime(new Date()); - user.setPwd(DigestUtils.md5DigestAsHex(defaultPwd.getBytes())); - userMapper.insertSelective(user); - String token = JwtTokenUtil.createJWT(user.getId(), user.getUserName(), "", user.getAppName(), user.getNickName()); - Cookie cookie = new Cookie("token", JwtTokenUtil.TOKEN_PREFIX + token); - cookie.setMaxAge(3600*24*7); - //cookie.setDomain("localhost"); - cookie.setPath("/"); - return cookie; - }else{ - user =users.get(0); - String token = JwtTokenUtil.createJWT(user.getId(), user.getUserName(), "", user.getAppName(), user.getNickName()); - Cookie cookie = new Cookie("token", JwtTokenUtil.TOKEN_PREFIX + token); - cookie.setMaxAge(3600*24*7); - //cookie.setDomain("localhost"); - cookie.setPath("/"); - return cookie; - } - } @Override public int deleteByPrimaryKey(int id) { @@ -150,5 +108,4 @@ public class UserServiceImpl implements UserService { return new PageInfo<>(users); } - } diff --git a/dashboard/src/main/resources/application.yml b/dashboard/src/main/resources/application.yml index 1678e74c5fcc2b262ffd71306c084db0ae54475b..2e1b78e89e38a52ed759b65aa84b9f1fc89f1181 100644 --- a/dashboard/src/main/resources/application.yml +++ b/dashboard/src/main/resources/application.yml @@ -1,5 +1,5 @@ server : - port : 8080 + port : 8081 servlet : context-path : / spring : @@ -27,26 +27,19 @@ spring : time-zone: GMT+8 date-format: yyyy-MM-dd HH:mm:ss datasource: - username: root - password: password - url: jdbc:mysql://localhost:3306/hotkey?useUnicode=true&characterEncoding=utf-8&useSSL=true&serverTimezone=UTC + username: ${MYSQL_USER:root} + password: ${MYSQL_PASS:JRTEST} + url: jdbc:mysql://${MYSQL_HOST:localhost}:3306/luck?useUnicode=true&characterEncoding=utf-8&useSSL=true&serverTimezone=UTC&useTimezone=true&serverTimezone=GMT%2B8 driver-class-name: com.mysql.cj.jdbc.Driver pagehelper: helperDialect: mysql reasonable: true supportMethodsArguments: true params: count=countSql -mybatis: - configuration: - log-impl: org.apache.ibatis.logging.stdout.StdOutImpl +#mybatis: +# configuration: +# log-impl: org.apache.ibatis.logging.stdout.StdOutImpl etcd: server: ${etcdServer:http://127.0.0.1:2379} -erp: - enabled: false - ssoAppKey: fangzhou - ssoAppToken: 3f48d7ade61940f2ba22079ebc273fb8 - loginUrl: https://ssa.jd.com/sso/login - excludePath: /login,/assets,/bootstarp,/common,/htmlformat,/images,/jn,/jquery,/user/login,login - ssoAppUrl: http://ssa.jd.com - defaultPwd: 123456 + diff --git a/dashboard/src/main/resources/db.sql b/dashboard/src/main/resources/db.sql index 06e6b0c4d8442092708ef8ae15666fd545131932..98fbd8249250c1793df22ba2660b33123b8a85ce 100644 --- a/dashboard/src/main/resources/db.sql +++ b/dashboard/src/main/resources/db.sql @@ -31,6 +31,11 @@ CREATE TABLE `hk_user` ( ) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8 COLLATE = utf8_bin ROW_FORMAT = Compact; +-- pwd: 123456 +INSERT INTO `hk_user` VALUES (2, 'admin', 'admin', 'e10adc3949ba59abbe56e057f20f883e', '1888888', 'ADMIN', '', '2020-07-28 14:01:03', 1); + + + DROP TABLE IF EXISTS `hk_key_record`; CREATE TABLE `hk_key_record` ( `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT '主键', diff --git a/dashboard/src/main/resources/logback-spring.xml b/dashboard/src/main/resources/logback-spring.xml index f97e83e65348a9ba35ad545f5dfc813a328375ac..9e1b14837b90a6f37add61dae0d4768bce25ea62 100644 --- a/dashboard/src/main/resources/logback-spring.xml +++ b/dashboard/src/main/resources/logback-spring.xml @@ -6,7 +6,7 @@ debug：当此属性设置为true时，将打印出logback内部日志信息， --> - + diff --git a/dashboard/src/main/resources/static/admin/assets/js/bootstrap/js/base_list.js b/dashboard/src/main/resources/static/admin/assets/js/bootstrap/js/base_list.js index 05eb56eb7f5ac72a9d28e8518840d0971a752788..59b7aacd288473060d2896611820853205342f75 100644 --- a/dashboard/src/main/resources/static/admin/assets/js/bootstrap/js/base_list.js +++ b/dashboard/src/main/resources/static/admin/assets/js/bootstrap/js/base_list.js @@ -53,7 +53,7 @@ }, onLoadError: function(status){ //加载失败时执行 let token = getCookie("token"); - if(status === 500 && ( token == "undefined" || token =="")){ + if(status === 1000 && ( token == "undefined" || token =="")){ top.location.href = '/user/login'; } console.info("加载数据失败"); diff --git a/dashboard/src/main/resources/templates/admin/common/html/footer.html b/dashboard/src/main/resources/templates/admin/common/html/footer.html index 662d4f30764f30e25475563e15ea89bfeb0405c9..703a3594a77952ccb19a881d1cf944cfee009c22 100644 --- a/dashboard/src/main/resources/templates/admin/common/html/footer.html +++ b/dashboard/src/main/resources/templates/admin/common/html/footer.html @@ -10,8 +10,8 @@ - 2020 京东零售-平台业务中心-平台业务研发部-基础业务研发部 版权所有 | 使用支持：武伟峰 | 咚咚：85134158 -
接入文档：https://cf.jd.com/pages/viewpage.action?pageId=277756998 + diff --git a/dashboard/src/main/resources/templates/admin/common/html/title.html b/dashboard/src/main/resources/templates/admin/common/html/title.html index 08cf7c852be0a4c250e6364ec7390222649a51e8..aa2fcafcb48c7a172780444f97aa42d28e7e9143 100644 --- a/dashboard/src/main/resources/templates/admin/common/html/title.html +++ b/dashboard/src/main/resources/templates/admin/common/html/title.html @@ -36,8 +36,8 @@
Widget