diff --git a/kafka/README.md b/kafka/README.md new file mode 100644 index 0000000000000000000000000000000000000000..f174e19697bb58b0fd5ef38a2e3b3b85fc667ebd --- /dev/null +++ b/kafka/README.md @@ -0,0 +1,15 @@ +# kafka + +#### 介绍 +以Kafka 2.5.0版本使能鲲鹏BoostKit分布式存储算法加速库CRC32加速算法,其他版本可参考本章节使能加速算法。使能该加速算法用于加速Kafka网络传输的CRC32算法计算性能,提升端到端性能。 + +#### 操作系统 +openEuler 22.03 LTS SP3 + +#### 使用说明 +用户指南链接: +https://www.hikunpeng.com/document/detail/zh/kunpengbds/ecosystemEnable/Kafka/kunpengkafkahdp_05_0020.html + +#### 参与贡献 + 如果您想为本仓库贡献代码,请向本仓库任意maintainer发送邮件 + 如果您找到产品中的任何Bug,欢迎您提出ISSUE diff --git a/load-based-schedule/README.md b/load-based-schedule/README.md index 33b81bc4bde04c55fc0720a305f44f1d9fa4659b..c48c6912e8b132e994a8d0e2a1143afd5dbe27ae 100644 --- a/load-based-schedule/README.md +++ b/load-based-schedule/README.md @@ -1 +1,15 @@ -大数据-Yarn负载调度算法 \ No newline at end of file +# load-based-schedule + +#### 介绍 +Yarn负载调度算法优化开源版本Capacity Scheduler调度器,基于集群节点物理资源权重计算及排序结果进行资源调度,实现资源的均衡配置和高效利用。 + +#### 操作系统 +openEuler 22.03 LTS SP3 + +#### 使用说明 +用户指南链接: +https://www.hikunpeng.com/document/detail/zh/kunpengbds/appAccelFeatures/sqlqueryaccelf/kunpengbds_omniruntime_20_0803.html + +#### 参与贡献 + 如果您想为本仓库贡献代码,请向本仓库任意maintainer发送邮件 + 如果您找到产品中的任何Bug,欢迎您提出ISSUE diff --git a/omnimv/omnimv-spark-extension/README.md b/omnimv/omnimv-spark-extension/README.md index c4e24f1d0ee3cc2c4dd7580f81d1770c5a53a65f..5f9568ecb9d58bc1fc660be179d05efbc08ced56 100644 --- a/omnimv/omnimv-spark-extension/README.md +++ b/omnimv/omnimv-spark-extension/README.md @@ -1,34 +1,15 @@ # omnimv-spark-extension -A SQL Engine Extension for Spark SQL to support Materialized View +#### 介绍 +Spark物化视图插件,提供自动改写SQL的执行计划替换成物化视图的能力。 -## Introduction +#### 操作系统 +openEuler 22.03 LTS SP1 -Experiments show that there are duplicate subqueries in batch query,and materialized view is a pre-computing and caching -technology to reduce repeated calculation in batch query in database field. -The AI algorithm is used to analyze historical Spark tasks and recommend the optimal materialized view under certain -conditions. -The Spark plugin is used to add materialized view management and execution plan rewriting capabilities, greatly -improving Spark computing efficiency. +#### 使用说明 +用户指南链接: +https://www.hikunpeng.com/document/detail/zh/kunpengbds/appAccelFeatures/sqlqueryaccelf/kunpengbds_omniruntime_20_0403.html -## Environment for building OmniMV - -```shell -# download -wget --no-check-certificate https://mirrors.huaweicloud.com/kunpeng/dist/Apache/hadoop-3.1.1.tar.gz -# unpack -tar -zxvf hadoop-3.1.1.tar.gz -# set environment variables -export HADOOP_HOME="${pwd}/haddoop-3.1.1" -``` - -## Build OmniMV - -pull the OmniMV code and compile it to get the jar package - -```shell -git clone https://gitee.com/kunpengcompute/boostkit-bigdata.git -cd boostkit-bigdata/omnimv/omnimv-spark-extension -# This step can be compiled, tested and packaged to get plugin/boostkit-omnimv-spark-${omniMV.version}.jar -mvn clean package -``` +#### 参与贡献 + 如果您想为本仓库贡献代码,请向本仓库任意maintainer发送邮件 + 如果您找到产品中的任何Bug,欢迎您提出ISSUE