From 960179dff90d3d4a8845b72ff923165c1dd9e983 Mon Sep 17 00:00:00 2001 From: helloxteen Date: Thu, 3 Jul 2025 20:02:03 +0800 Subject: [PATCH] modify README.md --- kafka/README.md | 15 ++++++++++ load-based-schedule/README.md | 16 +++++++++- omnimv/omnimv-spark-extension/README.md | 39 +++++++------------------ 3 files changed, 40 insertions(+), 30 deletions(-) create mode 100644 kafka/README.md diff --git a/kafka/README.md b/kafka/README.md new file mode 100644 index 000000000..f174e1969 --- /dev/null +++ b/kafka/README.md @@ -0,0 +1,15 @@ +# kafka + +#### 介绍 +以Kafka 2.5.0版本使能鲲鹏BoostKit分布式存储算法加速库CRC32加速算法,其他版本可参考本章节使能加速算法。使能该加速算法用于加速Kafka网络传输的CRC32算法计算性能,提升端到端性能。 + +#### 操作系统 +openEuler 22.03 LTS SP3 + +#### 使用说明 +用户指南链接: +https://www.hikunpeng.com/document/detail/zh/kunpengbds/ecosystemEnable/Kafka/kunpengkafkahdp_05_0020.html + +#### 参与贡献 + 如果您想为本仓库贡献代码,请向本仓库任意maintainer发送邮件 + 如果您找到产品中的任何Bug,欢迎您提出ISSUE diff --git a/load-based-schedule/README.md b/load-based-schedule/README.md index 33b81bc4b..c48c6912e 100644 --- a/load-based-schedule/README.md +++ b/load-based-schedule/README.md @@ -1 +1,15 @@ -大数据-Yarn负载调度算法 \ No newline at end of file +# load-based-schedule + +#### 介绍 +Yarn负载调度算法优化开源版本Capacity Scheduler调度器,基于集群节点物理资源权重计算及排序结果进行资源调度,实现资源的均衡配置和高效利用。 + +#### 操作系统 +openEuler 22.03 LTS SP3 + +#### 使用说明 +用户指南链接: +https://www.hikunpeng.com/document/detail/zh/kunpengbds/appAccelFeatures/sqlqueryaccelf/kunpengbds_omniruntime_20_0803.html + +#### 参与贡献 + 如果您想为本仓库贡献代码,请向本仓库任意maintainer发送邮件 + 如果您找到产品中的任何Bug,欢迎您提出ISSUE diff --git a/omnimv/omnimv-spark-extension/README.md b/omnimv/omnimv-spark-extension/README.md index c4e24f1d0..5f9568ecb 100644 --- a/omnimv/omnimv-spark-extension/README.md +++ b/omnimv/omnimv-spark-extension/README.md @@ -1,34 +1,15 @@ # omnimv-spark-extension -A SQL Engine Extension for Spark SQL to support Materialized View +#### 介绍 +Spark物化视图插件,提供自动改写SQL的执行计划替换成物化视图的能力。 -## Introduction +#### 操作系统 +openEuler 22.03 LTS SP1 -Experiments show that there are duplicate subqueries in batch query,and materialized view is a pre-computing and caching -technology to reduce repeated calculation in batch query in database field. -The AI algorithm is used to analyze historical Spark tasks and recommend the optimal materialized view under certain -conditions. -The Spark plugin is used to add materialized view management and execution plan rewriting capabilities, greatly -improving Spark computing efficiency. +#### 使用说明 +用户指南链接: +https://www.hikunpeng.com/document/detail/zh/kunpengbds/appAccelFeatures/sqlqueryaccelf/kunpengbds_omniruntime_20_0403.html -## Environment for building OmniMV - -```shell -# download -wget --no-check-certificate https://mirrors.huaweicloud.com/kunpeng/dist/Apache/hadoop-3.1.1.tar.gz -# unpack -tar -zxvf hadoop-3.1.1.tar.gz -# set environment variables -export HADOOP_HOME="${pwd}/haddoop-3.1.1" -``` - -## Build OmniMV - -pull the OmniMV code and compile it to get the jar package - -```shell -git clone https://gitee.com/kunpengcompute/boostkit-bigdata.git -cd boostkit-bigdata/omnimv/omnimv-spark-extension -# This step can be compiled, tested and packaged to get plugin/boostkit-omnimv-spark-${omniMV.version}.jar -mvn clean package -``` +#### 参与贡献 + 如果您想为本仓库贡献代码,请向本仓库任意maintainer发送邮件 + 如果您找到产品中的任何Bug,欢迎您提出ISSUE -- Gitee