diff --git a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-70B/README.md b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-70B/README.md index a2fa7b30acf955afd4bd39523d1e3c2ea6c2cae4..a6ffb286629b1957fb47681080efa9158e3f264c 100644 --- a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-70B/README.md +++ b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-70B/README.md @@ -101,11 +101,17 @@ python3 quant_llama.py --model_path {浮点权重路径} --save_directory {W8A8 ```shell cd $ATB_SPEED_HOME_PATH +# 如果是单独安装分包,请手动source环境变量,路径为安装路径 +source /usr/local/Ascend/ascend-toolkit/set_env.sh +source /usr/local/Ascend/nnal/atb/set_env.sh +source /usr/local/Ascend/atb-models/set_env.sh +source /usr/local/Ascend/mindie/set_env.sh ``` 执行对话测试 ```shell +export MINDIE_LOG_TO_STDOUT=1 torchrun --nproc_per_node 8 \ --master_port 20037 \ -m examples.run_pa \ @@ -118,6 +124,7 @@ torchrun --nproc_per_node 8 \ 进入ModelTest路径 ```shell cd $ATB_SPEED_HOME_PATH/tests/modeltest/ +export MINDIE_LOG_TO_STDOUT=1 ``` 运行测试脚本 ```shell @@ -142,6 +149,7 @@ vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json - 更改配置文件 +具体配置信息请参考[MindIE Service用户指南](https://www.hiascend.com/document/detail/zh/mindie/20RC2/mindieservice/servicedev/mindie_service0285.html) ```json { ... diff --git a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-8B/README.md b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-8B/README.md index 215e7f25fbaa71caf016c8d65fa8a13f16ec917f..a262d2975177f20c0570eb7f5a585cc1f2a01d00 100644 --- a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-8B/README.md +++ b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Llama-8B/README.md @@ -135,11 +135,17 @@ torchrun --nproc_per_node 2 -m examples.convert.model_slim.sparse_compressor --m ```shell cd $ATB_SPEED_HOME_PATH +# 如果是单独安装分包,请手动source环境变量,路径为安装路径 +source /usr/local/Ascend/ascend-toolkit/set_env.sh +source /usr/local/Ascend/nnal/atb/set_env.sh +source /usr/local/Ascend/atb-models/set_env.sh +source /usr/local/Ascend/mindie/set_env.sh ``` 执行对话测试 ```shell +export MINDIE_LOG_TO_STDOUT=1 torchrun --nproc_per_node 2 \ --master_port 20037 \ -m examples.run_pa \ @@ -152,6 +158,7 @@ torchrun --nproc_per_node 2 \ 进入ModelTest路径 ```shell cd $ATB_SPEED_HOME_PATH/tests/modeltest/ +export MINDIE_LOG_TO_STDOUT=1 ``` 运行测试脚本 ```shell @@ -181,6 +188,7 @@ vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json - 更改配置文件 +具体配置信息请参考[MindIE Service用户指南](https://www.hiascend.com/document/detail/zh/mindie/20RC2/mindieservice/servicedev/mindie_service0285.html) ```json { ... diff --git a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-1.5B/README.md b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-1.5B/README.md index c76608e744d1c75ba2565a7c0ebf979f1ddb3b5a..cc7e7c0c67d3987fc469e8eda018737e2e867e83 100644 --- a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-1.5B/README.md +++ b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-1.5B/README.md @@ -145,11 +145,17 @@ export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False ```shell cd $ATB_SPEED_HOME_PATH +# 如果是单独安装分包,请手动source环境变量,路径为安装路径 +source /usr/local/Ascend/ascend-toolkit/set_env.sh +source /usr/local/Ascend/nnal/atb/set_env.sh +source /usr/local/Ascend/atb-models/set_env.sh +source /usr/local/Ascend/mindie/set_env.sh ``` 执行对话测试 ```shell +export MINDIE_LOG_TO_STDOUT=1 torchrun --nproc_per_node 2 \ --master_port 20037 \ -m examples.run_pa \ @@ -161,6 +167,7 @@ torchrun --nproc_per_node 2 \ 进入ModelTest路径 ```shell cd $ATB_SPEED_HOME_PATH/tests/modeltest/ +export MINDIE_LOG_TO_STDOUT=1 ``` 运行测试脚本 ```shell @@ -183,6 +190,7 @@ vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json - 更改配置文件 +具体配置信息请参考[MindIE Service用户指南](https://www.hiascend.com/document/detail/zh/mindie/20RC2/mindieservice/servicedev/mindie_service0285.html) ```json { ... diff --git a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-14B/README.md b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-14B/README.md index 7de52872b06cfd498a533b398b24ffd482d79a41..657980c9314b7d6bc86dcd9b415f5983604e9b91 100644 --- a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-14B/README.md +++ b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-14B/README.md @@ -148,11 +148,17 @@ export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False ```shell cd $ATB_SPEED_HOME_PATH +# 如果是单独安装分包,请手动source环境变量,路径为安装路径 +source /usr/local/Ascend/ascend-toolkit/set_env.sh +source /usr/local/Ascend/nnal/atb/set_env.sh +source /usr/local/Ascend/atb-models/set_env.sh +source /usr/local/Ascend/mindie/set_env.sh ``` 执行对话测试 ```shell +export MINDIE_LOG_TO_STDOUT=1 torchrun --nproc_per_node 2 \ --master_port 20037 \ -m examples.run_pa \ @@ -164,6 +170,7 @@ torchrun --nproc_per_node 2 \ 进入ModelTest路径 ```shell cd $ATB_SPEED_HOME_PATH/tests/modeltest/ +export MINDIE_LOG_TO_STDOUT=1 ``` 运行测试脚本 ```shell @@ -186,6 +193,7 @@ vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json - 更改配置文件 +具体配置信息请参考[MindIE Service用户指南](https://www.hiascend.com/document/detail/zh/mindie/20RC2/mindieservice/servicedev/mindie_service0285.html) ```json { ... diff --git a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-32B/README.md b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-32B/README.md index 7481e73fc0a22d682e51ed1059fb4b4314a0b4e8..afe4a400dac7c731e967f64201ea284ec6a9ca31 100644 --- a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-32B/README.md +++ b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-32B/README.md @@ -121,11 +121,17 @@ export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False ```shell cd $ATB_SPEED_HOME_PATH +# 如果是单独安装分包,请手动source环境变量,路径为安装路径 +source /usr/local/Ascend/ascend-toolkit/set_env.sh +source /usr/local/Ascend/nnal/atb/set_env.sh +source /usr/local/Ascend/atb-models/set_env.sh +source /usr/local/Ascend/mindie/set_env.sh ``` 执行对话测试 -```shell +``` +export MINDIE_LOG_TO_STDOUT=1 torchrun --nproc_per_node 4 \ --master_port 20037 \ -m examples.run_pa \ @@ -137,6 +143,7 @@ torchrun --nproc_per_node 4 \ 进入ModelTest路径 ```shell cd $ATB_SPEED_HOME_PATH/tests/modeltest/ +export MINDIE_LOG_TO_STDOUT=1 ``` 运行测试脚本 ```shell @@ -159,6 +166,7 @@ vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json - 更改配置文件 +具体配置信息请参考[MindIE Service用户指南](https://www.hiascend.com/document/detail/zh/mindie/20RC2/mindieservice/servicedev/mindie_service0285.html) ```json { ... diff --git a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-7B/README.md b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-7B/README.md index 3b44be8c74448dd6e120d1a0bf490f535bf84f29..fb573b7d917496548ba4634c94dd13e16bf514bc 100644 --- a/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-7B/README.md +++ b/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-7B/README.md @@ -144,11 +144,17 @@ export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False ```shell cd $ATB_SPEED_HOME_PATH +# 如果是单独安装分包,请手动source环境变量,路径为安装路径 +source /usr/local/Ascend/ascend-toolkit/set_env.sh +source /usr/local/Ascend/nnal/atb/set_env.sh +source /usr/local/Ascend/atb-models/set_env.sh +source /usr/local/Ascend/mindie/set_env.sh ``` 执行对话测试 ```shell +export MINDIE_LOG_TO_STDOUT=1 torchrun --nproc_per_node 2 \ --master_port 20037 \ -m examples.run_pa \ @@ -160,6 +166,7 @@ torchrun --nproc_per_node 2 \ 进入ModelTest路径 ```shell cd $ATB_SPEED_HOME_PATH/tests/modeltest/ +export MINDIE_LOG_TO_STDOUT=1 ``` 运行测试脚本 ```shell @@ -182,6 +189,7 @@ vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json - 更改配置文件 +具体配置信息请参考[MindIE Service用户指南](https://www.hiascend.com/document/detail/zh/mindie/20RC2/mindieservice/servicedev/mindie_service0285.html) ```json { ...