diff --git a/PyTorch/built-in/foundation/ChatGLM2-6B/README.md b/PyTorch/built-in/foundation/ChatGLM2-6B/README.md index aef72c5b0aa6e8c39b46f6664a9708423df3096d..e472e67dbf0566f2e8846752281e0dfb673cd8c9 100644 --- a/PyTorch/built-in/foundation/ChatGLM2-6B/README.md +++ b/PyTorch/built-in/foundation/ChatGLM2-6B/README.md @@ -1,3 +1,5 @@ + + # Contents - [ChatGLM2-6B](#contents) @@ -32,7 +34,7 @@ Here's a software summary of pre-training ChatGLM2-6B: | CANN | Ascend-cann-toolkit_7.0.RC1.1_linux | [link](https://support.huawei.com/carriersoftwareapply/softwareApp?nid=PSW2001680585&pbi=PBI1-261422362&swlang=zh&supPbi=PBI1-251168373&path=PBI1-21430725/PBI1-21430756/PBI1-22892969/PBI1-23710427/PBI1-251168373) | | binary arithmetic package | Ascend-cann-kernels-910b_7.0.RC1.1_linux | [link](https://support.huawei.com/carriersoftwareapply/softwareApp?nid=PSW2001680596&pbi=PBI1-261422362&swlang=zh&supPbi=PBI1-251168373&path=PBI1-21430725/PBI1-21430756/PBI1-22892969/PBI1-23710427/PBI1-251168373) | | torch | 1.11.0 | [link](https://pytorch-package.obs.cn-north-4.myhuaweicloud.com/pta/Daily/v1.11.0/20230915.2/pytorch_v1.11.0_py37.tar.gz) | -| torch_npu | 1.11.0.post5 | [link](https://gitee.com/ascend/pytorch/archive/refs/tags/v5.0.rc3.1-pytorch1.11.0.tar.gz) | +| torch_npu | 1.11.0.post4 | [link](https://gitee.com/ascend/pytorch/releases/download/v5.0.rc3-pytorch1.11.0/torch_npu-1.11.0.post4-cp37-cp37m-linux_aarch64.whl) | ### Script @@ -71,11 +73,12 @@ pip install -r requirements.txt # cp fix/utils.py /root/miniconda3/envs/conda环境名/lib/python3.7/site-packages/transformers/generation/ ``` 3. Prepare pretrained weights - 1)Download the ChatGLM2-6B checkpoint from [here]([THUDM/chatglm2-6b at v1.0 (huggingface.co)](https://huggingface.co/THUDM/chatglm2-6b/tree/v1.0)) ; After downloading, place it in the "model" directory . - 2)Please Do NOT overwrite modeling_chatglm.py + 1)Download the ChatGLM2-6B checkpoint from [here]([THUDM/chatglm2-6b at v1.0 (huggingface.co)](https://huggingface.co/THUDM/chatglm2-6b/tree/dev)) ; After downloading, place it in the "model" directory . - The "model" directory is as follows + 2)Please Do NOT overwrite modeling_chatglm.py + + The "model" directory is as follows ```shell ├── model @@ -153,9 +156,9 @@ source env_npu.sh - 全参数finetune， 启动8卡微调。 -```shell -bash ds_train_fintune.sh -``` + ```shell + bash ds_train_finetune.sh + ``` - P-Tuning v2 @@ -170,7 +173,7 @@ bash ds_train_fintune.sh ``` cd /${模型文件夹名称}/ptuning - bash evaluate_fintune.sh + bash evaluate_finetune.sh ``` ### Performance @@ -179,10 +182,10 @@ bash ds_train_fintune.sh The performance of ChatGLM2-6B in **Ascend NPU** and **Reference**: -| Device | Model | total Iterations | throughput rate (samples/s/p) | throughput rate (tokens/s/p) | single-step time (s/step) | floating point operation (TFLOPs/s) | -| --------- | ----------- | ---------------- | ----------------------------- | ---------------------------- | ------------------------- | ----------------------------------- | -| NPUs | ChatGLM2-6B | 1000 | 待补充 | 1927 | 4.25 | 待补充 | -| Reference | ChatGLM2-6B | 1000 | 待补充 | 1820 | 4.5 | 待补充 | +| Device | Model | total Iterations | throughput rate (tokens/s/p) | single-step time (s/step) | +| --------- | ----------- | ---------------- | ---------------------------- | ------------------------- | +| NPUs | ChatGLM2-6B | 1000 | 1927 | 4.25 | +| Reference | ChatGLM2-6B | 1000 | 1820 | 4.5 | 评估结果展示表 @@ -254,7 +257,7 @@ The NPU runs smoothly, the resource usage is stable, no errors are reported in t `cp fix/utils.py /root/miniconda3/envs/conda环境名/lib/python3.7/site-packages/transformers/generation/` ``` -1. 微调时出现AttributeError或RuntimeError +6. 微调时出现AttributeError或RuntimeError module 'torch_npu' has no attribute 'npu_rotary_mul' 或 @@ -268,7 +271,7 @@ The NPU runs smoothly, the resource usage is stable, no errors are reported in t PS: 设置为True能提高性能 -2. 如果cann不支持flash_attention +7. 如果cann不支持flash_attention 报错提示为module 'torch_npu' has no attribute 'npu_flash_attention' @@ -279,6 +282,9 @@ USE_FLASH=False ​ PS: 设置为True能提高性能 -# 公网地址说明 +​ 8.预训练 + +​ Chatglm官方不支持预训练，只提供微调，推理功能； + +​ 抱歉也暂时没有适配cli_demo, web_demo程序， 如客户需要，可以向我们提需求； -代码涉及公网地址参考 public_address_statement.md \ No newline at end of file diff --git a/PyTorch/built-in/foundation/ChatGLM2-6B/images/ChatGLM2-6B_loss_compare.png b/PyTorch/built-in/foundation/ChatGLM2-6B/images/ChatGLM2-6B_loss_compare.png new file mode 100644 index 0000000000000000000000000000000000000000..e47384db4d985def5e40055bc92190d8d8b32e15 Binary files /dev/null and b/PyTorch/built-in/foundation/ChatGLM2-6B/images/ChatGLM2-6B_loss_compare.png differ diff --git a/PyTorch/built-in/foundation/ChatGLM2-6B/ptuning/preprocess.sh b/PyTorch/built-in/foundation/ChatGLM2-6B/ptuning/preprocess.sh index 5908367e99e6760b602d1ab0675f9458797e71f5..78698c0619f00bc27755683115d194241c0a20d1 100644 --- a/PyTorch/built-in/foundation/ChatGLM2-6B/ptuning/preprocess.sh +++ b/PyTorch/built-in/foundation/ChatGLM2-6B/ptuning/preprocess.sh @@ -1,3 +1,4 @@ +set -e source env_npu.sh python preprocess.py \ --do_train \ @@ -8,18 +9,18 @@ python preprocess.py \ --model_name_or_path ../model/ \ --overwrite_cache \ --output_dir ./output/adgen-chatglm-6b-ft-$LR \ - --max_source_length 1024 \ - --max_target_length 1024 + --max_source_length 4096 \ + --max_target_length 4096 -#python preprocess.py \ -# --do_predict \ -# --train_file AdvertiseGen/train.json \ -# --test_file AdvertiseGen/dev.json \ -# --prompt_column content \ -# --response_column summary \ -# --model_name_or_path ../model/ \ -# --overwrite_cache \ -# --output_dir ./output/adgen-chatglm-6b-ft-$LR \ -# --max_source_length 256 \ -# --max_target_length 256 +python preprocess.py \ + --do_predict \ + --train_file AdvertiseGen/train.json \ + --test_file AdvertiseGen/dev.json \ + --prompt_column content \ + --response_column summary \ + --model_name_or_path ../model/ \ + --overwrite_cache \ + --output_dir ./output/adgen-chatglm-6b-ft-$LR \ + --max_source_length 256 \ + --max_target_length 256