From 330d7856a0e7155ecf75d20890461e4e9cfbe4b9 Mon Sep 17 00:00:00 2001 From: smallsilly <1045916357@qq.com> Date: Tue, 15 Oct 2024 11:48:39 +0800 Subject: [PATCH] =?UTF-8?q?=E3=80=90=E9=97=AE=E9=A2=98=E5=8D=95=E4=BF=AE?= =?UTF-8?q?=E5=A4=8D=E3=80=91=E4=BF=AE=E5=A4=8D=E4=BD=BF=E7=94=A8MindForme?= =?UTF-8?q?rs=E6=A1=86=E6=9E=B6=E6=8E=A8=E7=90=86=E5=A4=A7=E6=A8=A1?= =?UTF-8?q?=E5=9E=8B=EF=BC=88llama2-70b=EF=BC=89=E6=B5=81=E7=A8=8B?= =?UTF-8?q?=E5=AD=98=E5=9C=A8=E7=9A=84=E9=83=A8=E5=88=86=E8=B5=84=E6=96=99?= =?UTF-8?q?=E9=97=AE=E9=A2=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../source_en/function/weight_conversion.md | 42 +++++++------------ .../docs/source_en/usage/inference.md | 4 +- .../function/weight_conversion.md | 42 +++++++------------ .../docs/source_zh_cn/usage/inference.md | 4 +- 4 files changed, 38 insertions(+), 54 deletions(-) diff --git a/docs/mindformers/docs/source_en/function/weight_conversion.md b/docs/mindformers/docs/source_en/function/weight_conversion.md index f4179098c6..dc24739755 100644 --- a/docs/mindformers/docs/source_en/function/weight_conversion.md +++ b/docs/mindformers/docs/source_en/function/weight_conversion.md @@ -36,38 +36,28 @@ python convert_weight.py [-h] --model MODEL [--reversed] --input_path INPUT_PATH Assume that you have downloaded the [Llama2 model weight](https://gitee.com/mindspore/mindformers/blob/dev/docs/model_cards/llama2.md#%E6%A8%A1%E5%9E%8B%E6%9D%83%E9%87%8D%E4%B8%8B%E8%BD%BD) and saved it in the `/home/user/torch_weights` path. To convert it to the MindFormers weight and save it in the `/home/user/ms_weights` path, run the following command: ```bash -python convert_weight.py --model llama2 --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt +python convert_weight.py --model llama --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt ``` After the preceding steps are performed, the HuggingFace weight is successfully converted to a MindFormers weight, facilitating model training or inference on MindFormers. ## Supported Models -- Baichuan -- BLIP -- BLOOM -- CodeGeeX2 -- CogVLM2 -- DeepSeek -- GLM -- GLM-n -- GPT -- InternLM -- InternLM2 -- knowlm -- Llama -- MAE -- Mixtral -- Qwen -- Qwen2 -- Qwen2-MoE -- Qwen-VL -- Skywork -- Swin -- TeleChat -- ViT -- WizardCoder -- Yi +| Parameter Value | Supported models | +|-----------|-------------------------------------------| +| llama | Llama2、Llama3、Llama3.1、CodeLlama | +| baichuan2 | Baichuan2 | +| glm-n | GLM2、GLM3、GLM3-32K、GLM4 | +| cogvlm2 | CogVLM2-Video、CogVLM2-Image | +| qwen | Qwen、Qwen1.5、Qwen2 | +| qwenvl | QwenVL | +| internlm | InternLM | +| internlm2 | InternLM2 | +| yi | Yi | +| mixtral | Mixtral | +| deepseek | DeepSeekCoder、DeepSeekCoder1.5、DeepSeekV2 | +| gpt | GPT2 | +| whisper | Whisper | ## Developing Weight Conversion for Unsupported Models diff --git a/docs/mindformers/docs/source_en/usage/inference.md b/docs/mindformers/docs/source_en/usage/inference.md index a855cdaef6..4186b8c619 100644 --- a/docs/mindformers/docs/source_en/usage/inference.md +++ b/docs/mindformers/docs/source_en/usage/inference.md @@ -113,7 +113,7 @@ The inference result is as follows: ## Inference Based on the run_mindformer Script -For single-device inference, you can directly run [run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py). For multi-device inference, you need to run [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh). Take Llama2 as an example. You are advised to configure the [predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml) file. +For single-device inference, you can directly run [run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py). For multi-device inference, you need to run [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh). Take Llama2 as an example. You are advised to configure the [predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml) file.During inference, the vocabulary file `tokenizer.model` required for the Llama2 model will be automatically downloaded (ensuring smooth network connectivity). If there is this file locally, you can place it in the `./checkpoint_dewnload/Llama2/` directory in advance. ## Single-Device Inference @@ -128,6 +128,8 @@ python run_mindformer.py \ ## Multi-Device Inference +Executing the script will start the multi card process, and the logs will be redirected to the `./output/msrun_log` directory. Please check the log files in it. When the inference result is printed, it proves that the inference is successful. + ```shell bash scripts/msrun_launcher.sh "python run_mindformer.py \ --config configs/llama2/predict_llama2_7b.yaml \ diff --git a/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md b/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md index 5586bc75aa..15139854bd 100644 --- a/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md +++ b/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md @@ -38,38 +38,28 @@ python convert_weight.py [-h] --model MODEL [--reversed] --input_path INPUT_PATH ，并保存在路径`/home/user/torch_weights`中，用户希望将其转换为MindFormers权重并保存在路径`/home/user/ms_weights`中，可以使用以下命令： ```bash -python convert_weight.py --model llama2 --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt +python convert_weight.py --model llama --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt ``` 通过以上步骤，可将HuggingFace权重成功转换为MindFormers权重，方便在MindFormers中继续模型训练或推理。 ## 已支持模型 -- Baichuan -- BLIP -- BLOOM -- CodeGeeX2 -- CogVLM2 -- DeepSeek -- GLM -- GLM-n -- GPT -- InternLM -- InternLM2 -- knowlm -- Llama -- MAE -- Mixtral -- Qwen -- Qwen2 -- Qwen2-MoE -- Qwen-VL -- Skywork -- Swin -- TeleChat -- ViT -- WizardCoder -- Yi +| 参数取值 | 支持模型 | +|-----------|-------------------------------------------| +| llama | Llama2、Llama3、Llama3.1、CodeLlama | +| baichuan2 | Baichuan2 | +| glm-n | GLM2、GLM3、GLM3-32K、GLM4 | +| cogvlm2 | CogVLM2-Video、CogVLM2-Image | +| qwen | Qwen、Qwen1.5、Qwen2 | +| qwenvl | QwenVL | +| internlm | InternLM | +| internlm2 | InternLM2 | +| yi | Yi | +| mixtral | Mixtral | +| deepseek | DeepSeekCoder、DeepSeekCoder1.5、DeepSeekV2 | +| gpt | GPT2 | +| whisper | Whisper | ## 未支持模型权重转换开发 diff --git a/docs/mindformers/docs/source_zh_cn/usage/inference.md b/docs/mindformers/docs/source_zh_cn/usage/inference.md index d6aa0a104a..a56b364e4a 100644 --- a/docs/mindformers/docs/source_zh_cn/usage/inference.md +++ b/docs/mindformers/docs/source_zh_cn/usage/inference.md @@ -113,7 +113,7 @@ python pipeline_inference.py ## 基于 run_mindformer 脚本推理 -单卡推理可以直接执行[run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py)，多卡推理需要借助 [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh) 启动。以 Llama2 为例，推荐配置为[predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml)文件。 +单卡推理可以直接执行[run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py)，多卡推理需要借助 [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh) 启动。以 Llama2 为例，推荐配置为[predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml)文件。推理时会自动下载Llama2模型所需词表文件 `tokenizer.model` （需要保障网络畅通）。如果本地有这个文件，可以提前把它放在 `./checkpoint_download/llama2/` 目录下。 ## 单卡推理 @@ -128,6 +128,8 @@ python run_mindformer.py \ ## 多卡推理 +执行脚本会拉起多卡进程，日志会重定向至 `./output/msrun_log` 下，请查看其中的日志文件。当有推理结果打印，证明推理成功。 + ```shell bash scripts/msrun_launcher.sh "python run_mindformer.py \ --config configs/llama2/predict_llama2_7b.yaml \ -- Gitee