diff --git a/docs/mindformers/docs/source_en/function/weight_conversion.md b/docs/mindformers/docs/source_en/function/weight_conversion.md index f4179098c6beea2a0b24f3bffb730210934a5712..dc247397558a306738a09ed904a0bd048f483603 100644 --- a/docs/mindformers/docs/source_en/function/weight_conversion.md +++ b/docs/mindformers/docs/source_en/function/weight_conversion.md @@ -36,38 +36,28 @@ python convert_weight.py [-h] --model MODEL [--reversed] --input_path INPUT_PATH Assume that you have downloaded the [Llama2 model weight](https://gitee.com/mindspore/mindformers/blob/dev/docs/model_cards/llama2.md#%E6%A8%A1%E5%9E%8B%E6%9D%83%E9%87%8D%E4%B8%8B%E8%BD%BD) and saved it in the `/home/user/torch_weights` path. To convert it to the MindFormers weight and save it in the `/home/user/ms_weights` path, run the following command: ```bash -python convert_weight.py --model llama2 --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt +python convert_weight.py --model llama --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt ``` After the preceding steps are performed, the HuggingFace weight is successfully converted to a MindFormers weight, facilitating model training or inference on MindFormers. ## Supported Models -- Baichuan -- BLIP -- BLOOM -- CodeGeeX2 -- CogVLM2 -- DeepSeek -- GLM -- GLM-n -- GPT -- InternLM -- InternLM2 -- knowlm -- Llama -- MAE -- Mixtral -- Qwen -- Qwen2 -- Qwen2-MoE -- Qwen-VL -- Skywork -- Swin -- TeleChat -- ViT -- WizardCoder -- Yi +| Parameter Value | Supported models | +|-----------|-------------------------------------------| +| llama | Llama2、Llama3、Llama3.1、CodeLlama | +| baichuan2 | Baichuan2 | +| glm-n | GLM2、GLM3、GLM3-32K、GLM4 | +| cogvlm2 | CogVLM2-Video、CogVLM2-Image | +| qwen | Qwen、Qwen1.5、Qwen2 | +| qwenvl | QwenVL | +| internlm | InternLM | +| internlm2 | InternLM2 | +| yi | Yi | +| mixtral | Mixtral | +| deepseek | DeepSeekCoder、DeepSeekCoder1.5、DeepSeekV2 | +| gpt | GPT2 | +| whisper | Whisper | ## Developing Weight Conversion for Unsupported Models diff --git a/docs/mindformers/docs/source_en/usage/inference.md b/docs/mindformers/docs/source_en/usage/inference.md index a855cdaef63944f17a45a1b72afc1fc4e151147d..4186b8c619988faa3a681f33f662659c65277b45 100644 --- a/docs/mindformers/docs/source_en/usage/inference.md +++ b/docs/mindformers/docs/source_en/usage/inference.md @@ -113,7 +113,7 @@ The inference result is as follows: ## Inference Based on the run_mindformer Script -For single-device inference, you can directly run [run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py). For multi-device inference, you need to run [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh). Take Llama2 as an example. You are advised to configure the [predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml) file. +For single-device inference, you can directly run [run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py). For multi-device inference, you need to run [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh). Take Llama2 as an example. You are advised to configure the [predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml) file.During inference, the vocabulary file `tokenizer.model` required for the Llama2 model will be automatically downloaded (ensuring smooth network connectivity). If there is this file locally, you can place it in the `./checkpoint_dewnload/Llama2/` directory in advance. ## Single-Device Inference @@ -128,6 +128,8 @@ python run_mindformer.py \ ## Multi-Device Inference +Executing the script will start the multi card process, and the logs will be redirected to the `./output/msrun_log` directory. Please check the log files in it. When the inference result is printed, it proves that the inference is successful. + ```shell bash scripts/msrun_launcher.sh "python run_mindformer.py \ --config configs/llama2/predict_llama2_7b.yaml \ diff --git a/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md b/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md index 5586bc75aa0458c05451dabcb9f79e90dc5dde9f..15139854bdc813fd0e82f047ea38acf9a1465cf0 100644 --- a/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md +++ b/docs/mindformers/docs/source_zh_cn/function/weight_conversion.md @@ -38,38 +38,28 @@ python convert_weight.py [-h] --model MODEL [--reversed] --input_path INPUT_PATH ，并保存在路径`/home/user/torch_weights`中，用户希望将其转换为MindFormers权重并保存在路径`/home/user/ms_weights`中，可以使用以下命令： ```bash -python convert_weight.py --model llama2 --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt +python convert_weight.py --model llama --input_path /home/user/torch_weights --output_path /home/user/ms_weights/llama.ckpt ``` 通过以上步骤，可将HuggingFace权重成功转换为MindFormers权重，方便在MindFormers中继续模型训练或推理。 ## 已支持模型 -- Baichuan -- BLIP -- BLOOM -- CodeGeeX2 -- CogVLM2 -- DeepSeek -- GLM -- GLM-n -- GPT -- InternLM -- InternLM2 -- knowlm -- Llama -- MAE -- Mixtral -- Qwen -- Qwen2 -- Qwen2-MoE -- Qwen-VL -- Skywork -- Swin -- TeleChat -- ViT -- WizardCoder -- Yi +| 参数取值 | 支持模型 | +|-----------|-------------------------------------------| +| llama | Llama2、Llama3、Llama3.1、CodeLlama | +| baichuan2 | Baichuan2 | +| glm-n | GLM2、GLM3、GLM3-32K、GLM4 | +| cogvlm2 | CogVLM2-Video、CogVLM2-Image | +| qwen | Qwen、Qwen1.5、Qwen2 | +| qwenvl | QwenVL | +| internlm | InternLM | +| internlm2 | InternLM2 | +| yi | Yi | +| mixtral | Mixtral | +| deepseek | DeepSeekCoder、DeepSeekCoder1.5、DeepSeekV2 | +| gpt | GPT2 | +| whisper | Whisper | ## 未支持模型权重转换开发 diff --git a/docs/mindformers/docs/source_zh_cn/usage/inference.md b/docs/mindformers/docs/source_zh_cn/usage/inference.md index d6aa0a104a071933fbe8869a77f76f81180b6d41..a56b364e4a6fc8bb67ef8a2410a6c809e7369e44 100644 --- a/docs/mindformers/docs/source_zh_cn/usage/inference.md +++ b/docs/mindformers/docs/source_zh_cn/usage/inference.md @@ -113,7 +113,7 @@ python pipeline_inference.py ## 基于 run_mindformer 脚本推理 -单卡推理可以直接执行[run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py)，多卡推理需要借助 [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh) 启动。以 Llama2 为例，推荐配置为[predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml)文件。 +单卡推理可以直接执行[run_mindformer.py](https://gitee.com/mindspore/mindformers/blob/dev/run_mindformer.py)，多卡推理需要借助 [scripts/msrun_launcher.sh](https://gitee.com/mindspore/mindformers/blob/dev/scripts/msrun_launcher.sh) 启动。以 Llama2 为例，推荐配置为[predict_llama2_7b.yaml](https://gitee.com/mindspore/mindformers/blob/dev/configs/llama2/predict_llama2_7b.yaml)文件。推理时会自动下载Llama2模型所需词表文件 `tokenizer.model` （需要保障网络畅通）。如果本地有这个文件，可以提前把它放在 `./checkpoint_download/llama2/` 目录下。 ## 单卡推理 @@ -128,6 +128,8 @@ python run_mindformer.py \ ## 多卡推理 +执行脚本会拉起多卡进程，日志会重定向至 `./output/msrun_log` 下，请查看其中的日志文件。当有推理结果打印，证明推理成功。 + ```shell bash scripts/msrun_launcher.sh "python run_mindformer.py \ --config configs/llama2/predict_llama2_7b.yaml \