diff --git a/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md b/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md index 5b2dd780f804a2cb0593b8ebba64e31ef0594b17..26739499c0c99f2f29bf766f425ee7edacebb125 100644 --- a/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md +++ b/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md @@ -242,10 +242,10 @@ The Ascend runtime package can be downloaded from the Ascend homepage. For examp Before managing a multi-node cluster, check that the hostnames of all nodes are different. If any are the same, set different hostnames using `hostname `. -1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. For example, in an environment with IP `192.5.5.5`, running `ray start --head --port=6379` will prompt: +1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. Configure as follows, replacing `IP` and `address` with the actual environment information. ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* ------------------- Ray runtime started. @@ -253,7 +253,7 @@ Before managing a multi-node cluster, check that the hostnames of all nodes are Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray diff --git a/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md b/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md index 54ae31c5581623e0265b8fac2f6457fa8651d70c..ea6aeb84953b84fbecc1f64523b587793d950796 100644 --- a/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md +++ b/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md @@ -163,10 +163,10 @@ The Ascend runtime package can be downloaded from the Ascend homepage. For examp Before managing a multi-node cluster, check that the hostnames of all nodes are different. If any are the same, set different hostnames using `hostname `. -1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. For example, in an environment with IP `192.5.5.5`, running `ray start --head --port=6379` will prompt: +1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. Configure as follows, replacing `IP` and `address` with the actual environment information. ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* ------------------- Ray runtime started. @@ -174,7 +174,7 @@ Before managing a multi-node cluster, check that the hostnames of all nodes are Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray @@ -296,12 +296,14 @@ The following are execution examples for the multiprocess and Ray startup method ```bash # Master Node: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' # Worker Node: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' ``` +Specifically, `data-parallel-address` and `--data-parallel-rpc-port` must be configured with the actual environment information for the running instance. + **Ray Startup Method** ```bash diff --git a/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md b/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md index c1a9dccf643855f3ca3fe4b10fa4d70b80228e90..1bda8b8f4ff9f1d4aafb49b5808434a0d0f5e12e 100644 --- a/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md +++ b/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md @@ -242,10 +242,10 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run 多节点集群管理前,需要检查各节点的 hostname 是否各异。如果存在相同的,需要通过 `hostname ` 设置不同的 hostname。 -1. 启动主节点 `ray start --head --port=`。启动成功后,会提示从节点的连接方式。例如,在 IP 为 `192.5.5.5` 的环境中,通过 `ray start --head --port=6379`,提示如下: +1. 启动主节点 `ray start --head --port=`。启动成功后,会提示从节点的连接方式。配置方式如下,其中`IP`和`address`需要修改成实际运行的环境信息。 ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* -------------------- Ray runtime started. @@ -253,7 +253,7 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray diff --git a/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md b/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md index 346e82cecb7596e62f11c0f899940bcb1e3ad7be..5e84bb709b823c46c0e1c7c1fafe3f0b66dac6fd 100644 --- a/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md +++ b/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md @@ -163,10 +163,10 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run 多节点集群管理前,需要检查各节点的 hostname 是否各异。如果存在相同的,需要通过 `hostname ` 设置不同的 hostname。 -1. 启动主节点 `ray start --head --port=`,启动成功后,会提示从节点的连接方式。如在 IP 为 `192.5.5.5` 的环境中,通过 `ray start --head --port=6379`,提示如下: +1. 启动主节点 `ray start --head --port=`,启动成功后,会提示从节点的连接方式。配置方式如下,其中`IP`和`address`需要修改成实际运行的环境信息。 ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* ------------------- Ray runtime started. @@ -174,7 +174,7 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray @@ -297,12 +297,14 @@ vllm-mindspore serve ```bash # 主节点: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' # 从节点: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' ``` +其中,`data-parallel-address`和`--data-parallel-rpc-port`需要设置成实际运行的环境信息。 + **Ray启动方式** ```bash