diff --git a/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md b/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md index cb7a0d2945a2deb3ed60a83d36fd8a89a8477e66..0a786c659c0ed5614188466ebb25e5909ea71b65 100644 --- a/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md +++ b/docs/vllm_mindspore/docs/source_en/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md @@ -242,10 +242,10 @@ The Ascend runtime package can be downloaded from the Ascend homepage. For examp Before managing a multi-node cluster, check that the hostnames of all nodes are different. If any are the same, set different hostnames using `hostname `. -1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. For example, in an environment with IP `192.5.5.5`, running `ray start --head --port=6379` will prompt: +1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. Configure as follows, replacing `IP` and `address` with the actual environment information. ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* ------------------- Ray runtime started. @@ -253,7 +253,7 @@ Before managing a multi-node cluster, check that the hostnames of all nodes are Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray diff --git a/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md b/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md index f734fb1fe2b6953089ff705b3c635239453c01bd..7d529f196cbb33e3968cc6c4f7f736a37d096000 100644 --- a/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md +++ b/docs/vllm_mindspore/docs/source_en/user_guide/supported_features/parallel/parallel.md @@ -163,10 +163,10 @@ The Ascend runtime package can be downloaded from the Ascend homepage. For examp Before managing a multi-node cluster, check that the hostnames of all nodes are different. If any are the same, set different hostnames using `hostname `. -1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. For example, in an environment with IP `192.5.5.5`, running `ray start --head --port=6379` will prompt: +1. Start the head node: `ray start --head --port=`. Upon successful startup, the connection method for worker nodes will be displayed. Configure as follows, replacing `IP` and `address` with the actual environment information. ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* ------------------- Ray runtime started. @@ -174,7 +174,7 @@ Before managing a multi-node cluster, check that the hostnames of all nodes are Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray @@ -296,12 +296,14 @@ The following are execution examples for the multiprocess and Ray startup method ```bash # Master Node: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' # Worker Node: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' ``` +Specifically, `data-parallel-address` and `--data-parallel-rpc-port` must be configured with the actual environment information for the running instance. + **Ray Startup Method** ```bash diff --git a/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md b/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md index cd7967a4bf429384ccfe02aa2a71672972bdaf29..8f509ca66d242d6eab1010f7d8a246be7c4e000d 100644 --- a/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md +++ b/docs/vllm_mindspore/docs/source_zh_cn/getting_started/tutorials/deepseek_parallel/deepseek_r1_671b_w8a8_dp4_tp4_ep4.md @@ -242,10 +242,10 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run 多节点集群管理前,需要检查各节点的 hostname 是否各异。如果存在相同的,需要通过 `hostname ` 设置不同的 hostname。 -1. 启动主节点 `ray start --head --port=`。启动成功后,会提示从节点的连接方式。例如,在 IP 为 `192.5.5.5` 的环境中,通过 `ray start --head --port=6379`,提示如下: +1. 启动主节点 `ray start --head --port=`。启动成功后,会提示从节点的连接方式。配置方式如下,其中`IP`和`address`需要修改成实际运行的环境信息。 ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* -------------------- Ray runtime started. @@ -253,7 +253,7 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray diff --git a/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md b/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md index 9c652b1e1e6e01b3bdddef2d9f97fdde8888f63e..359cf45f81786b3294c8fb1503fc8a5f3954fa3b 100644 --- a/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md +++ b/docs/vllm_mindspore/docs/source_zh_cn/user_guide/supported_features/parallel/parallel.md @@ -163,10 +163,10 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run 多节点集群管理前,需要检查各节点的 hostname 是否各异。如果存在相同的,需要通过 `hostname ` 设置不同的 hostname。 -1. 启动主节点 `ray start --head --port=`,启动成功后,会提示从节点的连接方式。如在 IP 为 `192.5.5.5` 的环境中,通过 `ray start --head --port=6379`,提示如下: +1. 启动主节点 `ray start --head --port=`,启动成功后,会提示从节点的连接方式。配置方式如下,其中`IP`和`address`需要修改成实际运行的环境信息。 ```text - Local node IP: 192.5.5.5 + Local node IP: *.*.*.* ------------------- Ray runtime started. @@ -174,7 +174,7 @@ chmod -R 777 ./Ascend-pyACL_8.0.RC1_linux-aarch64.run Next steps To add another node to this Ray cluster, run - ray start --address='192.5.5.5:6379' + ray start --address='*.*.*.*:*' To connect to this Ray cluster: import ray @@ -297,12 +297,14 @@ vllm-mindspore serve ```bash # 主节点: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 0 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' # 从节点: -vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 192.10.10.10 --data-parallel-rpc-port 12370 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' +vllm-mindspore serve MindSpore-Lab/DeepSeek-R1-0528-A8W8 --headless --trust-remote-code --max-num-seqs=256 --max-model-len=32768 --max-num-batched-tokens=4096 --block-size=128 --gpu-memory-utilization=0.9 --tensor-parallel-size 4 --data-parallel-size 4 --data-parallel-size-local 2 --data-parallel-start-rank 2 --data-parallel-address 127.0.0.1 --data-parallel-rpc-port 29550 --enable-expert-parallel --addition-config '{"data_parallel": 4, "model_parallel": 4, "expert_parallel": 4}' ``` +其中,`data-parallel-address`和`--data-parallel-rpc-port`需要设置成实际运行的环境信息。 + **Ray启动方式** ```bash