[Feature] V1 PD disaggregation, DLLM vllm v0.8.3 · Pull Request !369 · MindSpore/vllm-mindspore

[Feature] V1 PD disaggregation with DLLM
*The V1 PD disaggregation features can only be used with:
-	DLLM patch (adopted some V1 features from newer versions) applied on vLLM v0.8.3;
-	datasystem installed
*but below codes would not cause conflicts if you are using original vLLM v0.8.3 (which does not support any V1 PD disaggregation features)

Modified files:
vllm_mindspore/init.py
-	Register the DLLMDsConnector to KVConnectorFactory;
-	DLLMDsConnector is the connector we use to transfer KV cache.

vllm_mindspore/engine/arg_utils.py:
-	Removed the restriction of using "--kv-transfer-config".
-	If you try to use --kv-transfer-config without applying patch, NameError of some functions (e.g. maybe_save_kv_layer_to_connector ) will be raised in other files as an indication.

vllm_mindspore/model_executor/model_loader/weight_utils.py:
-	Adding enable_tqdm as an optional input parameter to safetensors_weights_iterator.
-	After applying patch, enable_tqdm is inputted when we use safetensors_weights_iterator; while the original vLLM v0.8.3 does not.

vllm_mindspore/model_executor/models/mf_models/deepseek_v3.py:
-	Adding connector_send_kvcache for deepseek_v3 models to send kv cache with DLLM connector for V1 PD disagg.
-	**Function `maybe_save_kv_layer_to_connector` only exists after applying patch. Try import it and except import error if patch is not applied.**

vllm_mindspore/model_executor/models/mf_models/mf_model_base.py:
-	Adding is_decoder_task, is_prefill_task to help determine if the instance should do a prefill/decode task.
-	Adding connector_send_kvcache, and connector_wait_for_kv_layer for models in general to send/receive kv cache with DLLM connector for V1 PD disagg. The transfer is doing in layer-wise.
-	Using function is_v1_kv_transfer_group (only exists after applying patch) to help determine if the instance is doing a V1 PD disagg task.
-	**Try import relevant packages and except import error if patch is not applied. `kv_transfer_supported` helps avoid calling related functions.**

vllm_mindspore/v1/core/sched/scheduler.py:
-	Updates the num_computed_tokens information by adding (including) the “num_external_tokens”, which is the number of tokens that been computed from the kv cache in connector.

vllm_mindspore/v1/worker/gpu_worker.py:
-	If we input kv_transfer_config (using PD disagg features), we put the config with kv_transfer_config to init_worker_distributed_environment instead of parallel_config.
-	This is crucial to handle the rank issue, especially when running with tensor parallel. The actual initialization codes are applied with the patch.

vllm_mindspore/worker/worker.py:
-	ensure_kv_transfer_initialized is no longer used in this file?
-	ensure_kv_transfer_initialized is under vllm.distributed.kv_transfer after patch is applied.

类型	指派人员	状态
审查	TronZhang	已完成 (0/0人)
测试	TronZhang	已完成 (0/0人)

MindSpore/vllm-mindspore .gitee-modal { width: 500px !important; }

搜索帮助

MindSpore/vllm-mindspore