modelscope / modelscope/ms-swift

ms-swift4.0.1+transformer 5.2.0微调qwen3-vl-30b-a3b，部署量化模型时IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

Open
#8,329 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question stale
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Checklist / 检查清单
  • I have searched existing issues, and this is a new question or discussion topic. / 我已经搜索过现有的 issues，确认这是一个新的问题与讨论。
Question Description / 问题描述

问题描述

使用megatron 微调 qwen3-vl-30b-a3b后，再使用swift export导出fp8模型，部署时报错。

环境：

torch                         2.10.0+cu130
transformers                  5.2.0
triton                        3.6.0
trl                           0.28.0
ms_swift                      4.0.1
megatron-core                 0.15.3
flash_attn                    2.8.3

megatron sft 命令

# torch 2.10+cu130+swift4
export CUDA_HOME="/usr/local/cuda"
# MEGATRON_LM_PATH="/home/jovyan/wangdonghua/code/Megatron-LM-r15" \
PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True' \
IMAGE_MAX_TOKEN_NUM=1024 \
VIDEO_MAX_TOKEN_NUM=128 \
FPS_MAX_FRAMES=16 \
MASTER_PORT=29500 \
NPROC_PER_NODE=2 \
CUDA_VISIBLE_DEVICES=1,6 \
megatron sft \
    --model /mnt/public_data/modelscope/Qwen/Qwen3-VL-30B-A3B-Instruct_ \
    --save_safetensors true \
    --model_type qwen3_vl_moe \
    --dataset /home/jovyan/wdh/llama-factory/datas/ms-swift/train_data_260116.jsonl \
    --loss_scale default \
    --loss_scale ignore_empty_think \
    --tuner_type lora \
    --freeze_vit false \
    --freeze_aligner false \
    --vit_gradient_checkpointing true \
    --lora_rank 8 \
    --lora_alpha 32 \
    --target_modules all-linear \
    --tensor_model_parallel_size 2 \
    --expert_model_parallel_size 2 \
    --moe_permute_fusion true \
    --moe_grouped_gemm true \
    --moe_shared_expert_overlap true \
    --moe_aux_loss_coeff 1e-6 \
    --micro_batch_size 1 \
    --global_batch_size 4 \
    --recompute_granularity full \
    --recompute_method uniform \
    --recompute_num_layers 1 \
    --num_train_epochs 6 \
    --finetune true \
    --cross_entropy_loss_fusion true \
    --lr 1e-4 \
    --vit_lr 5e-5 \
    --aligner_lr 5e-5 \
    --lr_warmup_fraction 0.05 \
    --min_lr 1e-5 \
    --output_dir /home/jovyan/wdh/llama-factory/models/finetune/Qwen3-VL-30B-A3B-Instruct-0312-finetune-merged \
    --eval_steps 500 \
    --save_steps 500 \
    --max_length 16384 \
    --dataloader_num_workers 8 \
    --dataset_num_proc 8 \
    --no_save_optim true \
    --no_save_rng true \
    --sequence_parallel true \
    --attention-backend flash

vllm 命令

CUDA_VISIBLE_DEVICES=1,6 vllm serve \ /home/jovyan/wdh/llama-factory/models/quantization/Qwen3-VL-30B-A3B-Instruct-0312-finetune-merged-step3500 --served-model-name Qwen3-VL \
--tensor-parallel-size 2 \
--gpu-memory-utilization 0.85 \
--max-model-len 8096 \
--port 30005

报错信息

(APIServer pid=59342) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=59342) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=59342) INFO 03-13 15:59:14 [model.py:531] Resolved architecture: Qwen3VLMoeForConditionalGeneration
(APIServer pid=59342) INFO 03-13 15:59:14 [model.py:1554] Using max model len 16384
(APIServer pid=59342) INFO 03-13 16:00:04 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=8192.
(APIServer pid=59342) INFO 03-13 16:00:04 [vllm.py:747] Asynchronous scheduling is enabled.
(EngineCore_DP0 pid=68432) INFO 03-13 16:08:48 [core.py:101] Initializing a V1 LLM engine (v0.17.0) with config: model='/home/jovyan/wdh/llama-factory/models/quantization/Qwen3-VL-30B-A3B-Instruct-0312-finetune-merged-step3500', speculative_config=None, tokenizer='/home/jovyan/wdh/llama-factory/models/quantization/Qwen3-VL-30B-A3B-Instruct-0312-finetune-merged-step3500', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=16384, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=fp8, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=Qwen3-VL, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['+quant_fp8', 'none', '+quant_fp8'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []}
(EngineCore_DP0 pid=68432) WARNING 03-13 16:08:48 [multiproc_executor.py:945] Reducing Torch parallelism from 64 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
(EngineCore_DP0 pid=68432) INFO 03-13 16:08:48 [multiproc_executor.py:134] DP group leader: node_rank=0, node_rank_within_dp=0, master_addr=127.0.0.1, mq_connect_ip=10.244.1.103 (local), world_size=2, local_world_size=2
(Worker pid=76716) INFO 03-13 16:15:04 [parallel_state.py:1393] world_size=2 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:55520 backend=nccl
(Worker pid=82213) INFO 03-13 16:22:50 [parallel_state.py:1393] world_size=2 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:55520 backend=nccl
(Worker pid=82213) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
(Worker pid=76716) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
(Worker pid=82213) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
(Worker pid=76716) <frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
(Worker pid=76716) INFO 03-13 16:23:20 [pynccl.py:111] vLLM is using nccl==2.27.5
(Worker pid=76716) WARNING 03-13 16:23:20 [symm_mem.py:67] SymmMemCommunicator: Device capability 8.0 not supported, communicator is not available.
(Worker pid=82213) WARNING 03-13 16:23:20 [symm_mem.py:67] SymmMemCommunicator: Device capability 8.0 not supported, communicator is not available.
(Worker pid=76716) INFO 03-13 16:23:21 [parallel_state.py:1715] rank 0 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A
(Worker pid=82213) INFO 03-13 16:23:21 [parallel_state.py:1715] rank 1 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 1, EP rank 1, EPLB rank N/A
(Worker pid=76716) INFO 03-13 16:23:40 [base.py:106] Offloader set to NoopOffloader
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:40 [gpu_model_runner.py:4255] Starting to load model /home/jovyan/wdh/llama-factory/models/quantization/Qwen3-VL-30B-A3B-Instruct-0312-finetune-merged-step3500...
(Worker pid=82213) INFO 03-13 16:23:40 [base.py:106] Offloader set to NoopOffloader
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:40 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:40 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:40 [vllm.py:747] Asynchronous scheduling is enabled.
(Worker pid=82213) (Worker_TP1 pid=82213) INFO 03-13 16:23:41 [cuda.py:453] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(Worker pid=82213) (Worker_TP1 pid=82213) INFO 03-13 16:23:41 [mm_encoder_attention.py:215] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(Worker pid=82213) (Worker_TP1 pid=82213) INFO 03-13 16:23:41 [vllm.py:747] Asynchronous scheduling is enabled.
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:41 [cuda.py:405] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:41 [flash_attn.py:587] Using FlashAttention version 2
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:41 [fp8.py:390] Using MARLIN Fp8 MoE backend out of potential backends: ['AITER', 'FLASHINFER_TRTLLM', 'FLASHINFER_CUTLASS', 'DEEPGEMM', 'TRITON', 'MARLIN', 'BATCHED_DEEPGEMM', 'BATCHED_TRITON', 'XPU'].
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800] WorkerProc failed to start.
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800] Traceback (most recent call last):
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 771, in worker_main
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     worker = WorkerProc(*args, **kwargs)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 597, in __init__
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     self.worker.load_model()
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 337, in load_model
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     self.model_runner.load_model(load_dummy_weights=dummy_weights)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 4271, in load_model
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     self.model = model_loader.load_model(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]                  ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 62, in load_model
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     self.load_weights(model, model_config)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 290, in load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     loaded_weights = model.load_weights(self.get_all_weights(model_config, model))
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_vl.py", line 2405, in load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return loader.load_weights(weights, mapper=self.hf_to_vllm_mapper)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/reload/torchao_decorator.py", line 50, in patched_model_load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return original_load_weights(self, weights, *args, **kwargs)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 340, in load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     autoloaded_weights = set(self._load_module("", self.module, weights))
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 287, in _load_module
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     yield from self._load_module(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 260, in _load_module
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     loaded_params = module_load_weights(weights)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 797, in load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return loader.load_weights(weights)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/reload/torchao_decorator.py", line 50, in patched_model_load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     return original_load_weights(self, weights, *args, **kwargs)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 340, in load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     autoloaded_weights = set(self._load_module("", self.module, weights))
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 287, in _load_module
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     yield from self._load_module(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 260, in _load_module
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     loaded_params = module_load_weights(weights)
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_vl_moe.py", line 264, in load_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     success = self.load_fused_expert_weights(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_vl_moe.py", line 149, in load_fused_expert_weights
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     success = weight_loader(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]               ^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/layers/fused_moe/layer.py", line 1308, in weight_loader
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     self._load_model_weight_or_group_weight_scale(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/layers/fused_moe/layer.py", line 900, in _load_model_weight_or_group_weight_scale
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     self._load_w2(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/layers/fused_moe/layer.py", line 982, in _load_w2
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]     loaded_weight = loaded_weight.narrow(
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800]                     ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=82213) (Worker_TP1 pid=82213) ERROR 03-13 16:23:44 [multiproc_executor.py:800] IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800] WorkerProc failed to start.
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800] Traceback (most recent call last):
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 771, in worker_main
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     worker = WorkerProc(*args, **kwargs)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 597, in __init__
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     self.worker.load_model()
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 337, in load_model
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     self.model_runner.load_model(load_dummy_weights=dummy_weights)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 4271, in load_model
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     self.model = model_loader.load_model(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]                  ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 62, in load_model
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     self.load_weights(model, model_config)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return func(*args, **kwargs)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 290, in load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     loaded_weights = model.load_weights(self.get_all_weights(model_config, model))
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_vl.py", line 2405, in load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return loader.load_weights(weights, mapper=self.hf_to_vllm_mapper)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/reload/torchao_decorator.py", line 50, in patched_model_load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return original_load_weights(self, weights, *args, **kwargs)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 340, in load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     autoloaded_weights = set(self._load_module("", self.module, weights))
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 287, in _load_module
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     yield from self._load_module(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 260, in _load_module
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     loaded_params = module_load_weights(weights)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 797, in load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return loader.load_weights(weights)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/model_loader/reload/torchao_decorator.py", line 50, in patched_model_load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     return original_load_weights(self, weights, *args, **kwargs)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 340, in load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     autoloaded_weights = set(self._load_module("", self.module, weights))
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 287, in _load_module
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     yield from self._load_module(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 260, in _load_module
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     loaded_params = module_load_weights(weights)
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_vl_moe.py", line 264, in load_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     success = self.load_fused_expert_weights(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_vl_moe.py", line 149, in load_fused_expert_weights
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     success = weight_loader(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]               ^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/layers/fused_moe/layer.py", line 1308, in weight_loader
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     self._load_model_weight_or_group_weight_scale(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/layers/fused_moe/layer.py", line 900, in _load_model_weight_or_group_weight_scale
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     self._load_w2(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/model_executor/layers/fused_moe/layer.py", line 982, in _load_w2
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]     loaded_weight = loaded_weight.narrow(
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800]                     ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=76716) (Worker_TP0 pid=76716) ERROR 03-13 16:23:45 [multiproc_executor.py:800] IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:01<?, ?it/s]
(Worker pid=76716) (Worker_TP0 pid=76716)
(Worker pid=82213) (Worker_TP1 pid=82213) INFO 03-13 16:23:45 [multiproc_executor.py:749] Parent process exited, terminating worker
(Worker pid=76716) (Worker_TP0 pid=76716) INFO 03-13 16:23:45 [multiproc_executor.py:749] Parent process exited, terminating worker
[rank0]:[W313 16:23:45.207205080 ProcessGroupNCCL.cpp:1553] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100] EngineCore failed to start.
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100] Traceback (most recent call last):
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 1090, in run_engine_core
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     return func(*args, **kwargs)
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 834, in __init__
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     super().__init__(
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 110, in __init__
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 100, in __init__
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     super().__init__(vllm_config)
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     return func(*args, **kwargs)
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/abstract.py", line 103, in __init__
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     self._init_executor()
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 180, in _init_executor
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 697, in wait_for_ready
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100]     raise e from None
(EngineCore_DP0 pid=68432) ERROR 03-13 16:23:46 [core.py:1100] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(EngineCore_DP0 pid=68432) Process EngineCore_DP0:
(EngineCore_DP0 pid=68432) Traceback (most recent call last):
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=68432)     self.run()
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=68432)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 1104, in run_engine_core
(EngineCore_DP0 pid=68432)     raise e
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 1090, in run_engine_core
(EngineCore_DP0 pid=68432)     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore_DP0 pid=68432)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore_DP0 pid=68432)     return func(*args, **kwargs)
(EngineCore_DP0 pid=68432)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 834, in __init__
(EngineCore_DP0 pid=68432)     super().__init__(
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 110, in __init__
(EngineCore_DP0 pid=68432)     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=68432)                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 100, in __init__
(EngineCore_DP0 pid=68432)     super().__init__(vllm_config)
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore_DP0 pid=68432)     return func(*args, **kwargs)
(EngineCore_DP0 pid=68432)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/abstract.py", line 103, in __init__
(EngineCore_DP0 pid=68432)     self._init_executor()
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 180, in _init_executor
(EngineCore_DP0 pid=68432)     self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore_DP0 pid=68432)                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=68432)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 697, in wait_for_ready
(EngineCore_DP0 pid=68432)     raise e from None
(EngineCore_DP0 pid=68432) Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(APIServer pid=59342) Traceback (most recent call last):
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/bin/vllm", line 7, in <module>
(APIServer pid=59342)     sys.exit(main())
(APIServer pid=59342)              ^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=59342)     args.dispatch_function(args)
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/entrypoints/cli/serve.py", line 112, in cmd
(APIServer pid=59342)     uvloop.run(run_server(args))
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/uvloop/__init__.py", line 92, in run
(APIServer pid=59342)     return runner.run(wrapper())
(APIServer pid=59342)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/asyncio/runners.py", line 118, in run
(APIServer pid=59342)     return self._loop.run_until_complete(task)
(APIServer pid=59342)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=59342)     return await main
(APIServer pid=59342)            ^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 471, in run_server
(APIServer pid=59342)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 490, in run_server_worker
(APIServer pid=59342)     async with build_async_engine_client(
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=59342)     return await anext(self.gen)
(APIServer pid=59342)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 96, in build_async_engine_client
(APIServer pid=59342)     async with build_async_engine_client_from_engine_args(
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/contextlib.py", line 210, in __aenter__
(APIServer pid=59342)     return await anext(self.gen)
(APIServer pid=59342)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 137, in build_async_engine_client_from_engine_args
(APIServer pid=59342)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=59342)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/async_llm.py", line 225, in from_vllm_config
(APIServer pid=59342)     return cls(
(APIServer pid=59342)            ^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/async_llm.py", line 154, in __init__
(APIServer pid=59342)     self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=59342)                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=59342)     return func(*args, **kwargs)
(APIServer pid=59342)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 127, in make_async_mp_client
(APIServer pid=59342)     return AsyncMPClient(*client_args)
(APIServer pid=59342)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=59342)     return func(*args, **kwargs)
(APIServer pid=59342)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 911, in __init__
(APIServer pid=59342)     super().__init__(
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 569, in __init__
(APIServer pid=59342)     with launch_core_engines(
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/contextlib.py", line 144, in __exit__
(APIServer pid=59342)     next(self.gen)
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/utils.py", line 951, in launch_core_engines
(APIServer pid=59342)     wait_for_engine_startup(
(APIServer pid=59342)   File "/home/jovyan/wdh/env/wdh-vllm01011/lib/python3.11/site-packages/vllm/v1/engine/utils.py", line 1010, in wait_for_engine_startup
(APIServer pid=59342)     raise RuntimeError(
(APIServer pid=59342) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

做过尝试：

  1. 降低vllm部署时的transformer环境（5.2.0->4.57.6） ==>无效
  2. 使用原生模型的config ==> 无效

请问这是meagtron sft 的参数不对，还是什么原因啊？

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported path with the listed megatron sft, swift export, and vllm serve commands, then capture the complete traceback beyond the truncated log. Compare the exported FP8 model metadata and configuration with the vLLM Qwen3-VL loading entry point; done means identifying the incompatible dimension handling and confirming that the quantized model loads successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.