Failed to running model “Qwen3.6-27B” + ”DFlash“ with container intel/llm-scaler-vllm:0.21.0-b3

Failed to running model “Qwen3.6-27B” + ”DFlash“ with container intel/llm-scaler-vllm:0.21.0-b3
please help to solve it.
thanks!

### ENV
OS：Ubuntu 24.04.4 LTS
kernel： 6.17.0-1009-intel
GPU： B70 * 2
Container： intel/llm-scaler-vllm:0.21.0-b3
Model： Qwen3.6-27B + Qwen3.6-27B-DFlash

### launch container
docker run -td --privileged \
--ipc=host \
--net=host \
--device=/dev/dri \
--hostname=lsv-container \
--name=lsv-container \
-v /ai-data/:/llm/models/ \
--shm-size="32g" \
--entrypoint /bin/bash \
intel/llm-scaler-vllm:0.21.0-b3

### launch vllm
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 && export VLLM_WORKER_MULTIPROC_METHOD=spawn && export ZE_AFFINITY_MASK=0,1
vllm serve --port 28100 \
--host 0.0.0.0 \
--gpu-memory-util 0.9 \
--max-num-batched-tokens 8192 \
--max-model-len 64000 \
--dtype float16 \
--block-size 64 \
--model /llm/models/models/Qwen3.6-27B \
--served-model-name Qwen3.6-27B \
--tensor-parallel-size 2 \
--quantization fp8 \
--enforce-eager \
--trust-remote-code \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--kv-cache-metrics \
--enable-prefix-caching \
--kv-cache-dtype fp8 \
--speculative-config '{"method":"dflash","num_speculative_tokens":8, "model":"/llm/models/models/Qwen3.6-27B-DFlash"}'

### error log：
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 && export VLLM_WORKER_MULTIPROC_METHOD=spawn && export ZE_AFFINITY_MASK=0,1
vllm serve --port 28100 \
--host 0.0.0.0 \
--gpu-memory-util 0.9 \
--max-num-batched-tokens 8192 \
--max-model-len 64000 \
--dtype float16 \
--block-size 64 \
--model /llm/models/models/Qwen3.6-27B \
--served-model-name Qwen3.6-27B \
--tensor-parallel-size 2 \
--quantization fp8 \
--enforce-eager \
--trust-remote-code \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--kv-cache-metrics \
--enable-prefix-caching \
--kv-cache-dtype fp8 \
--speculative-config '{"method":"dflash","num_speculative_tokens":8, "model":"/llm/models/models/Qwen3.6-27B-DFlash"}'

WARNING 08-13 07:29:07 [argparse_utils.py:257] With `vllm serve`, you should provide the model as a positional argument or in a config file instead of via the `--model` option. The `--model` option will be removed in a future version.
(APIServer pid=6868) INFO 08-13 07:29:07 [utils.py:306]
(APIServer pid=6868) INFO 08-13 07:29:07 [utils.py:306] █ █ █▄ ▄█
(APIServer pid=6868) INFO 08-13 07:29:07 [utils.py:306] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.21.1.dev0+gad7125a43.d20260810
(APIServer pid=6868) INFO 08-13 07:29:07 [utils.py:306] █▄█▀ █ █ █ █ model /llm/models/models/Qwen3.6-27B
(APIServer pid=6868) INFO 08-13 07:29:07 [utils.py:306] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=6868) INFO 08-13 07:29:07 [utils.py:306]
(APIServer pid=6868) INFO 08-13 07:29:07 [utils.py:240] non-default args: {'model_tag': '/llm/models/models/Qwen3.6-27B', 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_coder', 'host': '0.0.0.0', 'port': 28100, 'model': '/llm/models/models/Qwen3.6-27B', 'trust_remote_code': True, 'dtype': 'float16', 'max_model_len': 64000, 'quantization': 'fp8', 'enforce_eager': True, 'served_model_name': ['Qwen3.6-27B'], 'tensor_parallel_size': 2, 'block_size': 64, 'gpu_memory_utilization': 0.9, 'kv_cache_dtype': 'fp8', 'enable_prefix_caching': True, 'kv_cache_metrics': True, 'max_num_batched_tokens': 8192, 'speculative_config': {'method': 'dflash', 'num_speculative_tokens': 8, 'model': '/llm/models/models/Qwen3.6-27B-DFlash'}}
(APIServer pid=6868) WARNING 08-13 07:29:07 [envs.py:1881] Unknown vLLM environment variable detected: VLLM_ENGINE_VERSION
(APIServer pid=6868) WARNING 08-13 07:29:07 [envs.py:1881] Unknown vLLM environment variable detected: VLLM_ATTENTION_BACKEND
(APIServer pid=6868) INFO 08-13 07:29:07 [model.py:568] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=6868) WARNING 08-13 07:29:07 [model.py:2040] Casting torch.bfloat16 to torch.float16.
(APIServer pid=6868) INFO 08-13 07:29:07 [model.py:1702] Using max model len 64000
(APIServer pid=6868) INFO 08-13 07:29:07 [cache.py:261] Using fp8 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor
(APIServer pid=6868) INFO 08-13 07:29:07 [model.py:568] Resolved architecture: DFlashDraftModel
(APIServer pid=6868) WARNING 08-13 07:29:07 [model.py:2040] Casting torch.bfloat16 to torch.float16.
(APIServer pid=6868) INFO 08-13 07:29:07 [model.py:1702] Using max model len 262144
(APIServer pid=6868) INFO 08-13 07:29:07 [speculative.py:841] Overriding draft model max model len from 262144 to 64000
(APIServer pid=6868) INFO 08-13 07:29:07 [scheduler.py:239] Chunked prefill is enabled with max_num_batched_tokens=8192.
(APIServer pid=6868) WARNING 08-13 07:29:07 [config.py:441] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(APIServer pid=6868) INFO 08-13 07:29:07 [config.py:461] Warning: Prefix caching in Mamba cache 'align' mode is currently enabled. Its support for Mamba layers is experimental. Please report any issues you may observe.
(APIServer pid=6868) INFO 08-13 07:29:08 [esimd_fake_ops.py:220] Registered fake impls for custom_esimd_kernels_vllm / moe_ops ops (torch.compile / XPU graph support).
(APIServer pid=6868) INFO 08-13 07:29:08 [vllm.py:895] Asynchronous scheduling is enabled.
(APIServer pid=6868) WARNING 08-13 07:29:08 [vllm.py:951] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none
(APIServer pid=6868) WARNING 08-13 07:29:08 [vllm.py:969] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
(APIServer pid=6868) INFO 08-13 07:29:08 [kernel.py:212] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['xpu_kernels', 'native'], fused_add_rms_norm=['xpu_kernels', 'native'])
(APIServer pid=6868) WARNING 08-13 07:29:08 [vllm.py:1464] max_num_scheduled_tokens is set to 6400 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens or max_num_seqs.
(APIServer pid=6868) INFO 08-13 07:29:08 [vllm.py:1144] Cudagraph is disabled under eager mode
(APIServer pid=6868) WARNING 08-13 07:29:08 [xpu.py:250] XPU Graph is disabled by environment variable, please set VLLM_XPU_ENABLE_XPU_GRAPH=1 to enable it.
(APIServer pid=6868) WARNING 08-13 07:29:08 [xpu.py:317] Feature 'RMSNorm + quant fusion' is not yet supported on XPU and will be disabled.
(APIServer pid=6868) WARNING 08-13 07:29:08 [xpu.py:317] Feature 'Activation + quant fusion' is not yet supported on XPU and will be disabled.
(APIServer pid=6868) [transformers] `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
(APIServer pid=6868) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
INFO 08-13 07:29:25 [esimd_fake_ops.py:220] Registered fake impls for custom_esimd_kernels_vllm / moe_ops ops (torch.compile / XPU graph support).
(EngineCore pid=7023) INFO 08-13 07:29:25 [core.py:109] Initializing a V1 LLM engine (v0.21.1.dev0+gad7125a43.d20260810) with config: model='/llm/models/models/Qwen3.6-27B', speculative_config=SpeculativeConfig(method='dflash', model='/llm/models/models/Qwen3.6-27B-DFlash', num_spec_tokens=8), tokenizer='/llm/models/models/Qwen3.6-27B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=64000, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=True, quantization=fp8, quantization_config=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=fp8, device_config=xpu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=True, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=Qwen3.6-27B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'ir_enable_torch_wrap': False, 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['xpu_kernels', 'native'], fused_add_rms_norm=['xpu_kernels', 'native']), enable_flashinfer_autotune=False, moe_backend='auto')
(EngineCore pid=7023) WARNING 08-13 07:29:25 [multiproc_executor.py:1029] Reducing Torch parallelism from 24 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
(EngineCore pid=7023) INFO 08-13 07:29:25 [multiproc_executor.py:139] DP group leader: node_rank=0, node_rank_within_dp=0, master_addr=127.0.0.1, mq_connect_ip=172.16.118.100 (local), world_size=2, local_world_size=2
INFO 08-13 07:29:33 [esimd_fake_ops.py:220] Registered fake impls for custom_esimd_kernels_vllm / moe_ops ops (torch.compile / XPU graph support).
INFO 08-13 07:29:33 [esimd_fake_ops.py:220] Registered fake impls for custom_esimd_kernels_vllm / moe_ops ops (torch.compile / XPU graph support).
[transformers] `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
[transformers] `Qwen2VLImageProcessorFast` is deprecated. The `Fast` suffix for image processors has been removed; use `Qwen2VLImageProcessor` instead.
(Worker pid=7173) INFO 08-13 07:29:34 [parallel_state.py:1411] world_size=2 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:58395 backend=xccl
(Worker pid=7174) INFO 08-13 07:29:34 [parallel_state.py:1411] world_size=2 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:58395 backend=xccl
[rank1]:[W813 07:29:35.859655409 ProcessGroupGloo.cpp:524] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator())
[rank0]:[W813 07:29:35.859780314 ProcessGroupGloo.cpp:524] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator())
2026:08:13-07:29:35:( 7174) |CCL_WARN| value of CCL_ATL_TRANSPORT changed to be ofi (default:mpi)
2026:08:13-07:29:35:( 7174) |CCL_WARN| could not get local_idx/count from environment variables, trying to get them from ATL
(Worker pid=7173) INFO 08-13 07:29:35 [parallel_state.py:1724] rank 0 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
2026:08:13-07:29:35:( 7173) |CCL_WARN| value of CCL_ATL_TRANSPORT changed to be ofi (default:mpi)
2026:08:13-07:29:35:( 7173) |CCL_WARN| could not get local_idx/count from environment variables, trying to get them from ATL
2026:08:13-07:29:35:( 7174) |CCL_WARN| topology recognition shows PCIe connection between devices. If this is not correct, you can disable topology recognition, with CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0. This will assume XeLinks across devices
2026:08:13-07:29:35:( 7173) |CCL_WARN| topology recognition shows PCIe connection between devices. If this is not correct, you can disable topology recognition, with CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0. This will assume XeLinks across devices
(Worker pid=7174) WARNING 08-13 07:29:36 [__init__.py:204] min_p and logit_bias parameters won't work with speculative decoding.
(Worker pid=7173) WARNING 08-13 07:29:36 [__init__.py:204] min_p and logit_bias parameters won't work with speculative decoding.
(Worker pid=7174) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
(Worker pid=7173) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
(Worker_TP0 pid=7173) INFO 08-13 07:29:41 [gpu_model_runner.py:4901] Starting to load model /llm/models/models/Qwen3.6-27B...
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [__init__.py:430] Selected XPUFP8ScaledMMLinearKernel for Fp8OnlineLinearMethod
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [xpu.py:167] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(Worker_TP1 pid=7174) INFO 08-13 07:29:42 [xpu.py:59] Setting VLLM_KV_CACHE_LAYOUT to 'NHD' for XPU; only NHD layout is supported by XPU attention kernels.
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [gdn_linear_attn.py:184] Using Triton/FLA GDN prefill kernel
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [xpu.py:59] Setting VLLM_KV_CACHE_LAYOUT to 'NHD' for XPU; only NHD layout is supported by XPU attention kernels.
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [xpu.py:132] Using Flash Attention backend.
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [flash_attn.py:834] Using FlashAttention version 2
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [weight_utils.py:938] Filesystem type for checkpoints: EXT4. Checkpoint size: 51.75 GiB. Available RAM: 115.33 GiB.
(Worker_TP0 pid=7173) INFO 08-13 07:29:42 [weight_utils.py:961] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
Loading safetensors checkpoint shards: 0% Completed | 0/15 [00:00= mamba page size.
(Worker_TP0 pid=7173) INFO 08-13 07:30:08 [interface.py:669] Padding mamba page size by 1.09% to ensure that mamba page size and attention page size are exactly equal.
(Worker_TP0 pid=7173) INFO 08-13 07:30:08 [xpu.py:396] [XPU]Setting attention block size to 2048 tokens to ensure multiple of 64, set mamba_page_size_padded to 2097152 bytes accordingly, before was 1703936 bytes.
(Worker_TP1 pid=7174) INFO 08-13 07:30:08 [interface.py:645] Setting attention block size to 1664 tokens to ensure that attention page size is >= mamba page size.
(Worker_TP1 pid=7174) INFO 08-13 07:30:08 [interface.py:669] Padding mamba page size by 1.09% to ensure that mamba page size and attention page size are exactly equal.
(Worker_TP1 pid=7174) INFO 08-13 07:30:08 [xpu.py:396] [XPU]Setting attention block size to 2048 tokens to ensure multiple of 64, set mamba_page_size_padded to 2097152 bytes accordingly, before was 1703936 bytes.
(Worker_TP0 pid=7173) INFO 08-13 07:30:09 [gpu_model_runner.py:5965] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(Worker_TP0 pid=7173) WARNING 08-13 07:30:12 [topk_topp_sampler.py:259] xpu kernel topk_topp_sampler does not support per-request generators. Falling back to PyTorch-native implementation.
(Worker_TP0 pid=7173) INFO 08-13 07:30:22 [gpu_worker.py:462] Available KV cache memory: 10.24 GiB
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] EngineCore failed to start.
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] Traceback (most recent call last):
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1115, in run_engine_core
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] return func(*args, **kwargs)
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 881, in __init__
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] super().__init__(
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 128, in __init__
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] kv_cache_config = self._initialize_kv_caches(vllm_config)
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] return func(*args, **kwargs)
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 264, in _initialize_kv_caches
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] kv_cache_configs = get_kv_cache_configs(
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1974, in get_kv_cache_configs
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] global_kv_cache_groups = get_kv_cache_groups(vllm_config, merged_kv_cache_specs)
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1656, in get_kv_cache_groups
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] kv_cache_spec = unify_kv_cache_spec_page_size(kv_cache_spec)
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1042, in unify_kv_cache_spec_page_size
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] assert new_spec.page_size_bytes == max_page_size
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) ERROR 08-13 07:30:22 [core.py:1141] AssertionError
(Worker_TP0 pid=7173) WARNING 08-13 07:30:22 [multiproc_executor.py:884] WorkerProc was terminated
(Worker_TP1 pid=7174) WARNING 08-13 07:30:22 [multiproc_executor.py:884] WorkerProc was terminated
(EngineCore pid=7023) ERROR 08-13 07:30:24 [multiproc_executor.py:283] Worker proc VllmWorker-0 died unexpectedly, shutting down executor.
(EngineCore pid=7023) Process EngineCore:
(EngineCore pid=7023) Traceback (most recent call last):
(EngineCore pid=7023) File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore pid=7023) self.run()
(EngineCore pid=7023) File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore pid=7023) self._target(*self._args, **self._kwargs)
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1145, in run_engine_core
(EngineCore pid=7023) raise e
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1115, in run_engine_core
(EngineCore pid=7023) engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=7023) return func(*args, **kwargs)
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 881, in __init__
(EngineCore pid=7023) super().__init__(
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 128, in __init__
(EngineCore pid=7023) kv_cache_config = self._initialize_kv_caches(vllm_config)
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=7023) return func(*args, **kwargs)
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 264, in _initialize_kv_caches
(EngineCore pid=7023) kv_cache_configs = get_kv_cache_configs(
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1974, in get_kv_cache_configs
(EngineCore pid=7023) global_kv_cache_groups = get_kv_cache_groups(vllm_config, merged_kv_cache_specs)
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1656, in get_kv_cache_groups
(EngineCore pid=7023) kv_cache_spec = unify_kv_cache_spec_page_size(kv_cache_spec)
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1042, in unify_kv_cache_spec_page_size
(EngineCore pid=7023) assert new_spec.page_size_bytes == max_page_size
(EngineCore pid=7023) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=7023) AssertionError
(APIServer pid=6868) Traceback (most recent call last):
(APIServer pid=6868) File "/opt/venv/bin/vllm", line 6, in
(APIServer pid=6868) sys.exit(main())
(APIServer pid=6868) ^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 92, in main
(APIServer pid=6868) args.dispatch_function(args)
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 122, in cmd
(APIServer pid=6868) uvloop.run(run_server(args))
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=6868) return __asyncio.run(
(APIServer pid=6868) ^^^^^^^^^^^^^^
(APIServer pid=6868) File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
(APIServer pid=6868) return runner.run(main)
(APIServer pid=6868) ^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=6868) return self._loop.run_until_complete(task)
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=6868) return await main
(APIServer pid=6868) ^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 693, in run_server
(APIServer pid=6868) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 707, in run_server_worker
(APIServer pid=6868) async with build_async_engine_client(
(APIServer pid=6868) File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=6868) return await anext(self.gen)
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 100, in build_async_engine_client
(APIServer pid=6868) async with build_async_engine_client_from_engine_args(
(APIServer pid=6868) File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=6868) return await anext(self.gen)
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 136, in build_async_engine_client_from_engine_args
(APIServer pid=6868) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 217, in from_vllm_config
(APIServer pid=6868) return cls(
(APIServer pid=6868) ^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 146, in __init__
(APIServer pid=6868) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=6868) return func(*args, **kwargs)
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 130, in make_async_mp_client
(APIServer pid=6868) return AsyncMPClient(*client_args)
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=6868) return func(*args, **kwargs)
(APIServer pid=6868) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 907, in __init__
(APIServer pid=6868) super().__init__(
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 535, in __init__
(APIServer pid=6868) with launch_core_engines(
(APIServer pid=6868) File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
(APIServer pid=6868) next(self.gen)
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 1128, in launch_core_engines
(APIServer pid=6868) wait_for_engine_startup(
(APIServer pid=6868) File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 1187, in wait_for_engine_startup
(APIServer pid=6868) raise RuntimeError(
(APIServer pid=6868) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}