Running 4 * B70 met warning on conrainer “intel/llm-scaler-vllm:0.14.0-b8.1” and got low perf

# ENV:
Ubuntu 24.04.4 LTS + B70 + intel/llm-scaler-vllm:0.14.0-b8.1
already installed B70 dirver kit : multi-arc-bmg-offline-installer-26.18.8.2-combo

Can found B70 device :
sycl-ls
[level_zero:gpu][level_zero:0] Intel(R) oneAPI Unified Runtime over Level-Zero V2, Intel(R) Graphics [0xe223] 20.2.0 [1.14.37435+12]
[level_zero:gpu][level_zero:1] Intel(R) oneAPI Unified Runtime over Level-Zero V2, Intel(R) Graphics [0xe223] 20.2.0 [1.14.37435+12]
[level_zero:gpu][level_zero:2] Intel(R) oneAPI Unified Runtime over Level-Zero V2, Intel(R) Graphics [0xe223] 20.2.0 [1.14.37435+12]
[level_zero:gpu][level_zero:3] Intel(R) oneAPI Unified Runtime over Level-Zero V2, Intel(R) Graphics [0xe223] 20.2.0 [1.14.37435+12]
[opencl:cpu][opencl:0] Intel(R) OpenCL, Intel(R) Xeon(R) Gold 6438N OpenCL 3.0 (Build 0) [2026.20.1.0.12_160000]
[opencl:gpu][opencl:1] Intel(R) OpenCL Graphics, Intel(R) Graphics [0xe223] OpenCL 3.0 NEO [26.09.37435.12]
[opencl:gpu][opencl:2] Intel(R) OpenCL Graphics, Intel(R) Graphics [0xe223] OpenCL 3.0 NEO [26.09.37435.12]
[opencl:gpu][opencl:3] Intel(R) OpenCL Graphics, Intel(R) Graphics [0xe223] OpenCL 3.0 NEO [26.09.37435.12]
[opencl:gpu][opencl:4] Intel(R) OpenCL Graphics, Intel(R) Graphics [0xe223] OpenCL 3.0 NEO [26.09.37435.12]

# Log info:
/llm#` VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 VLLM_WORKER_MULTIPROC_METHOD=spawn vllm serve --model /llm/models/Qwen3.5-9B --served-model-name Qwen3.5-9B --dtype=float16 --enforce-eager --port 8300 --host 0.0.0.0 --trust-remote-code --disable-sliding-window --gpu-memory-util=0.8 --max-num-batched-tokens=40960 --disable-log-requests --max-model-len=40960 --block-size 64 --quantization fp8 -tp=4`
[W509 08:13:57.944128964 OperatorEntry.cpp:208] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)
registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /pytorch/aten/src/ATen/VmapModeRegistrations.cpp:36
new kernel: registered at /root/workspace/frameworks.ai.pytorch.ipex-gpu/build/Release/csrc/gpu/csrc/gpu/xpu/ATen/RegisterXPU_0.cpp:172 (function operator())
WARNING 05-09 08:14:05 [_logger.py:68] With `vllm serve`, you should provide the model as a positional argument or in a config file instead of via the `--model` option. The `--model` option will be removed in v0.13.
(APIServer pid=2921) INFO 05-09 08:14:05 [api_server.py:1272] vLLM API server version 0.14.1.dev0+gb17039bcc.d20260311
(APIServer pid=2921) INFO 05-09 08:14:05 [utils.py:263] non-default args: {'model_tag': '/llm/models/Qwen3.5-9B', 'host': '0.0.0.0', 'port': 8300, 'model': '/llm/models/Qwen3.5-9B', 'trust_remote_code': True, 'dtype': 'float16', 'max_model_len': 40960, 'quantization': 'fp8', 'enforce_eager': True, 'disable_sliding_window': True, 'served_model_name': ['Qwen3.5-9B'], 'tensor_parallel_size': 4, 'block_size': 64, 'gpu_memory_utilization': 0.8, 'max_num_batched_tokens': 40960}
(APIServer pid=2921) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=2921) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) INFO 05-09 08:14:05 [model.py:533] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=2921) WARNING 05-09 08:14:05 [_logger.py:68] Casting torch.bfloat16 to torch.float16.
(APIServer pid=2921) INFO 05-09 08:14:05 [model.py:1549] Using max model len 40960
(APIServer pid=2921) INFO 05-09 08:14:06 [scheduler.py:229] Chunked prefill is enabled with max_num_batched_tokens=40960.
(APIServer pid=2921) INFO 05-09 08:14:06 [config.py:479] Setting attention block size to 320 tokens to ensure that attention page size is >= mamba page size.
(APIServer pid=2921) INFO 05-09 08:14:06 [config.py:503] Padding mamba page size by 19.40% to ensure that mamba page size and attention page size are exactly equal.
(APIServer pid=2921) INFO 05-09 08:14:06 [vllm.py:636] Asynchronous scheduling is disabled.
(APIServer pid=2921) WARNING 05-09 08:14:06 [_logger.py:68] Enforce eager set, overriding optimization level to -O0
(APIServer pid=2921) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
[W509 08:14:13.701705043 OperatorEntry.cpp:208] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)
registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /pytorch/aten/src/ATen/VmapModeRegistrations.cpp:36
new kernel: registered at /root/workspace/frameworks.ai.pytorch.ipex-gpu/build/Release/csrc/gpu/csrc/gpu/xpu/ATen/RegisterXPU_0.cpp:172 (function operator())
(EngineCore_DP0 pid=2980) INFO 05-09 08:14:19 [core.py:97] Initializing a V1 LLM engine (v0.14.1.dev0+gb17039bcc.d20260311) with config: model='/llm/models/Qwen3.5-9B', speculative_config=None, tokenizer='/llm/models/Qwen3.5-9B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=40960, download_dir=None, load_format=auto, tensor_parallel_size=4, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=fp8, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=xpu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=Qwen3.5-9B, enable_prefix_caching=False, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [40960], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': True}, 'local_cache_dir': None}
(EngineCore_DP0 pid=2980) WARNING 05-09 08:14:19 [_logger.py:68] Reducing Torch parallelism from 16 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
[W509 08:14:23.729319520 OperatorEntry.cpp:208] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)
registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /pytorch/aten/src/ATen/VmapModeRegistrations.cpp:36
new kernel: registered at /root/workspace/frameworks.ai.pytorch.ipex-gpu/build/Release/csrc/gpu/csrc/gpu/xpu/ATen/RegisterXPU_0.cpp:172 (function operator())
[W509 08:14:23.372969638 OperatorEntry.cpp:208] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)
registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /pytorch/aten/src/ATen/VmapModeRegistrations.cpp:36
new kernel: registered at /root/workspace/frameworks.ai.pytorch.ipex-gpu/build/Release/csrc/gpu/csrc/gpu/xpu/ATen/RegisterXPU_0.cpp:172 (function operator())
[W509 08:14:24.839577004 OperatorEntry.cpp:208] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)
registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /pytorch/aten/src/ATen/VmapModeRegistrations.cpp:36
new kernel: registered at /root/workspace/frameworks.ai.pytorch.ipex-gpu/build/Release/csrc/gpu/csrc/gpu/xpu/ATen/RegisterXPU_0.cpp:172 (function operator())
[W509 08:14:24.066333040 OperatorEntry.cpp:208] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)
registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /pytorch/aten/src/ATen/VmapModeRegistrations.cpp:36
new kernel: registered at /root/workspace/frameworks.ai.pytorch.ipex-gpu/build/Release/csrc/gpu/csrc/gpu/xpu/ATen/RegisterXPU_0.cpp:172 (function operator())
INFO 05-09 08:14:31 [parallel_state.py:1214] world_size=4 rank=2 local_rank=2 distributed_init_method=tcp://127.0.0.1:46807 backend=xccl
INFO 05-09 08:14:32 [parallel_state.py:1214] world_size=4 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:46807 backend=xccl
INFO 05-09 08:14:33 [parallel_state.py:1214] world_size=4 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:46807 backend=xccl
[rank1]:[W509 08:14:33.861377322 ProcessGroupGloo.cpp:511] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator())
INFO 05-09 08:14:33 [parallel_state.py:1214] world_size=4 rank=3 local_rank=3 distributed_init_method=tcp://127.0.0.1:46807 backend=xccl
[rank3]:[W509 08:14:33.037095418 ProcessGroupGloo.cpp:511] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator())
[rank0]:[W509 08:14:34.813788085 ProcessGroupGloo.cpp:511] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator())
[rank2]:[W509 08:14:34.813893261 ProcessGroupGloo.cpp:511] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator())
INFO 05-09 08:14:36 [parallel_state.py:1425] rank 0 in world size 4 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A
INFO 05-09 08:14:36 [parallel_state.py:1425] rank 2 in world size 4 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 2, EP rank N/A
INFO 05-09 08:14:36 [parallel_state.py:1425] rank 3 in world size 4 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 3, EP rank N/A
INFO 05-09 08:14:36 [parallel_state.py:1425] rank 1 in world size 4 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 1, EP rank N/A
2026:05:09-08:14:36:( 3036) |CCL_WARN| value of CCL_ATL_TRANSPORT changed to be ofi (default:mpi)
2026:05:09-08:14:36:( 3036) |CCL_WARN| could not get local_idx/count from environment variables, trying to get them from ATL
2026:05:09-08:14:36:( 3034) |CCL_WARN| value of CCL_ATL_TRANSPORT changed to be ofi (default:mpi)
2026:05:09-08:14:36:( 3034) |CCL_WARN| could not get local_idx/count from environment variables, trying to get them from ATL
2026:05:09-08:14:36:( 3035) |CCL_WARN| value of CCL_ATL_TRANSPORT changed to be ofi (default:mpi)
2026:05:09-08:14:36:( 3035) |CCL_WARN| could not get local_idx/count from environment variables, trying to get them from ATL
2026:05:09-08:14:36:( 3037) |CCL_WARN| value of CCL_ATL_TRANSPORT changed to be ofi (default:mpi)
2026:05:09-08:14:36:( 3037) |CCL_WARN| could not get local_idx/count from environment variables, trying to get them from ATL
2026:05:09-08:14:37:( 3036) |CCL_WARN| topology recognition shows PCIe connection between devices. If this is not correct, you can disable topology recognition, with CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0. This will assume XeLinks across devices
2026:05:09-08:14:37:( 3034) |CCL_WARN| topology recognition shows PCIe connection between devices. If this is not correct, you can disable topology recognition, with CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0. This will assume XeLinks across devices
2026:05:09-08:14:37:( 3037) |CCL_WARN| topology recognition shows PCIe connection between devices. If this is not correct, you can disable topology recognition, with CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0. This will assume XeLinks across devices
2026:05:09-08:14:37:( 3035) |CCL_WARN| topology recognition shows PCIe connection between devices. If this is not correct, you can disable topology recognition, with CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0. This will assume XeLinks across devices
(Worker_TP0 pid=3034) INFO 05-09 08:14:47 [gpu_model_runner.py:3811] Starting to load model /llm/models/Qwen3.5-9B...
(Worker_TP2 pid=3036) INFO 05-09 08:14:48 [xpu.py:106] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(Worker_TP2 pid=3036) INFO 05-09 08:14:48 [xpu.py:103] Using backend AttentionBackendEnum.IPEX for vit attention
(Worker_TP2 pid=3036) INFO 05-09 08:14:48 [mm_encoder_attention.py:89] Using AttentionBackendEnum.IPEX for MMEncoderAttention.
(Worker_TP2 pid=3036) INFO 05-09 08:14:49 [xpu.py:53] Setting VLLM_KV_CACHE_LAYOUT to 'NHD' for XPU; only NHD layout is supported by XPU attention kernels.
(Worker_TP2 pid=3036) INFO 05-09 08:14:49 [xpu.py:79] Using Flash Attention backend.
(Worker_TP0 pid=3034) INFO 05-09 08:14:49 [xpu.py:106] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(Worker_TP0 pid=3034) INFO 05-09 08:14:49 [xpu.py:103] Using backend AttentionBackendEnum.IPEX for vit attention
(Worker_TP0 pid=3034) INFO 05-09 08:14:49 [mm_encoder_attention.py:89] Using AttentionBackendEnum.IPEX for MMEncoderAttention.
(Worker_TP0 pid=3034) INFO 05-09 08:14:49 [xpu.py:53] Setting VLLM_KV_CACHE_LAYOUT to 'NHD' for XPU; only NHD layout is supported by XPU attention kernels.
(Worker_TP0 pid=3034) INFO 05-09 08:14:49 [xpu.py:79] Using Flash Attention backend.
Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 Peak Allocated (Real Need) : 7.60 GB
(Worker_TP1 pid=3035) > Model memory usage : 2.91 GB
(Worker_TP1 pid=3035) > Current Reserved (Footprint): 10.10 GB
(Worker_TP1 pid=3035) > Fragmentation (Wasted) : 2.51 GB
(Worker_TP3 pid=3037)
(Worker_TP3 pid=3037) [Memory Profiling Analysis]
(Worker_TP3 pid=3037) > Peak Allocated (Real Need) : 7.60 GB
(Worker_TP3 pid=3037) > Model memory usage : 2.91 GB
(Worker_TP3 pid=3037) > Current Reserved (Footprint): 10.10 GB
(Worker_TP3 pid=3037) > Fragmentation (Wasted) : 2.51 GB
(Worker_TP0 pid=3034)
(Worker_TP0 pid=3034) [Memory Profiling Analysis]
(Worker_TP0 pid=3034) > Peak Allocated (Real Need) : 7.60 GB
(Worker_TP0 pid=3034) > Model memory usage : 2.91 GB
(Worker_TP0 pid=3034) > Current Reserved (Footprint): 10.10 GB
(Worker_TP0 pid=3034) > Fragmentation (Wasted) : 2.51 GB
(Worker_TP2 pid=3036)
(Worker_TP2 pid=3036) [Memory Profiling Analysis]
(Worker_TP2 pid=3036) > Peak Allocated (Real Need) : 7.60 GB
(Worker_TP2 pid=3036) > Model memory usage : 2.91 GB
(Worker_TP2 pid=3036) > Current Reserved (Footprint): 10.10 GB
(Worker_TP2 pid=3036) > Fragmentation (Wasted) : 2.51 GB
(EngineCore_DP0 pid=2980) INFO 05-09 08:18:18 [kv_cache_utils.py:1308] GPU KV cache size: 545,280 tokens
(EngineCore_DP0 pid=2980) INFO 05-09 08:18:18 [kv_cache_utils.py:1313] Maximum concurrency for 40,960 tokens per request: 52.03x
(Worker_TP0 pid=3034) INFO 05-09 08:18:18 [utils.py:189] `_KV_CACHE_LAYOUT_OVERRIDE` variable detected. Setting KV cache layout to NHD.
(Worker_TP2 pid=3036) INFO 05-09 08:18:18 [utils.py:189] `_KV_CACHE_LAYOUT_OVERRIDE` variable detected. Setting KV cache layout to NHD.
(Worker_TP3 pid=3037) INFO 05-09 08:18:18 [utils.py:189] `_KV_CACHE_LAYOUT_OVERRIDE` variable detected. Setting KV cache layout to NHD.
(Worker_TP1 pid=3035) INFO 05-09 08:18:18 [utils.py:189] `_KV_CACHE_LAYOUT_OVERRIDE` variable detected. Setting KV cache layout to NHD.
(EngineCore_DP0 pid=2980) INFO 05-09 08:18:19 [core.py:273] init engine (profile, create kv cache, warmup model) took 203.29 seconds
(EngineCore_DP0 pid=2980) INFO 05-09 08:18:28 [vllm.py:636] Asynchronous scheduling is disabled.
(EngineCore_DP0 pid=2980) WARNING 05-09 08:18:28 [_logger.py:68] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
(APIServer pid=2921) INFO 05-09 08:18:28 [api_server.py:1014] Supported tasks: ['generate']
(APIServer pid=2921) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) INFO 05-09 08:18:28 [serving_chat.py:182] Warming up chat template processing...
(APIServer pid=2921) INFO 05-09 08:18:30 [chat_utils.py:599] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(APIServer pid=2921) INFO 05-09 08:18:30 [serving_chat.py:218] Chat template warmup completed in 2426.9ms
(APIServer pid=2921) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_section', 'mrope_interleaved'}
(APIServer pid=2921) INFO 05-09 08:18:30 [api_server.py:1346] Starting vLLM API server 0 on http://0.0.0.0:8300
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:38] Available routes are:
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /openapi.json, Methods: HEAD, GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /docs, Methods: HEAD, GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: HEAD, GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /redoc, Methods: HEAD, GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /tokenize, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /detokenize, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /inference/v1/generate, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /pause, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /resume, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /is_paused, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /metrics, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /health, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /load, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/models, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /version, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/responses, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/messages, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/completions, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/audio/transcriptions, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/audio/translations, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /ping, Methods: GET
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /ping, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /invocations, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /classify, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/embeddings, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /score, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/score, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /rerank, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v1/rerank, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /v2/rerank, Methods: POST
(APIServer pid=2921) INFO 05-09 08:18:30 [launcher.py:46] Route: /pooling, Methods: POST
(APIServer pid=2921) INFO: Started server process [2921]
(APIServer pid=2921) INFO: Waiting for application startup.
(APIServer pid=2921) INFO: Application startup complete.

(APIServer pid=2921) INFO: 127.0.0.1:35570 - "POST /v1/completions HTTP/1.1" 200 OK
(EngineCore_DP0 pid=2980) INFO 05-09 08:19:54 [shm_broadcast.py:542] No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).
(APIServer pid=2921) INFO: 127.0.0.1:35570 - "GET /metrics HTTP/1.1" 200 OK
(APIServer pid=2921) INFO: 127.0.0.1:35570 - "POST /v1/completions HTTP/1.1" 200 OK
(APIServer pid=2921) INFO 05-09 08:20:31 [loggers.py:257] Engine 000: Avg prompt throughput: 3276.6 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO 05-09 08:20:41 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(EngineCore_DP0 pid=2980) INFO 05-09 08:21:24 [shm_broadcast.py:542] `No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).`
(APIServer pid=2921) INFO 05-09 08:22:01 [loggers.py:257] Engine 000: Avg prompt throughput: 3276.6 tokens/s, Avg generation throughput: 3.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO 05-09 08:22:11 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO 05-09 08:22:21 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO 05-09 08:22:31 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO 05-09 08:22:41 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 18.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO 05-09 08:22:51 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO: 127.0.0.1:35570 - "GET /metrics HTTP/1.1" 200 OK
(APIServer pid=2921) INFO 05-09 08:23:01 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2.2 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(APIServer pid=2921) INFO: 127.0.0.1:56112 - "POST /v1/completions HTTP/1.1" 200 OK
(APIServer pid=2921) INFO 05-09 08:23:11 [loggers.py:257] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
(EngineCore_DP0 pid=2980) INFO 05-09 08:24:06 [shm_broadcast.py:542] `No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).`
(APIServer pid=2921) INFO: 127.0.0.1:56112 - "GET /metrics HTTP/1.1" 200 OK
(APIServer pid=2921) INFO: 127.0.0.1:56112 - "POST /v1/completions HTTP/1.1" 200 OK
(APIServer pid=2921) INFO: 127.0.0.1:35636 - "POST /v1/completions HTTP/1.1" 200 OK
(APIServer pid=2921) INFO: 127.0.0.1:35652 - "POST /v1/completions HTTP/1.1" 200 OK
(APIServer pid=2921) INFO: 127.0.0.1:35656 - "POST /v1/completions HTTP/1.1" 200 OK

# Question:
when running b70 * 4 with container "intel/llm-scaler-vllm:0.14.0-b8.1" , It could work but with very slow performance,
the warning: "No available shared memory broadcast block found in 60 seconds" always occured .
please help to check what happened while 4 of B70 work together.
other test ， 2 * B70 didn't have this warning show.

thanks！