Out-of-memory when serving Qwen3.8-27B with image `intel/llm-scaler-vllm:0.26.0-b2`

Using the image `intel/llm-scaler-vllm:0.26.0-b2`, when try to serve Qwen3.8 on 4x B70 (which should have ample VRAM, 4x 32GB = 128 GB). I noticed 2 behaviors.

1. The host memory will spike by more than 120GB causing my machine to OOM.
2. vLLM also complaints not enough VRAM and the server is killed.

Solution that works for me:
1. Add swap to avoid system OOM.
2. Reducing the gpu memory utilization to 0.85, then the server can start. However it's still hogging close to 135GB of memory.

I didn't observe the same kind of behavior on a 4x B60 machine.

vllm serve command:
```
vllm serve Qwen/Qwen3.8-27B --tensor-parallel-size 4 --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --mm-encoder-tp-mode data --port 8020 --quantization sym_int4 --dtype float16 --block-size 64 --trust-remote-code --enforce-eager --enable-prefix-caching
```

logs:
- observation I suspect is due to this `qwen_triton_warmup.py` when warming up triton. Because the host memory spike happened right before printing this.
```
Loading safetensors checkpoint shards: 100% Completed | 18/18 [00:26<00:00, 1.35s/it]
Loading safetensors checkpoint shards: 100% Completed | 18/18 [00:26<00:00, 1.46s/it]
(Worker_TP0 pid=741)
(Worker_TP0 pid=741) INFO 09-11 13:57:46 [default_loader.py:430] Loading weights took 26.37 seconds
(Worker_TP3 pid=744) INFO 09-11 13:57:47 [interface.py:905] Setting attention block size to 832 tokens to ensure that attention page size is >= mamba page size.
(Worker_TP3 pid=744) INFO 09-11 13:57:47 [interface.py:929] Padding mamba page size by 6.26% to ensure that mamba page size and attention page size are exactly equal.
(Worker_TP3 pid=744) INFO 09-11 13:57:47 [xpu.py:388] [XPU]Setting attention block size to 1024 tokens to ensure multiple of 64, set mamba_page_size_padded to 1048576 bytes accordingly, before was 851968 bytes.
(Worker_TP0 pid=741) INFO 09-11 13:57:47 [gpu_model_runner.py:5396] Model loading took 5.22 GiB memory and 29.327146 seconds
(Worker_TP0 pid=741) INFO 09-11 13:57:47 [interface.py:905] Setting attention block size to 832 tokens to ensure that attention page size is >= mamba page size.
(Worker_TP0 pid=741) INFO 09-11 13:57:47 [interface.py:929] Padding mamba page size by 6.26% to ensure that mamba page size and attention page size are exactly equal.
(Worker_TP0 pid=741) INFO 09-11 13:57:47 [xpu.py:388] [XPU]Setting attention block size to 1024 tokens to ensure multiple of 64, set mamba_page_size_padded to 1048576 bytes accordingly, before was 851968 bytes.
(Worker_TP1 pid=742) INFO 09-11 13:57:47 [interface.py:905] Setting attention block size to 832 tokens to ensure that attention page size is >= mamba page size.
(Worker_TP1 pid=742) INFO 09-11 13:57:47 [interface.py:929] Padding mamba page size by 6.26% to ensure that mamba page size and attention page size are exactly equal.
(Worker_TP1 pid=742) INFO 09-11 13:57:47 [xpu.py:388] [XPU]Setting attention block size to 1024 tokens to ensure multiple of 64, set mamba_page_size_padded to 1048576 bytes accordingly, before was 851968 bytes.
(Worker_TP2 pid=743) INFO 09-11 13:57:47 [interface.py:905] Setting attention block size to 832 tokens to ensure that attention page size is >= mamba page size.
(Worker_TP2 pid=743) INFO 09-11 13:57:47 [interface.py:929] Padding mamba page size by 6.26% to ensure that mamba page size and attention page size are exactly equal.
(Worker_TP2 pid=743) INFO 09-11 13:57:47 [xpu.py:388] [XPU]Setting attention block size to 1024 tokens to ensure multiple of 64, set mamba_page_size_padded to 1048576 bytes accordingly, before was 851968 bytes.
(Worker_TP0 pid=741) INFO 09-11 13:57:47 [gpu_model_runner.py:6445] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(Worker_TP0 pid=741) INFO 09-11 13:58:07 [sym_int4.py:402] sym_int4 linear execution is using the XPU W4A16 kernel.
(Worker_TP0 pid=741) WARNING 09-11 13:58:10 [topk_topp_sampler.py:293] xpu kernel topk_topp_sampler does not support per-request generators. Falling back to PyTorch-native implementation.
(Worker_TP0 pid=741) INFO 09-11 13:58:20 [gpu_worker.py:560] Available KV cache memory: 20.62 GiB
(EngineCore pid=542) INFO 09-11 13:58:20 [kv_cache_utils.py:2177] GPU KV cache size: 1,319,724 tokens
(EngineCore pid=542) INFO 09-11 13:58:20 [kv_cache_utils.py:2178] Maximum concurrency for 262,144 tokens per request: 5.03x
(Worker_TP2 pid=743) INFO 09-11 13:58:43 [qwen_triton_warmup.py:370] Warming up Qwen Triton kernels for model_type=qwen3_5_text.
(Worker_TP3 pid=744) INFO 09-11 13:58:43 [qwen_triton_warmup.py:370] Warming up Qwen Triton kernels for model_type=qwen3_5_text.
(Worker_TP1 pid=742) INFO 09-11 13:58:43 [qwen_triton_warmup.py:370] Warming up Qwen Triton kernels for model_type=qwen3_5_text.
(Worker_TP0 pid=741) INFO 09-11 13:58:43 [qwen_triton_warmup.py:370] Warming up Qwen Triton kernels for model_type=qwen3_5_text.
(Worker_TP1 pid=742) INFO 09-11 13:58:57 [cutedsl_warmup.py:96] Skipping CuTeDSL warmup on non-CUDA platform.
(Worker_TP0 pid=741) INFO 09-11 13:58:57 [cutedsl_warmup.py:96] Skipping CuTeDSL warmup on non-CUDA platform.
(Worker_TP3 pid=744) INFO 09-11 13:58:57 [cutedsl_warmup.py:96] Skipping CuTeDSL warmup on non-CUDA platform.
(Worker_TP1 pid=742) INFO 09-11 13:58:57 [gpu_worker.py:857] Free memory on device (30.56/31.89 GiB) on startup. Desired GPU memory utilization is (0.92, 29.34 GiB). Actual usage is 5.22 GiB for weight, 1.9 GiB for peak activation, 1.46 GiB for non-torch memory, and 0.0 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=22132079084` (20.61 GiB) to fit into requested memory, or `--kv-cache-memory=23443965952` (21.83 GiB) to fully utilize gpu memory. Current kv cache memory in use is 20.76 GiB.
(Worker_TP0 pid=741) INFO 09-11 13:58:57 [gpu_worker.py:857] Free memory on device (30.33/31.89 GiB) on startup. Desired GPU memory utilization is (0.92, 29.34 GiB). Actual usage is 5.22 GiB for weight, 2.07 GiB for peak activation, 1.43 GiB for non-torch memory, and 0.0 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=21978402796` (20.47 GiB) to fit into requested memory, or `--kv-cache-memory=23043972608` (21.46 GiB) to fully utilize gpu memory. Current kv cache memory in use is 20.62 GiB.
(Worker_TP3 pid=744) INFO 09-11 13:58:57 [gpu_worker.py:857] Free memory on device (30.56/31.89 GiB) on startup. Desired GPU memory utilization is (0.92, 29.34 GiB). Actual usage is 5.22 GiB for weight, 1.9 GiB for peak activation, 1.46 GiB for non-torch memory, and 0.0 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=22132103660` (20.61 GiB) to fit into requested memory, or `--kv-cache-memory=23444056064` (21.83 GiB) to fully utilize gpu memory. Current kv cache memory in use is 20.76 GiB.
(Worker_TP2 pid=743) INFO 09-11 13:58:57 [cutedsl_warmup.py:96] Skipping CuTeDSL warmup on non-CUDA platform.
(Worker_TP2 pid=743) INFO 09-11 13:58:57 [gpu_worker.py:857] Free memory on device (30.49/31.89 GiB) on startup. Desired GPU memory utilization is (0.92, 29.34 GiB). Actual usage is 5.22 GiB for weight, 1.9 GiB for peak activation, 1.46 GiB for non-torch memory, and 0.0 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=22132103660` (20.61 GiB) to fit into requested memory, or `--kv-cache-memory=23368284160` (21.76 GiB) to fully utilize gpu memory. Current kv cache memory in use is 20.76 GiB.
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] WorkerProc hit an exception.
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] Traceback (most recent call last):
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 999, in worker_busy_loop
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] output = func(*args, **kwargs)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return func(*args, **kwargs)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 884, in compile_or_warm_up_model
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] self.model_runner._dummy_sampler_run(hidden_states=last_hidden_states)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return func(*args, **kwargs)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 6234, in _dummy_sampler_run
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] logits = self.model.compute_logits(hidden_states)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/model_executor/models/qwen3_vl.py", line 2932, in compute_logits
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return self.language_model.compute_logits(hidden_states)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/model_executor/models/qwen3_5.py", line 368, in compute_logits
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return self.logits_processor(self.lm_head, hidden_states)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return self._call_impl(*args, **kwargs)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return forward_call(*args, **kwargs)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/model_executor/layers/logits_processor.py", line 81, in forward
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] logits = self._get_logits(hidden_states, lm_head, embedding_bias)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/model_executor/layers/logits_processor.py", line 155, in _get_logits
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] logits = self._gather_logits(logits)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/model_executor/layers/logits_processor.py", line 100, in _gather_logits
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] logits = tensor_model_parallel_all_gather(logits)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/distributed/communication_op.py", line 21, in tensor_model_parallel_all_gather
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return get_tp_group().all_gather(input_, dim)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/distributed/parallel_state.py", line 686, in all_gather
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return torch.ops.vllm.all_gather(
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/torch/_ops.py", line 1275, in __call__
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return self._op(*args, **kwargs)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/distributed/parallel_state.py", line 167, in all_gather
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return group._all_gather_out_place(tensor, dim)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/distributed/parallel_state.py", line 695, in _all_gather_out_place
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] return self.device_communicator.all_gather(input_, dim)
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] File "/opt/venv/lib/python3.12/site-packages/vllm/distributed/device_communicators/base_device_communicator.py", line 209, in all_gather
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] output_tensor = torch.empty(
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] ^^^^^^^^^^^^
(Worker_TP1 pid=742) ERROR 09-11 14:20:06 [multiproc_executor.py:1007] torch.OutOfMemoryError: XPU out of memory. Tried to allocate 122.00 MiB. GPU 1 has a total capacity of 31.89 GiB of which 2.56 GiB is free. Of the allocated memory 26.14 GiB is allocated by PyTorch, and 162.84 MiB is reserved by PyTorch but unallocated. Please use `empty_cache` to release all unoccupied cached memory.
```