0.26.0-b2: XPU Graph / torch.compile cannot start with sym_int4 — logger.info_once() inside the traced apply() path hard-fails Dynamo
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
## Summary
On `intel/llm-scaler-vllm:0.26.0-b2`, **any compiled execution mode fails at engine
initialisation** when serving a `sym_int4` model. This is not a corruption issue — the server
never reaches graph capture:
```
torch._dynamo.exc.Unsupported: logging.Logger method not supported for non-export cases
Explanation: logging.Logger methods are not supported for non-export cases.
Hint: Add the logging method to `torch._dynamo.config.ignore_logging_functions`.
Developer debug context: method: .info_once
```
`vllm/model_executor/layers/quantization/sym_int4.py` calls `logger.info_once()` **inside the
traced `apply()` forward path** — two call sites in the `int4_gemm_w4a16` branch (around lines
396 and 402 in the shipped file):
```python
if output is not None:
logger.info_once(
"sym_int4 linear execution is using the guarded ESIMD path.", scope="local",
)
return output.reshape(...)
logger.info_once(
"sym_int4 linear execution is using the XPU W4A16 kernel.", scope="local",
)
output = torch.ops._xpu_C.int4_gemm_w4a16(...)
```
TorchDynamo refuses to trace a `logging.Logger` method and raises, aborting startup.
## Impact
This blocks **both** compiled modes, so it is broader than an XPU-Graph issue:
- `VLLM_XPU_ENABLE_XPU_GRAPH=1` (compile + graph) — fails
- compile-only (`cudagraph_mode=NONE`, XPU Graph off, no `--enforce-eager`) — fails
Only `--enforce-eager` starts. That makes it impossible to evaluate compiled performance or to
run a compile-vs-graph correctness comparison on this image without patching.
`TORCHDYNAMO_SUPPRESS_ERRORS=1` does **not** work around it: we verified
`torch._dynamo.config.suppress_errors == True` was live in the container, but the failing path
is `torch/_dynamo/aot_compile.py`, and AOT compilation raises rather than falling back to eager.
## Environment
| | |
|---|---|
| Image | `intel/llm-scaler-vllm:0.26.0-b2` (`sha256:52218ad85513ab6686d4c090c83c2bd8c5b02423c63aa4dabd41837fe641fe3b`) |
| vLLM | `0.26.1.dev0+g568afb3a1.d20260907` |
| torch | `2.12.0+xpu` |
| GPUs | 4x Intel Arc Pro B60 (Battlemage G21, `8086:e211`), 24 GB each |
| Model | `Qwen/Qwen3.6-35B-A3B`, revision `995ad96eacd98c81ed38be0c5b274b04031597b0` |
| Quant | online `sym_int4`, `--dtype float16` |
| TP | reproduces at `-tp 1`, `-tp 2` and `-tp 4` |
| Host | Ubuntu, kernel 7.0.0-31-generic, driver `xe`, Level Zero UMD 26.05.037020 |
## Reproduce
```bash
docker run -d --name repro --net=host --privileged \
--device /dev/dri:/dev/dri -v /dev/dri/by-path:/dev/dri/by-path \
-v /var/lib/models:/models:ro --shm-size 32g \
-e ZE_AFFINITY_MASK=0 -e VLLM_XPU_ENABLE_XPU_GRAPH=1 \
-e VLLM_WORKER_MULTIPROC_METHOD=spawn \
--entrypoint bash intel/llm-scaler-vllm:0.26.0-b2 \
-c "vllm serve /models/.../Qwen3.6-35B-A3B/snapshots/995ad96e... \
--dtype float16 --quantization sym_int4 --allow-deprecated-quantization \
--compilation-config '{\"cudagraph_capture_sizes\":[1,2,4,8,16]}' \
--max-model-len 8192 -tp 1 --max-num-seqs 32 --host 0.0.0.0 --port 8000"
```
Engine init aborts. Same with `--compilation-config '{"cudagraph_mode":"NONE"}'` and
`VLLM_XPU_ENABLE_XPU_GRAPH=0`.
## Suggested fix
Register the custom logger helpers with Dynamo where the module defines its logger — the
mechanism Dynamo's own hint names. Note the methods are attached to the logger **instance**
(`type(logger)` is plain `logging.Logger`), so they must be taken off the instance:
```python
logger = init_logger(__name__)
try:
import torch._dynamo.config as _dynamo_config
for _m in ("info_once", "warning_once", "debug_once", "error_once"):
_bound = getattr(logger, _m, None)
if _bound is None:
continue
_dynamo_config.ignore_logging_functions.add(_bound)
_raw = getattr(_bound, "__func__", None)
if _raw is not None:
_dynamo_config.ignore_logging_functions.add(_raw)
except Exception:
pass
```
It must be done **in-module**, not from the parent process: workers are spawned, so a mutation
made before `vllm serve` never reaches the process that actually compiles.
Alternatively, hoist the two `info_once()` calls out of `apply()` (e.g. log the selected kernel
once at layer construction), which avoids the traced-path call entirely.
## Verification of the workaround
With the registration above applied, on the same image and model:
- graph reaches capture (`Graph capturing finished in 9 secs, took 1.89 GiB`; PIECEWISE and FULL
decode graphs across capture sizes `[1,2,4,8,16]`)
- serves coherently at tp=1 (0/24 degenerate across two 12-run gates at 8k and 32k)
- **1.66-1.67x** output throughput vs a verified-eager control at identical context and
`max_num_seqs` (196.23 -> 328.01 tok/s at 8k; 198.10 -> 328.42 at 32k, c=8, in=512/out=256)
## Separate note (filed here only for context, not as part of this report)
Graph capture additionally requires `max_num_seqs` <= available mamba cache blocks:
```
ValueError: max_num_seqs (256) exceeds available Mamba cache blocks (44).
```
The default `max_num_seqs=256` therefore fails capture out of the box on this model at 24 GB.
Happy to file that separately if useful.
Contributor guide
Assessment
This issue has not been assessed yet.