gpu_memory_utilization does not account for the DFlash2 draft speculative context, which is 1.25 GiB at max_num_seqs 8

Row: -

Owed by .agents/specs/bench-qwen38-exl3-variadic.md under ## Owed.

gpu_memory_utilization is the knob an operator sizes a deployment with. On a
speculative DFlash2 configuration it does not account for the draft speculative
context, and the engine says so in its own startup line rather than in the
number the operator set.

Measured on dgx:gpu0 during the variadic serving run
(#2970), rc job
7b5084ab-f214-4d8d-b1fe-1eca86efb1e8, tree 3351ec54f, serving
Mia-AiLab/Qwen3.8-27B-EXL3-3.5bpw with its DFlash2 draft at
num_speculative_tokens: 7, --max-model-len 8192 --num-blocks 2048 --max-num-seqs 8:

vllm.cpp: draft speculative context is limited to 8208 tokens (513 pages x 16,
160.312 MiB per concurrent request across 5 draft layers, so 1282.5 MiB of
device memory at max_num_seqs 8 — NOT counted by gpu_memory_utilization).

src/vllm/v1/worker/gpu/runner.cpp:4090-4098 prints it and its comment states
the reason plainly: one store is built per batch row, so the device holds
bytes_per_request * max_num_reqs and gpu_memory_utilization accounts none of
it.

Why this is more than a log line

The term scales with concurrency and with max_model_len, and both are knobs an
operator raises to serve more traffic. At max_num_seqs 8 it is 1.25 GiB. At 32
it would be 5 GiB, and at a longer context it grows again — none of it inside the
fraction the operator believes bounds the engine's device memory.

Two consequences that show up as something other than a memory bug:

• An operator who sets gpu_memory_utilization to fit a box gets an engine that
  exceeds it, and the overrun grows with the concurrency they configure.
• A concurrency ladder that does not know this term exists measures it as noise.
  This run pinned --num-blocks explicitly for exactly that reason.

Ask

Account for the draft speculative context inside gpu_memory_utilization, the
way the KV pool is accounted, or state in the flag's own documentation which
allocations it does not bound. Mirror whatever vLLM does for its own speculative
proposer state.

The startup line is already correct and already names the aggregate. What is
missing is that the number the operator sets does not include it.

Refs #2970

Start at src/vllm/v1/worker/gpu/runner.cpp:4090-4098 and inspect how gpu_memory_utilization and the KV pool size are calculated. Reproduce the reported DFlash2 configuration, then trace how draft speculative context scales with max_num_seqs and max_model_len. Done means the allocation is included in the operator's bound, or the flag documentation explicitly states the omission.