perf(PERF-THOR-DECODE-PARITY): bring Qwen decode to vLLM parity on NVIDIA Thor

Row: PERF-THOR-DECODE-PARITY

This is the current live owner for the Thor decode-parity campaign. It supersedes historical issue references that no longer resolve after the repository issue reset; those old identifiers remain only as immutable artifact provenance.

Target

On thor:gpu0 (NVIDIA Thor / CUDA sm_110), measure and improve vllm.cpp decode for Qwen3-4B and Qwen3.6-35B-A3B or the closest supported equivalent. Use pinned vLLM as the primary oracle and llama.cpp only where no equivalent vLLM arm is available.

Required matrix: input lengths 16 and 1024, concurrency 1/4/8, matched correctness and profiler windows. LocalAI and Kubernetes are excluded because the rc lease removes the LocalAI worker. All GPU work must run directly through rc; do not combine rc with flock.

Current finding

The earlier Qwen3-4B observation was about 2.5 tok/s through LocalAI. An initial direct trace was invalid for parity because synchronous profiler start captured idle/downclock transition. The campaign now has delayed profiler activation matching pinned vLLM semantics: warm the matching concurrency, arm at delay 257, execute an exact 256-step c1 primer, then activate immediately before the timed engine execution. Host implementation is under fresh review before integration.

Acceptance

• Valid matched vLLM/vllm.cpp traces with one delayed CUDA range, no primer kernels in-range, stable clocks, and matching token/correctness evidence.
• Throughput parity with pinned vLLM at i16/i1024 and c1/c4/c8; if an equivalent vLLM arm is unavailable, use pinned llama.cpp.
• Trace-proven optimization only; no architecture guesswork.
• Inspect Triton AOT reachability for sm_110 and other omitted CUDA architectures. Enable an AOT target only if the Qwen3.6/GDN trace proves it material, with portable fallback and exact embedded-image verification. Dense Qwen3-4B does not by itself justify a GDN AOT change.
• Clean host gates, sm_110 warnings-as-errors build, correctness gates, immutable evidence bundle, fresh review, operator verification, then local merge to current main. No push unless separately authorized.

Integration note

Current main has advanced since the campaign branch. Reconciliation must preserve current Qwen graph predicates, request-boundary state, model/device handling, and GDN chunked-prefill behavior while adding only the reviewed Thor instrumentation and measured optimizations.

FOLLOWING_AGENTS_PROTOCOL

Start on thor:gpu0 through rc and reproduce the matched pinned-vLLM/vllm.cpp Qwen3-4B and Qwen3.6-35B-A3B matrix at input lengths 16/1024 and concurrency 1/4/8. Review current main's Qwen graph predicates, request-boundary state, model/device handling, and GDN chunked-prefill behavior before examining the delayed profiler sequence. Done requires valid traces, correctness and throughput parity, clean gates, verified evidence, fresh review, and operator verification.