[Performance] CUDA MatMulNBits M=1 GEMV is grid-starved at decode shapes: 16-62% of HBM bandwidth

### Summary

The CUDA int4 `MatMulNBits` decode kernel (`MatMulFloatInt4Kernel`, the M=1 path) launches a 1-D grid of `N / 8` thread blocks with no K-splitting. At the `N` values that occur in transformer decode (1024-8192) this leaves only 1-9 CTAs per SM on an H100, far too few to hide HBM latency. The same kernel reaches 95% of achievable bandwidth when the grid is wide.

This kernel is **68.7% of all GPU kernel time** in int4 batch-1 decode — 1.958 ms/token of a 2.802 ms/token step (Phi-4-mini-instruct int4, batch 1, prompt 128, H100 PCIe, CUDA graphs enabled).

### Measurements

H100 PCIe 80GB (114 SM, 52.4 MB L2), driver 580.126.09 / CUDA 13.0, `onnxruntime-gpu` 1.29.0 stock wheel.
Achievable bandwidth measured on this card: **1808 GB/s** (536 MB D2D copy).

Kernel durations read from Nsight Systems, not wall-clock, with weight working sets sized to ~120 MB so they do not sit in L2.

| K | N | CTAs | CTA/SM | kernel us | GB/s | % of 1808 |
|---|---|---|---|---|---|---|
| 3072 | 1024 | 128 | 1.12 | 6.27 | 282 | 15.6% |
| 3072 | 3072 | 384 | 3.37 | 7.01 | 757 | 41.9% |
| 3072 | 5120 | 640 | 5.61 | 8.26 | 1072 | 59.3% |
| 3072 | 8192 | 1024 | 8.98 | 13.98 | 1012 | 56.0% |
| 8192 | 3072 | 384 | 3.37 | 12.64 | 1120 | 61.9% |
| 3072 | 200064 | 25008 | 219.4 | 201.89 | **1712** | **94.7%** |

Efficiency tracks CTAs/SM monotonically. Every `N` in a decoder layer sits in the starved regime; only the LM head (N = vocab) is wide enough to saturate.

### Root cause

https://github.com/microsoft/onnxruntime/blob/ff8d83b82d87a0b019e40a8c0ff11621a7caf39b/onnxruntime/contrib_ops/cuda/quantization/matmul_4bits_m1_impl.cuh#L135-L136

Parallelism is derived solely from `N`. As GPUs get wider (114 SM on H100, more on B200) the decode shapes fall further behind.

### Split-K: tried, negative

Two split-K mechanisms benchmarked against the real kernel (weights extracted verbatim from `matmul_4bits.cu`, real PTX dequant path), median of 200 CUDA-event-timed runs, L2-cold, on a 4x H100 80GB HBM3 (132 SM, sm_90):

| shape | split | baseline us | atomic split-K | fused (grid.sync) |
|---|---|---|---|---|
| down (K=8192,N=3072) | 2 | 15.14 | 0.77x | 0.92x |
| o_proj (K=3072,N=3072) | 2 | 10.14 | 0.66x | 0.78x |
| kv (K=3072,N=1024) | 5 | 9.66 | 0.85x | 0.86x |

Both are net negative on every shape. Atomic split-K adds two dependent kernel launches (memset + reduce) on top of the split compute; fusing all three into one `cudaLaunchCooperativeKernel` + `grid.sync()` recovers about half the deficit but still doesn't clear 1x — at 6-15us baseline durations, the barrier and per-slice setup cost more than the added occupancy buys back. Correctness held in both (~5-6e-4 max relative error) — this is a genuine performance result, not a harness bug.

I don't have a further mechanism to try that stays within this kernel's structure. A fix likely needs to attack occupancy from outside the kernel (e.g. fusing multiple decode GEMVs or layers into fewer launches) rather than splitting K within one.

### Relation to #32240

#32240 targets the same regime (gated below 8 CTAs/SM) but adds explicit prefetching, measured at 1.01x on an RTX A1000 (18 SM) at N=256/1024, K=4096.

The data above suggests the limiter at decode shapes is insufficient parallelism rather than load latency, and that the regime is worth considerably more than 1% on server GPUs. Happy to re-run any of the above on request.

Start with onnxruntime/contrib_ops/cuda/quantization/matmul_4bits_m1_impl.cuh around lines 135-136 and inspect how the M=1 grid is derived from N. Review the reported split-K measurements and relation to #32240 before proposing an alternative. The issue does not define an implementable fix or acceptance test; done would require a measured improvement without regressing correctness.

Written by the indexing model from the issue text.