CUDA: 81 GB q2 quant OOMs at session create on 96 GB GPU (arena fragmentation + q8-f16 cache reserve)

**Summary:** The 80.76 GiB `ds4f-q2` quant fails to start on a 96 GB RTX PRO 6000 Blackwell (`sm_120`) with a storm of `ds4: CUDA tensor alloc failed: out of memory` at session creation — even at `--ctx-alloc 65536`. The memory exists; it's consumed by weight-arena fragmentation plus the q8→fp16 dequant cache before the session's KV/context buffers ever allocate. Measurements and a working env workaround below.

## Environment

- GPU: RTX PRO 6000 Blackwell Workstation 96 GB (97,887 MiB), driver 595.91.07
- Build: `make cuda CUDA_ARCH=sm_120` in `nvidia/cuda:13.0.1-devel-ubuntu24.04` (x86_64, EPYC host, 503 GB RAM)
- Model: `DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf` (80.76 GiB)
- Repro: `./ds4-bench --prompt-file ds4.c --cuda --ctx-alloc 393216 --ctx-start 2048 --ctx-max 32768 --step-mul 4 --gen-tokens 128`
- Note: model mmap is on ZFS, where `cudaHostRegister` returns "operation not supported", so the host-pointer fallbacks can't mask any of this (and `DS4_CUDA_WEIGHT_CACHE_LIMIT_GB` fails with `accelerator failed to prepare model tensor span` for the over-budget spans, since the raw mmap pointer isn't device-visible without registration).

## Measurements (LD_PRELOAD cudaMalloc spy, logging every alloc + `cudaMemGetInfo`)

1. **Weight-arena fragmentation ≈ 6.7 GiB.** `cuda_model_arena_alloc` packs the ~0.6 GiB weight spans first-fit into 1792 MiB chunks; loading 80.76 GiB of spans allocates **50 × 1792 MiB = 87.5 GiB** of arenas. Chunk size doesn't help: with `DS4_CUDA_WEIGHT_ARENA_CHUNK_MB=8192` it's 11 × 8192 MiB = **88.0 GiB**.
2. **q8→fp16 cache grows ~5.9 GiB during startup.** On ≥ 40 GiB cards `cuda_q8_f16_cache_reserve_bytes` reserves only `max(768 MiB, 1 %)`. The cache fills during model preparation, but the session's KV cache and context buffers allocate *after* it — the reserve never accounts for them.
3. Net: **94.0 GiB allocated before session creation; free = 0.14 GiB** at the first failing session alloc (which is a modest 192 MiB). At `--ctx-alloc 393216` the session needs ~10 GiB (`KV 5.23 GiB + buffers 1.50 GiB` planned, plus score/stage buffers), so everything from 65 K up fails identically.

## Workaround that runs

```
DS4_CUDA_NO_FD_CACHE=1 \
DS4_CUDA_Q8_F16_CACHE_RESERVE_MB=12288 \
./ds4-bench ... --ctx-alloc 393216 --prefill-chunk 2048
```

`DS4_CUDA_NO_FD_CACHE=1` falls through to per-range exact-size `cudaMalloc`+copy (no arenas, zero fragmentation, exactly 80.76 GiB resident). Peak VRAM **94,318 MiB**; results:

| ctx tokens | prefill t/s | decode t/s (steady) |
|---|---|---|
| 2,048 | 1,330 | 70.0 |
| 8,192 | 1,989 | 44.6 |
| 32,768 | 1,926 | 41.7 |

Costs of the workaround: model load goes from ~23 s (pinned staging ring, warm page cache) to a few minutes (pageable per-range copies), and the fp16 cache is mostly disabled.

## Suggested directions

- Size arenas from the span list in a pre-pass (span sizes are known before upload), or switch to exact per-span allocations when `model_size / free_vram` is high — the current first-fit binning wastes ~8 % regardless of chunk size.
- Make the q8→fp16 cache reserve account for the upcoming KV + context-buffer allocation (computable from ctx at that point), or defer cache growth until after session creation. The 768 MiB floor is calibrated for weights that leave slack; when the model is > ~85 % of VRAM it guarantees the session loses the race.

Happy to test patches on this hardware.