CUDA ssd-streaming: expert cache thrashes on mixed-quant GGUFs (decode ~0.9 t/s); prefill drops the whole cache every request

## Environment

- GPU: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation 96GB (sm_120)
- Driver 595.71.05, CUDA runtime 13.2, toolkit 13.1 (`make cuda-generic`)
- Host: EPYC 9124, 180 GiB RAM (VM, GPU passthrough), Ubuntu 26.04
- ds4 commit `80ebbc3`
- Model: official `DeepSeek-V4-Flash-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-fixed.gguf` (q2-q4 mixed, 91GB)

Command:

```bash
DS4_CUDA_NO_DIRECT_IO=1 DS4_CUDA_KEEP_MODEL_PAGES=1 ./ds4-server \
-m --ctx 32768 --host 0.0.0.0 --prefill-chunk 2048 \
--ssd-streaming --ssd-streaming-cache-experts 4500 \
--kv-disk-dir /data/ds4-kv --kv-disk-space-mb 16384
```

## Symptom

Decode is stuck at **~0.9 t/s** (never ramps up) and stderr floods with thousands of
alternating cap notices, two sizes flip-flopping forever:

```
ds4: CUDA streaming expert cache capped from 10619 to 4973 experts (available 81.57 GiB, reserve 16.00 GiB, 13.50 MiB/expert)
ds4: CUDA streaming expert cache capped from 10619 to 9947 experts (available 81.57 GiB, reserve 16.00 GiB, 6.75 MiB/expert)
ds4: CUDA streaming expert cache capped from 10619 to 4973 experts (available 81.57 GiB, reserve 16.00 GiB, 13.50 MiB/expert)
... (repeats for the whole run)
```

A 200-word completion does not finish within 400s. The same setup with the uniform
q2-imatrix or q4-imatrix GGUFs works fine.

## Root cause 1 — single-size expert cache released at every size transition

`ds4_cuda.cu` keeps ONE global `g_stream_expert_cache` keyed on the last-seen
`(gate_expert_bytes, down_expert_bytes)`:

- `cuda_stream_expert_cache_note_size()` resets the runtime cap whenever the size
differs from the previous call;
- `cuda_stream_expert_cache_prepare()` calls `release_all()` when dims differ from
the cached ones.

The mixed GGUF interleaves layers with 6.75 MiB and 13.50 MiB experts. The slab/bypass
logic in `ds4.c` (`weights_streaming_layer_experts_uniform`) covers the graph path, but
the CUDA cache layer still receives both sizes (e.g. via
`ds4_gpu_stream_expert_cache_budget_for_expert_size` and the seed/compact-load paths),
so the cache is torn down and rebuilt at every q2↔q4 layer transition — it never
accumulates a single hit.

## Root cause 2 — prefill drops the whole resident cache on every request

`ds4_gpu_stream_expert_cache_prepare_selected_batch` →
`cuda_stream_selected_cache_begin_compact_load(..., allow_global_cache=0)` executes:

```c
if (!allow_global_cache) {
cuda_stream_expert_cache_release_all();
}
```

unconditionally — i.e. **every prompt releases the entire warm expert cache**, even for
a 20-token chat prompt whose staging buffers are ~100 MB. Every request then re-warms
the cache from zero during decode (visible as a chunk-rate ramp 1.7 → 11 t/s on every
single request). This affects uniform models too, not just mixed ones.

## Fix (see PR)

1. Expert caches per size class (small fixed array keyed by `(gate,down)` bytes) —
removes the cross-class teardown entirely;
2. `begin_compact_load`: try the staging allocation first, release the expert cache
only if VRAM is actually short (then retry once);
3. Let the prefill batch path read the global cache (hits become device-to-device
copies) with an append-but-never-evict policy, so a long prompt cannot cycle out
the decode working set.

## Results (same GPU, same mixed GGUF)

| build | 200-word completion |
|---|---|
| stock `80ebbc3` | does not finish in 400s (~0.9 t/s, log flood) |
| patched, `--ssd-streaming-cache-experts 7500`, `DS4_CUDA_STREAMING_EXPERT_CACHE_RESERVE_GB=8` | ~12.6 t/s end-to-end, `DS4_CUDA_STREAMING_EXPERT_CACHE_VERBOSE` shows **97% hit rate** on the 2nd request, no cap-flood |
| patched, `--ssd-streaming-cache-experts 9472` (= all uniform-class experts resident, 62 GiB), reserve 8 | **~15 t/s end-to-end** after convergence |

Caveat found while tuning: the per-class budget can overcommit — a count budget of N
lets EVERY class try N slots, and the lazily-growing model-range arena (q8_0 chunks,
~1.8 GiB each) can then fail mid-decode (`cuda decode failed`). A shared byte budget
across classes (or reserving the arena's projected size upfront) would make sizing
safer; for now the practical rule on 96GB with this GGUF is: budget = uniform-class
expert count (9472), reserve ≥ 8 GB.

Uniform q2/q4 GGUFs behave as before (q2 fully-resident path untouched).

Remaining ceiling on mixed is the per-token device-to-device compact-buffer copies
(~2 GB/token even at 97% hit rate) — that seems related to the direction discussed in
#491 (serving experts directly from cache slots), out of scope here.

Reproduce the mixed-GGUF behavior with the supplied ds4-server command, then trace ds4_cuda.cu through cuda_stream_expert_cache_note_size(), cuda_stream_expert_cache_prepare(), and cuda_stream_selected_cache_begin_compact_load(). Check the related paths in ds4.c and ds4_gpu_stream_expert_cache_prepare_selected_batch. Done means mixed sizes no longer trigger repeated teardown, prefill preserves the warm cache when staging fits, and the reported benchmark shows no cap flood with improved hit rate.

Rédigé par le modèle d'indexation à partir du texte de l'issue.