antirez / antirez/ds4

ssd-streaming: --ssd-streaming-cache-experts budget not enforced (working set exceeds declared cap ~2x on cold expert-set transition)

Aperta
#802 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C
Stelle
22.3k
Fork
2.1k
Merge medio
1g 3h
PR unite (30g)
4

Descrizione

**Repo:** antirez/ds4 (reproduced on fork philmcneely/ds4 @ HEAD 84cc882 == upstream/main HEAD)
**Path affected:** SSD-streaming expert cache. **Whole-model-resident path is UNAFFECTED** (it honors its declared budget exactly).

## Environment
- Mac Studio M3 Ultra, 256GB unified, macOS 26.2
- Model: DeepSeek-V4-Flash MXFP4 GGUF (`...chat-v2-mxfp4-0731.gguf`, 145.26 GiB)
- ds4-server, default build (`make -j`), zero warnings

## Repro
1. Start `ds4-server` with an explicit hard cap: `--ssd-streaming-cold --ssd-streaming-cache-experts 20GB`, context 8192. Server prints planned budget ~21.47 GiB.
2. Prompt 1 (reasoning): answers correctly, memory stays flat — OK.
3. Prompt 2 (code-gen, needs a DIFFERENT expert subset than prompt 1): free memory craters ~50GB -> ~11GB in ~12s — roughly **2x the declared 20GB cap** — before an external killswitch fired.
- Log shows a cache miss at the transition: `live kv cache miss ... reason=token-mismatch`.

## Expected
Expert cache evicts to stay at/under the declared `--ssd-streaming-cache-experts` budget.

## Actual
On the first cold -> different-expert-set transition, the working set grows past the declared budget instead of evicting, driving toward OOM. (Auto-budget mode without an explicit cap is worse: planned 138.91 GiB, blew 97GB->9GB free on the FIRST 28-token prompt.)

## Notes
- Growth landed in reclaimable/mmap-backed pages (wired stayed ~3GB), so it may be less catastrophic than a true wired OOM — but it consumes real free memory fast and unpredictably.
- The production whole-model-resident path (no `--ssd-streaming`) does NOT exhibit this — it loaded 145GiB, honored its 146GiB plan, and stayed elastic under 4 concurrent + long-context load.
- Logs available on request (server.log, server2.log, killswitch.log, results.json) — happy to attach if useful.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.