ssd-streaming: --ssd-streaming-cache-experts budget not enforced (working set exceeds declared cap ~2x on cold expert-set transition)
- 主要言語
- C
- スター
- 22.3k
- フォーク
- 2.1k
- 平均マージ
- 1日 3時間
- マージ済み PR(30日)
- 4
説明
**Repo:** antirez/ds4 (reproduced on fork philmcneely/ds4 @ HEAD 84cc882 == upstream/main HEAD)
**Path affected:** SSD-streaming expert cache. **Whole-model-resident path is UNAFFECTED** (it honors its declared budget exactly).
## Environment
- Mac Studio M3 Ultra, 256GB unified, macOS 26.2
- Model: DeepSeek-V4-Flash MXFP4 GGUF (`...chat-v2-mxfp4-0731.gguf`, 145.26 GiB)
- ds4-server, default build (`make -j`), zero warnings
## Repro
1. Start `ds4-server` with an explicit hard cap: `--ssd-streaming-cold --ssd-streaming-cache-experts 20GB`, context 8192. Server prints planned budget ~21.47 GiB.
2. Prompt 1 (reasoning): answers correctly, memory stays flat — OK.
3. Prompt 2 (code-gen, needs a DIFFERENT expert subset than prompt 1): free memory craters ~50GB -> ~11GB in ~12s — roughly **2x the declared 20GB cap** — before an external killswitch fired.
- Log shows a cache miss at the transition: `live kv cache miss ... reason=token-mismatch`.
## Expected
Expert cache evicts to stay at/under the declared `--ssd-streaming-cache-experts` budget.
## Actual
On the first cold -> different-expert-set transition, the working set grows past the declared budget instead of evicting, driving toward OOM. (Auto-budget mode without an explicit cap is worse: planned 138.91 GiB, blew 97GB->9GB free on the FIRST 28-token prompt.)
## Notes
- Growth landed in reclaimable/mmap-backed pages (wired stayed ~3GB), so it may be less catastrophic than a true wired OOM — but it consumes real free memory fast and unpredictably.
- The production whole-model-resident path (no `--ssd-streaming`) does NOT exhibit this — it loaded 145GiB, honored its 146GiB plan, and stayed elastic under 4 concurrent + long-context load.
- Logs available on request (server.log, server2.log, killswitch.log, results.json) — happy to attach if useful.
コントリビューションガイド
評価
この issue はまだ評価されていません。