antirez / antirez/ds4

ssd-streaming: --ssd-streaming-cache-experts budget not enforced (working set exceeds declared cap ~2x on cold expert-set transition)

Offen
#802 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C
Sterne
22.3k
Forks
2.1k
Ø Merge
1 T. 3 Std.
Gemergte PRs (30 T.)
4

Beschreibung

**Repo:** antirez/ds4 (reproduced on fork philmcneely/ds4 @ HEAD 84cc882 == upstream/main HEAD)
**Path affected:** SSD-streaming expert cache. **Whole-model-resident path is UNAFFECTED** (it honors its declared budget exactly).

## Environment
- Mac Studio M3 Ultra, 256GB unified, macOS 26.2
- Model: DeepSeek-V4-Flash MXFP4 GGUF (`...chat-v2-mxfp4-0731.gguf`, 145.26 GiB)
- ds4-server, default build (`make -j`), zero warnings

## Repro
1. Start `ds4-server` with an explicit hard cap: `--ssd-streaming-cold --ssd-streaming-cache-experts 20GB`, context 8192. Server prints planned budget ~21.47 GiB.
2. Prompt 1 (reasoning): answers correctly, memory stays flat — OK.
3. Prompt 2 (code-gen, needs a DIFFERENT expert subset than prompt 1): free memory craters ~50GB -> ~11GB in ~12s — roughly **2x the declared 20GB cap** — before an external killswitch fired.
- Log shows a cache miss at the transition: `live kv cache miss ... reason=token-mismatch`.

## Expected
Expert cache evicts to stay at/under the declared `--ssd-streaming-cache-experts` budget.

## Actual
On the first cold -> different-expert-set transition, the working set grows past the declared budget instead of evicting, driving toward OOM. (Auto-budget mode without an explicit cap is worse: planned 138.91 GiB, blew 97GB->9GB free on the FIRST 28-token prompt.)

## Notes
- Growth landed in reclaimable/mmap-backed pages (wired stayed ~3GB), so it may be less catastrophic than a true wired OOM — but it consumes real free memory fast and unpredictably.
- The production whole-model-resident path (no `--ssd-streaming`) does NOT exhibit this — it loaded 145GiB, honored its 146GiB plan, and stayed elastic under 4 concurrent + long-context load.
- Logs available on request (server.log, server2.log, killswitch.log, results.json) — happy to attach if useful.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.