antirez / antirez/ds4

ssd-streaming: --ssd-streaming-cache-experts budget not enforced (working set exceeds declared cap ~2x on cold expert-set transition)

Đang mở
#802 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

**Repo:** antirez/ds4 (reproduced on fork philmcneely/ds4 @ HEAD 84cc882 == upstream/main HEAD)
**Path affected:** SSD-streaming expert cache. **Whole-model-resident path is UNAFFECTED** (it honors its declared budget exactly).

## Environment
- Mac Studio M3 Ultra, 256GB unified, macOS 26.2
- Model: DeepSeek-V4-Flash MXFP4 GGUF (`...chat-v2-mxfp4-0731.gguf`, 145.26 GiB)
- ds4-server, default build (`make -j`), zero warnings

## Repro
1. Start `ds4-server` with an explicit hard cap: `--ssd-streaming-cold --ssd-streaming-cache-experts 20GB`, context 8192. Server prints planned budget ~21.47 GiB.
2. Prompt 1 (reasoning): answers correctly, memory stays flat — OK.
3. Prompt 2 (code-gen, needs a DIFFERENT expert subset than prompt 1): free memory craters ~50GB -> ~11GB in ~12s — roughly **2x the declared 20GB cap** — before an external killswitch fired.
- Log shows a cache miss at the transition: `live kv cache miss ... reason=token-mismatch`.

## Expected
Expert cache evicts to stay at/under the declared `--ssd-streaming-cache-experts` budget.

## Actual
On the first cold -> different-expert-set transition, the working set grows past the declared budget instead of evicting, driving toward OOM. (Auto-budget mode without an explicit cap is worse: planned 138.91 GiB, blew 97GB->9GB free on the FIRST 28-token prompt.)

## Notes
- Growth landed in reclaimable/mmap-backed pages (wired stayed ~3GB), so it may be less catastrophic than a true wired OOM — but it consumes real free memory fast and unpredictably.
- The production whole-model-resident path (no `--ssd-streaming`) does NOT exhibit this — it loaded 145GiB, honored its 146GiB plan, and stayed elastic under 4 concurrent + long-context load.
- Logs available on request (server.log, server2.log, killswitch.log, results.json) — happy to attach if useful.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.