AlexsJones / AlexsJones/llmfit
Feature: disk/SSD planning for multi-model libraries (keep N models that fit)
- Dominant language
- Rust
- Stars
- 36.3k
- Forks
- 2.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 85
Description
## Summary
`llmfit fit` / `info` already expose `disk_size_gb` for the chosen quant, but there is no first-class way to answer:
> Given this machine (or `--memory` / `--ram` simulation), how much **internal SSD** do I need to comfortably download and keep N models that fit, plus download scratch?
That is a real buy/config decision for local LLM desktops (Mac mini / Mac Studio, etc.): RAM decides what loads; disk decides whether you can switch between models without delete-and-rewait.
## What works today
- `disk_size_gb` on fit / recommend / info JSON rows
- `llmfit plan` for VRAM/RAM/context/quant capacity planning
## What is missing
1. **`plan` does not surface disk/weight storage** for the planned quant (no `disk_size_gb` / `weight_gb` in the plan JSON I get today — only VRAM/RAM/KV paths).
2. No command or flag to aggregate storage across **multiple models that fit** a machine, e.g.:
- keep the top N fitting models on disk
- plus OS/apps reserve
- plus scratch equal to one more download
- round up to common SSD SKUs (512GB / 1TB / 2TB / 4TB …)
3. No guidance in docs that `disk_size_gb` is the field to use for library sizing (easy to miss; buyers still guess).
Related but different: #542 (server density / concurrent instances). This issue is about **on-disk library size for sequential switching**, not concurrent serving density.
## Proposed UX (sketch)
```bash
# Single model: include disk in plan output
llmfit plan "openai/gpt-oss-120b" --context 8192 --json
# → add disk_size_gb / weight_on_disk_gb for the planned quant
# Multi-model library sizing against simulated hardware
llmfit storage --memory 128G --ram 128G --cpu-cores 18 \
--keep 3 --os-reserve 100G --json
# → models considered = those that fit; report need_gb + suggested_ssd_gb
```
Optional filters: `--perfect`, model list / search query, exclude Too Tight.
## Why it matters
Without this, people configure Macs (and other local boxes) with enough unified memory but undersized SSDs, then discover a 120B-class weight is ~100GB+ on disk and three of them blow past a 512GB drive after the OS.
## Workaround
We currently post-process fit dumps in
https://github.com/saman-mb/mac-studio-m5-analysis/blob/main/scripts/disk_comfort.py
using llmfit’s `disk_size_gb`. Happy to adapt that formula if you want it upstream.
## Environment
- llmfit via pipx (`llmfit --help` shows fit JSON includes `disk_size_gb`)
- Use case: Apple Silicon desktop config planning (M6 mini / M5 Studio), 13 simulated configs via `--memory` / `--ram` / `--cpu-cores`
Contributor guide
Assessment
This issue has not been assessed yet.