AlexsJones / AlexsJones/llmfit

Feature: disk/SSD planning for multi-model libraries (keep N models that fit)

Open
#970 1 comment 0 reactions 0 assignees View on GitHub
enhancement Feature Request
Dominant language
Rust
Stars
36.3k
Forks
2.3k
Avg merge
2d 18h
Merged PRs (30d)
85

Description

## Summary

`llmfit fit` / `info` already expose `disk_size_gb` for the chosen quant, but there is no first-class way to answer:

> Given this machine (or `--memory` / `--ram` simulation), how much **internal SSD** do I need to comfortably download and keep N models that fit, plus download scratch?

That is a real buy/config decision for local LLM desktops (Mac mini / Mac Studio, etc.): RAM decides what loads; disk decides whether you can switch between models without delete-and-rewait.

## What works today

- `disk_size_gb` on fit / recommend / info JSON rows
- `llmfit plan` for VRAM/RAM/context/quant capacity planning

## What is missing

1. **`plan` does not surface disk/weight storage** for the planned quant (no `disk_size_gb` / `weight_gb` in the plan JSON I get today — only VRAM/RAM/KV paths).
2. No command or flag to aggregate storage across **multiple models that fit** a machine, e.g.:
- keep the top N fitting models on disk
- plus OS/apps reserve
- plus scratch equal to one more download
- round up to common SSD SKUs (512GB / 1TB / 2TB / 4TB …)
3. No guidance in docs that `disk_size_gb` is the field to use for library sizing (easy to miss; buyers still guess).

Related but different: #542 (server density / concurrent instances). This issue is about **on-disk library size for sequential switching**, not concurrent serving density.

## Proposed UX (sketch)

```bash
# Single model: include disk in plan output
llmfit plan "openai/gpt-oss-120b" --context 8192 --json
# → add disk_size_gb / weight_on_disk_gb for the planned quant

# Multi-model library sizing against simulated hardware
llmfit storage --memory 128G --ram 128G --cpu-cores 18 \
--keep 3 --os-reserve 100G --json
# → models considered = those that fit; report need_gb + suggested_ssd_gb
```

Optional filters: `--perfect`, model list / search query, exclude Too Tight.

## Why it matters

Without this, people configure Macs (and other local boxes) with enough unified memory but undersized SSDs, then discover a 120B-class weight is ~100GB+ on disk and three of them blow past a 512GB drive after the OS.

## Workaround

We currently post-process fit dumps in
https://github.com/saman-mb/mac-studio-m5-analysis/blob/main/scripts/disk_comfort.py
using llmfit’s `disk_size_gb`. Happy to adapt that formula if you want it upstream.

## Environment

- llmfit via pipx (`llmfit --help` shows fit JSON includes `disk_size_gb`)
- Use case: Apple Silicon desktop config planning (M6 mini / M5 Studio), 13 simulated configs via `--memory` / `--ram` / `--cpu-cores`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.