Alberto-Codes / Alberto-Codes/vramfit
Rented-GPU measurement lane: the 49B target on a 94 GB card, and multi-GPU
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 6h 8m
- Merged PRs (30d)
- 105
Description
Every pipeline step except the final serve-test is hardware-portable: sensitivity is a property of the model, the map and recipe are JSON, the packed GGUF is bytes. The 4090 is slow at measurement only because 93 GB of bf16 weights stream over PCIe from host RAM (73/82 groups offloaded, ~306 s/cell at 65,536 tokens ≈ 25 h/scan, plus OOM-cap gymnastics — three tries to find a working cap for the 64k validation).
**The lane:** rent a card that holds the checkpoint resident — H100 NVL (94 GB) or H200 (141 GB); a plain 80 GB H100 still offloads a sliver of the 93 GB — and run scan / validate / imatrix / evals there. Estimated full loop: an afternoon and ~$10–30 vs ~1.5 days local. Pack and smoke are CPU-only and run anywhere. The artifact must still prove it serves on the target card locally — that is the acceptance test, not a measurement.
Work items before trusting the lane:
- [x] **Instrument check — done for the 30B target (#163, 2026-08-14). The assumption was wrong.** 10 cells re-measured. Same-instrument noise is zero, and the H100 and the 4090 disagree by 0.3% to 10.6% per cell. bf16 math does **not** transfer within noise. #220 ruled the consequence: damage compares only within one instrument, magnitudes never cross, orderings may. Record: [ADR-0027](docs/adr/0027-instrument-frame-matching.md).
- [x] **Fingerprint portability — done for the 30B target (#163, 2026-08-14).** A mirrored absolute layout makes fingerprints portable, and cross-machine checkpoint resume works. One field resists: the CLI resolves `--imatrix` (ADR-0020), so a pod fingerprint records the pod's path there and the box reproduces it by rewriting that one field.
- [x] **Data movement runbook — done for the 30B target (#163, 2026-08-14).** Pull the checkpoint from Hugging Face onto the pod, never through the box: 61.32 GiB in 42 s at ~1.57 GB/s, against 57.2 minutes on the home link. Big files stay pod-side and only small artifacts return.
- [ ] Multi-GPU (2×H100 via `device_map auto`) is unexercised — single big card is the low-risk configuration.
**Scope of those three, stated 2026-08-14.** #163 closed them for the
30B-A3B target, whose 61.32 GiB bf16 checkpoint sits resident on an
80 GB H100. The 49B lane this issue was written for needs 93 GB and
stays unverified. Multi-GPU stays unexercised for both.
The lane has since run production work: #229 drove the whole
runtime-frame gate pod-side for $2.16.
Feeds the volume-and-velocity strategy recorded on #11: the loop speed is what makes zero-day cadence sustainable. The O(groups²) measurement for an interaction-aware solve (see ADR-0006's third measurement) is exactly the workload that wants this lane.
Contributor guide
Research direction
Start with docs/adr/0027-instrument-frame-matching.md and ADR-0020, then review the completed 30B work in #163 and the production run in #229. Run scan, validate, imatrix, and evals for the 49B checkpoint on a resident H100 NVL or H200, then exercise 2×H100 with device_map auto. Done means the lane is verified and the artifact passes the target-card serve-test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- infrastructure, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100