Alberto-Codes / Alberto-Codes/vramfit
test AutoRound rounding as a within-group method
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 6h 8m
- Merged PRs (30d)
- 105
Description
### C11. AutoRound rounding as a within-group method (pack)
- Change: a second pack backend path that exports GGUF through AutoRound
(`--format gguf:q3_k_s`, `iters=200`) for k-quant rows, with the
recipe's per-group types. AutoRound exports one scheme per model or an
AutoScheme mix; whether it accepts an external per-layer `layer_config`
for every layer is the question to check (`layer_config` exists for
`lm_head`).
- Evidence: SignRoundV2 abstract; AutoRound docs "tuning-aware" mode.
- Effort: M to L (a second backend is a port precedent, #105).
- Risk: adds torch to pack; breaks the "pack is a CPU subprocess driver"
rule (ADR-0016 decision 2).
- Experiment: outside vramfit, AutoRound `Q3_K_S` of the 49B vs the
imatrix `Q3_K_S` control (9.655 PPL at the fourth data point) on the
box.
Full evidence and its sources are in the NotebookLM notebook "vramfit method landscape 2026-09".
Contributor guide
Research direction
Start with the pack backend path and compare it with the port precedent in #105, then read ADR-0016 decision 2 to understand the CPU subprocess constraint. Check AutoRound's tuning-aware mode and whether an external per-layer layer_config is supported, then run the outside-vramfit Q3_K_S experiment against the imatrix control on the box. Done means a validated decision on feasibility and an agreed implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100