Alberto-Codes / Alberto-Codes/vramfit

test AutoRound rounding as a within-group method

Open
#545 0 comments 0 reactions 0 assignees View on GitHub
research
Dominant language
Python
Stars
1
Forks
1
Avg merge
6h 8m
Merged PRs (30d)
105

Description

### C11. AutoRound rounding as a within-group method (pack)

- Change: a second pack backend path that exports GGUF through AutoRound
(`--format gguf:q3_k_s`, `iters=200`) for k-quant rows, with the
recipe's per-group types. AutoRound exports one scheme per model or an
AutoScheme mix; whether it accepts an external per-layer `layer_config`
for every layer is the question to check (`layer_config` exists for
`lm_head`).
- Evidence: SignRoundV2 abstract; AutoRound docs "tuning-aware" mode.
- Effort: M to L (a second backend is a port precedent, #105).
- Risk: adds torch to pack; breaks the "pack is a CPU subprocess driver"
rule (ADR-0016 decision 2).
- Experiment: outside vramfit, AutoRound `Q3_K_S` of the 49B vs the
imatrix `Q3_K_S` control (9.655 PPL at the fourth data point) on the
box.

Full evidence and its sources are in the NotebookLM notebook "vramfit method landscape 2026-09".

Contributor guide

Open the contributing guide

Research direction

Start with the pack backend path and compare it with the port precedent in #105, then read ADR-0016 decision 2 to understand the CPU subprocess constraint. Check AutoRound's tuning-aware mode and whether an external per-layer layer_config is supported, then run the outside-vramfit Q3_K_S experiment against the imatrix control on the box. Done means a validated decision on feasibility and an agreed implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.