Alberto-Codes / Alberto-Codes/vramfit

Emit llama-quantize --tensor-type flags from a published map

Open
#271 2 comments 0 reactions 0 assignees View on GitHub
enhancement publication
Dominant language
Python
Stars
1
Forks
1
Avg merge
6h 8m
Merged PRs (30d)
105

Description

## What

Ship a standalone converter that emits `llama-quantize --tensor-type`
flags, so a llama.cpp user benefits from a published map without
adopting vramfit. Graduated from #11's box 4 on 2026-08-15, because
the trigger fired and the work had no ticket.

## Why now

Box 4's trigger reads "≥2 published maps". **Eight** ship at
[the maps dataset](https://huggingface.co/datasets/Alberto-Codes/Llama-3_3-Nemotron-Super-49B-v1_5-sensitivity-maps),
so it fired on 2026-08-12 when #85 closed. It does not depend on #270,
the registry.

#256 sharpens the case. Every published sub-4-bit GGUF of the 30B
target packs its routed experts at 4.5 bits per weight, because
`tensor_type_fallback` rewrites a k-quant the stack rows cannot take
and exits zero. bartowski's `IQ2_XXS` and unsloth's `UD-IQ1_M` both
carry IQ4_NL on all 46 backbone stacks. The whole low-bit shelf for
that model is mislabelled, and `--tensor-type` is the flag that fixes
it. Handing those flags to llama.cpp users is the artifact-as-standard
move box 4 names.

## Most of the translation already exists

`vramfit pack` builds the argv today at
`src/vramfit/adapters/outbound/gguf/pack.py:250`:

```python
command += ["--tensor-type", f"{override.pattern}={override.quant_type}"]
```

`tensor_overrides`, `ggml_type_for`, `base_type`,
`token_embedding_type`, and `output_tensor_type` in
`adapters/outbound/gguf/types.py` do the mapping, including ADR-0028's
expert-stack table and the layer-naming families from #180. This
issue is a **surface**, not new mapping logic.

There is no dry-run or print-argv flag on `pack` today.

## The box elides a step, and this issue must not

Box 4 says "map → `--tensor-type` converter". A map cannot produce
flags on its own. It records damage per group per precision. Choosing
a precision needs a budget and a solve, which is `vramfit plan`.

So the real shape is one of two, and picking is part of this ticket:

- **Recipe to flags.** Thin, and close to a `--print-argv` on `pack`.
The user runs `plan` first, which is torch-free (ADR-0011
amendment), so the whole path stays off the GPU stack.
- **Map plus budget to flags.** One command that solves and prints.
Friendlier to someone who has never heard of a recipe, and it
duplicates `plan`'s interface.

## The trap a standalone converter must not set

`vramfit pack` scans the quantizer's output for the type-fallback
warning and halts (ADR-0028 decision 3). A user who takes the emitted
flags and runs `llama-quantize` themselves **loses that guard**. They
get the exact silent substitution #256 documents, from a tool that
handed them the flags.

Whatever ships must either carry the warning in its output or state
the check the user has to run. Emitting flags with no such note would
make this project a source of the defect it just measured.

Related: #252 is open on the halt's own reliability, and #260's
review found the decode path is how that scan is reached at all.

## What closing looks like

A llama.cpp user with a published map and a budget can obtain a
correct `--tensor-type` argument list without installing the scan
extra, and the output says what guard they are not getting.

## Related

- #11 box 4 — the phase this graduated from. Its trigger is marked
fired.
- #270 — the registry, box 3's second half. Independent of this.
- #256 — the measurement that makes this worth shipping.
- ADR-0012, ADR-0028 — the mapping this would expose.

Contributor guide

Open the contributing guide

Research direction

Start with vramfit pack in src/vramfit/adapters/outbound/gguf/pack.py and the mapping helpers in adapters/outbound/gguf/types.py, then read the torch-free vramfit plan flow and ADR-0012/ADR-0028. Decide whether the surface consumes a recipe or a map plus budget, and specify how the output communicates the missing fallback-warning guard. Done means a user can obtain the argument list from a published map and budget with that guard limitation stated.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.