Alberto-Codes / Alberto-Codes/vramfit
Does a fine-tune inherit its base checkpoint's sensitivity map?
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 6h 8m
- Merged PRs (30d)
- 105
Description
## Question
Does a fine-tune inherit its base checkpoint's sensitivity map? If the
ranking survives through every rank the solver reads, then one base
scan serves every fine-tune of that model.
## Why it is live
The fine-tune-fitting lane parked on #68 prices each job at one scan.
A scan is the expensive step. If a LoRA or QLoRA fine-tune leaves the
damage ordering intact, the lane's unit cost collapses to a plan and a
pack. If it does not, every fine-tune pays a full scan, and the lane
carries a different economics.
Nothing in the records answers this. Every map measured so far reads
one checkpoint.
## The experiment already has a shape
Publication #2 ran the neighbouring test. Its card records:
> "A second sensitivity map, measured without the importance matrix,
> agrees with this ordering through every rank the solve reads and
> yields the identical placement."
That is map stability across a measurement change. This ticket asks for
map stability across a weight change. The comparison method carries
over: scan both checkpoints, compare the ordering through the ranks the
solve reads, and check whether the placement lands identical.
## What a result would settle
1. Whether a fine-tune needs its own scan, or reads the base map.
2. Whether the answer depends on the adapter rank, the epoch count, or
the fraction of weights the merge moves. A rank-16 QLoRA over 3
epochs moves less than a full fine-tune.
3. Whether a damage ordering is a property of the architecture or of
the trained weights. That reaches further than this lane.
## Cost
Two scans of one small dense target, plus one merged fine-tune of it.
Pick a target small enough that a scan is cheap, and dense, so the
expert-stack machinery stays out of the frame.
## Open
Which target? A small dense model keeps the scan cheap. The
answer may not generalize from small to large, and the ticket should
say so when it closes.
Does a merged QLoRA adapter count as a fine-tune for this purpose, or
does the question need a full fine-tune to be worth answering?
## Origin
Raised 2026-08-22 from an r/LocalLLaMA post recorded on #68. The author
fine-tuned Gemma 4 12B under a 16 GiB constraint. Their case is the
lane's shape, and their checkpoint would need a scan today.
## Related
- #68 parks the lane this prices.
- #144 asks whether per-layer sensitivity survives expert routing. This
asks whether it survives a weight update.
Contributor guide
Research direction
The payload names no files, tests, or entry points. Start by selecting a small dense target and run the stated comparison: scan the base and merged fine-tune, compare ordering through solver-read ranks, and check identical placement. Record adapter rank, epoch count, and moved-weight fraction, then document whether a separate scan is needed and whether results generalize.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100