Alberto-Codes / Alberto-Codes/vramfit
Measure throughput across size-matched mixed-precision recipes
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 6h 8m
- Merged PRs (30d)
- 105
Description
## Question
Do size-matched mixed-precision recipes produce materially different serving throughput?
## Why now
The solver optimizes predicted damage under a weight budget. It does not price decode speed, prefill speed, or runtime fallback behavior. Two recipes can occupy the same bytes while selecting different kernels and tensor-type mixtures.
This issue measures the effect before any solver change. It does not add a latency objective or schema field.
## Measurement
Choose one published target and at least three size-matched recipes with different type mixtures. Run them on one pinned llama.cpp build and one target GPU.
Record:
- Artifact bytes and resident VRAM.
- Prompt processing throughput at one fixed prompt length.
- Decode throughput at one and four concurrent streams.
- Load time and offload count.
- Any tensor-type fallback or unsupported kernel.
- Quality metrics already required by the publication lane.
Repeat enough runs to report variation. Keep runtime flags, context, batch sizes, and power state fixed.
## What closing looks like
Publish the measurements in the evidence ledger or a dedicated research record. If throughput differs materially, open a decision issue for runtime-aware planning. If it does not, record the bound and keep the solver focused on bytes and damage.
## Related
- ADR-0014 defines effective-bit size pricing.
- ADR-0017 defines the packed-artifact smoke test.
- ADR-0021 defines runtime-frame measurement.
Contributor guide
Research direction
Start with ADR-0014, ADR-0017, and ADR-0021, then identify one published target and three size-matched recipes for a pinned llama.cpp build and target GPU. Record artifact size, VRAM, prompt and decode throughput, load time, offload count, fallbacks, variation, and required quality metrics in the evidence ledger or a dedicated research record; close with the measured bound and whether a runtime-aware planning decision issue is needed.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning, performance
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100