Alberto-Codes / Alberto-Codes/vramfit

Measure throughput across size-matched mixed-precision recipes

Open
#488 1 comment 0 reactions 0 assignees View on GitHub
research
Dominant language
Python
Stars
1
Forks
1
Avg merge
6h 8m
Merged PRs (30d)
105

Description

## Question

Do size-matched mixed-precision recipes produce materially different serving throughput?

## Why now

The solver optimizes predicted damage under a weight budget. It does not price decode speed, prefill speed, or runtime fallback behavior. Two recipes can occupy the same bytes while selecting different kernels and tensor-type mixtures.

This issue measures the effect before any solver change. It does not add a latency objective or schema field.

## Measurement

Choose one published target and at least three size-matched recipes with different type mixtures. Run them on one pinned llama.cpp build and one target GPU.

Record:

- Artifact bytes and resident VRAM.
- Prompt processing throughput at one fixed prompt length.
- Decode throughput at one and four concurrent streams.
- Load time and offload count.
- Any tensor-type fallback or unsupported kernel.
- Quality metrics already required by the publication lane.

Repeat enough runs to report variation. Keep runtime flags, context, batch sizes, and power state fixed.

## What closing looks like

Publish the measurements in the evidence ledger or a dedicated research record. If throughput differs materially, open a decision issue for runtime-aware planning. If it does not, record the bound and keep the solver focused on bytes and damage.

## Related

- ADR-0014 defines effective-bit size pricing.
- ADR-0017 defines the packed-artifact smoke test.
- ADR-0021 defines runtime-frame measurement.

Contributor guide

Open the contributing guide

Research direction

Start with ADR-0014, ADR-0017, and ADR-0021, then identify one published target and three size-matched recipes for a pinned llama.cpp build and target GPU. Record artifact size, VRAM, prompt and decode throughput, load time, offload count, fallbacks, variation, and required quality metrics in the evidence ledger or a dedicated research record; close with the measured bound and whether a runtime-aware planning decision issue is needed.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning, performance
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.