uttrflow / uttrflow/uttrflow-swift
With Suggestions on, a 5,000-pass soak goes over the memory budget: 4.28 GB peak and 3.08 GB between passes
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 4h 27m
- Merged PRs (30d)
- 286
Description
## What happens
`Docs/performance.md` (the memory budget table, lines 226-234) sets two lines for Suggestions on: at most 3.0 GB between passes and at most 3.5 GB at the peak of a pass. A 5,000-pass soak of the suggestion model on current `main`, after the reload leak fixed in #462, measured:
| reading | budget | measured |
|---|---|---|
| process peak | 3.5 GB | about 4.28 GB |
| footprint between passes | 3.0 GB | about 3.08 GB |
The peak is well over the line; the between-passes figure is just over it. #462 measured 4.52 GB and 3.39 GB in the same kind of run before its fix, so the fix helped but did not bring either reading under the budget.
## Why it matters
The budget is written for an 8 GB Mac, where the suggestion model is already close to half the memory. A peak 0.8 GB over the line is the difference between fitting and pushing the user's other apps into swap.
## How to reproduce
The soak harness used in #462 is not in the repository. The closest committed equivalent is:
```bash
make bakeoff ARGS="gpu-memory --passes 5000 --release"
```
which checks every pass's peak and settled footprint against `ResourceBudget` and exits non-zero on a breach. `make perf-budget-models` runs only 12 passes, which is why it passes today. Run with `/usr/bin/time -l` to read the process peak as well.
## Acceptance criteria
- The source of the growth over a long run is identified: MLX's cache, the key-value cache, allocator fragmentation, or the remaining first-load cycles tracked in #411.
- A 5,000-pass run stays under both lines, or `Docs/performance.md` records a measured, justified change to the budget.
- The soak is runnable from the repository (a `gpu-memory` option or a documented command), so the next regression is found by running it rather than by rebuilding a harness.
Contributor guide
Research direction
Start with Docs/performance.md lines 226-234, ResourceBudget, and the gpu-memory path used by `make bakeoff ARGS="gpu-memory --passes 5000 --release"`; compare it with `make perf-budget-models`. Trace the long-run growth against the possibilities listed in the issue and #411/#462. Done means a repository-runnable 5,000-pass soak stays within both budgets, or the documentation records a measured, justified budget change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100