uttrflow / uttrflow/uttrflow-swift
Measure the memory peak of a cold speech-model compile with WhisperKit's prewarm off
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
## What happens
`WhisperKitBackend` loads the model with `prewarm: true` (`Sources/UttrflowSpeech/WhisperKitBackend.swift:46`). `Docs/performance.md` (the Launch table, around lines 1271-1279) measured a warm launch at 2.2-2.5 s with prewarm on and 1.2-1.3 s without it, and the first dictation after a warm launch waited the same either way. So on a warm launch, prewarm costs about 1.1 s and buys nothing visible.
WhisperKit prewarms to keep peak memory down during the first compile of the model on a cold launch. That peak was not measured, so prewarm stays on.
## Why it matters
A second off every warm launch is worth having if the cold-compile peak without prewarm is still inside the memory budget on an 8 GB Mac (see #102 for how long that first load already takes).
## How to measure
- Cold: remove the compiled model cache so the next launch compiles, and record peak footprint (`/usr/bin/time -l`, or `uttrflow-bakeoff profile`) with prewarm on and off. Repeat enough times to see the spread.
- Warm: confirm the 1.1 s difference on current `main`.
## Acceptance criteria
- Cold-compile peak footprint with and without prewarm is recorded in `Docs/performance.md`, with the machine's memory size.
- A decision follows: turn prewarm off, keep it, or make it depend on whether a compiled copy is already cached. If changed, the reason is a one-line comment at the call site.
Contributor guide
Research direction
Start with Sources/UttrflowSpeech/WhisperKitBackend.swift:46 and the Launch table in Docs/performance.md. Remove the compiled model cache, then run /usr/bin/time -l or uttrflow-bakeoff profile with prewarm on and off, repeating cold and warm launches. Record both cold peaks, the machine memory size, and the warm-launch difference in Docs/performance.md; finish with the documented prewarm decision and a call-site comment if behavior changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- desktop-dev, performance
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100