uttrflow / uttrflow/uttrflow-swift

Measure the memory peak of a cold speech-model compile with WhisperKit's prewarm off

Open
#481 0 comments 0 reactions 0 assignees View on GitHub
area:dictation enhancement P2 performance
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

`WhisperKitBackend` loads the model with `prewarm: true` (`Sources/UttrflowSpeech/WhisperKitBackend.swift:46`). `Docs/performance.md` (the Launch table, around lines 1271-1279) measured a warm launch at 2.2-2.5 s with prewarm on and 1.2-1.3 s without it, and the first dictation after a warm launch waited the same either way. So on a warm launch, prewarm costs about 1.1 s and buys nothing visible.

WhisperKit prewarms to keep peak memory down during the first compile of the model on a cold launch. That peak was not measured, so prewarm stays on.

## Why it matters

A second off every warm launch is worth having if the cold-compile peak without prewarm is still inside the memory budget on an 8 GB Mac (see #102 for how long that first load already takes).

## How to measure

- Cold: remove the compiled model cache so the next launch compiles, and record peak footprint (`/usr/bin/time -l`, or `uttrflow-bakeoff profile`) with prewarm on and off. Repeat enough times to see the spread.
- Warm: confirm the 1.1 s difference on current `main`.

## Acceptance criteria

- Cold-compile peak footprint with and without prewarm is recorded in `Docs/performance.md`, with the machine's memory size.
- A decision follows: turn prewarm off, keep it, or make it depend on whether a compiled copy is already cached. If changed, the reason is a one-line comment at the call site.

Contributor guide

Open the contributing guide

Research direction

Start with Sources/UttrflowSpeech/WhisperKitBackend.swift:46 and the Launch table in Docs/performance.md. Remove the compiled model cache, then run /usr/bin/time -l or uttrflow-bakeoff profile with prewarm on and off, repeating cold and warm launches. Record both cold peaks, the machine memory size, and the warm-launch difference in Docs/performance.md; finish with the documented prewarm decision and a call-site comment if behavior changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
desktop-dev, performance
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.