uttrflow / uttrflow/uttrflow-swift
A full dictionary prompt adds about 2.2 s per piece before the first word, one decoder step per prompt token
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
## What happens
The personal dictionary is turned into a prompt Whisper is conditioned on (`Sources/UttrflowSpeech/VocabularyPrompt.swift`), packed up to `maximumTokens = 111` (line 17) and re-forced for every 30-second window (line 75). Each prompt token costs one decoder step, for every piece. `Docs/performance.md` (around line 1266) measured it: five invented names, 20-26 tokens, took the median recognition of a 4.6 s clip from 1.61 s to 2.50 s under load. At about 20 ms a step, a full 111-token prompt is about 2.2 s more for every piece, before the first word appears.
## Why it matters
The prompt buys accuracy on the words people care about most, and it is paid by exactly the users who took the time to fill in their dictionary. Early transcription makes each piece's cost visible, since it is paid per piece.
## Decision needed
The same document lists "a shorter vocabulary prompt" under "measured and not taken", because the cost and the accuracy it buys have to be traded against each other on vocabularies of the size people really keep. Options to measure:
1. A lower `maximumTokens`.
2. A relevance-ranked prompt: the entries most likely in this dictation (recently used, matching the app or the text around the caret) first, within a smaller budget.
3. Full prompt for the first piece only, and a smaller one for later pieces.
## Acceptance criteria
- Recognition time and word accuracy on dictionary terms are measured for at least two of these options, on dictionaries of a realistic size, with the results in `Docs/performance.md`.
- The chosen option is recorded with its reason, or the current behaviour is kept with the measurements that justify it.
`needs-info`: which trade-off is acceptable is a product decision.
Contributor guide
Research direction
Start in Sources/UttrflowSpeech/VocabularyPrompt.swift, especially maximumTokens on line 17 and prompt rebuilding around line 75. Read the measurements around line 1266 of Docs/performance.md, then compare at least two prompt-size or prompt-selection options using realistic dictionaries. Done means recognition time and dictionary-term accuracy are documented, with the chosen option and rationale or measurements supporting the current behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, swift
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100