uttrflow / uttrflow/uttrflow-swift

A full dictionary prompt adds about 2.2 s per piece before the first word, one decoder step per prompt token

Open
#479 0 comments 0 reactions 0 assignees View on GitHub
area:dictation enhancement needs-info P2 performance
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

The personal dictionary is turned into a prompt Whisper is conditioned on (`Sources/UttrflowSpeech/VocabularyPrompt.swift`), packed up to `maximumTokens = 111` (line 17) and re-forced for every 30-second window (line 75). Each prompt token costs one decoder step, for every piece. `Docs/performance.md` (around line 1266) measured it: five invented names, 20-26 tokens, took the median recognition of a 4.6 s clip from 1.61 s to 2.50 s under load. At about 20 ms a step, a full 111-token prompt is about 2.2 s more for every piece, before the first word appears.

## Why it matters

The prompt buys accuracy on the words people care about most, and it is paid by exactly the users who took the time to fill in their dictionary. Early transcription makes each piece's cost visible, since it is paid per piece.

## Decision needed

The same document lists "a shorter vocabulary prompt" under "measured and not taken", because the cost and the accuracy it buys have to be traded against each other on vocabularies of the size people really keep. Options to measure:

1. A lower `maximumTokens`.
2. A relevance-ranked prompt: the entries most likely in this dictation (recently used, matching the app or the text around the caret) first, within a smaller budget.
3. Full prompt for the first piece only, and a smaller one for later pieces.

## Acceptance criteria

- Recognition time and word accuracy on dictionary terms are measured for at least two of these options, on dictionaries of a realistic size, with the results in `Docs/performance.md`.
- The chosen option is recorded with its reason, or the current behaviour is kept with the measurements that justify it.

`needs-info`: which trade-off is acceptable is a product decision.

Contributor guide

Open the contributing guide

Research direction

Start in Sources/UttrflowSpeech/VocabularyPrompt.swift, especially maximumTokens on line 17 and prompt rebuilding around line 75. Read the measurements around line 1266 of Docs/performance.md, then compare at least two prompt-size or prompt-selection options using realistic dictionaries. Done means recognition time and dictionary-term accuracy are documented, with the chosen option and rationale or measurements supporting the current behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
machine-learning, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.