uttrflow / uttrflow/uttrflow-swift
The repository depends on the WhisperKit version its own Docs/measuring-accuracy.md says was declined, and three speech documents still cite 0.18 file and line coordinates that no longer exist
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
**P2 — accuracy.**
### What happens
Nothing directly today. The consequence is that the repository's own account of how its recogniser behaves is wrong in specifics and contradictory in premise, which is how the three defects above went unnoticed — Docs/speech-engines.md:95 names a deletion condition that has since been met and nobody went back to it.
### Why
The product's knowledge of its recogniser lives in prose that names upstream files and line numbers, and prose cannot fail a build. A dependency version is a behavioural input to the dictation path with the same weight as the prompt, but only the prompt has a gate in front of it.
### What the code shows
The documentation contradiction is verified and is sharper than the finding puts it. Package.swift:56 declares WhisperKit from: "1.1.0" and Package.resolved:122-128 pins 1.1.0 from argmaxinc — while Docs/measuring-accuracy.md:3-4 opens 'A proposal, written after WhisperKit 1.1.0 was declined because nobody could say whether it was better or worse', and :98-101 records the decline reasoning. A shipped document says the version the manifest depends on was rejected. Docs/measuring-accuracy.md:18 confirms the corpus audio has never been recorded and :107-108 that the answer defaults to no for every speech-engine change, so the bump happened without the measurement that document says is required. The stale citations check out too: Docs/speech-vocabulary-prompt.md:17-18 and :60 cite 'WhisperKit 0.18, Core/TextDecoder.swift:339 / Core/Models.swift:1420 / Core/TextDecoder.swift:313-342' and Docs/speech-engines.md:71 repeats the first pair; in 1.1.0 the prompt-cap expression is at TextDecoder.swift:199 and the prefill build at :163-223. Docs/silence.md:37 cites TextDecoder.swift:993 for 'let noSpeechProb: Float = 0', which is now :817 — the claim still holds, the citation does not. Docs/speech-engines.md:95's stated deletion condition being met and the guard not deleted is verified in the finding above. Two errors to record. AGENTS.md does not mention WhisperKit 0.18 anywhere — I read it in full — so 'AGENTS.md's prompt notes' is wrong. And the new locations given ('Configurations.swift:199 and Models.swift:1340') are not right for the prompt-cap expression, which is TextDecoder.swift:199. The severity claim is also unsupportable: the finding itself says 'Nothing directly', and calling it P1/accuracy asserts a user-visible harm it does not demonstrate. What is verified is a documentation defect in the repository's own load-bearing knowledge store, plus a dependency on a version its own document says was declined.
### Where
- `Package.swift:56`
- `Package.resolved:122`
- `Docs/measuring-accuracy.md:3`
- `Docs/measuring-accuracy.md:18`
- `Docs/speech-engines.md:71`
- `Docs/speech-vocabulary-prompt.md:17`
- `Docs/silence.md:37`
### Fix direction
Turn the load-bearing claims into assertions, since AGENTS.md already makes that argument for comments and prose cannot fail a build. Add a WhisperKitContract test in UttrflowSpeech that reads from the linked WhisperKit rather than from a comment: Constants.maxTokenContext == 224; the prompt cap (maxTokenContext / 2) - 1 == VocabularyPrompt.maximumTokens; the prefill sequence prefillDecoderInputs actually builds for a prompted multilingual decode, compared against DecoderPrefill from the first finding; and every DecodingOptions default the product relies on, windowClipTime included. That single test also closes the three findings above at their source. Then make the corpus real — record the fifteen minutes, save a baseline, and gate any change to the whisperkit pin or to VocabularyPrompt on uttrflow-eval transcribe --fail-on-regression, which Docs/measuring-accuracy.md says already exists. Separately, fix the documents themselves: correct Docs/measuring-accuracy.md's premise so it does not describe a declined version the package depends on, and replace the 0.18 coordinates in Docs/speech-engines.md and Docs/speech-vocabulary-prompt.md with the checked facts the contract test now holds, so the numbers live somewhere that fails when they move. Pinning whisperkit exactly is a smaller call than the finding implies — Package.swift:64-68 pins Sparkle exactly because it ships as a remote binary target whose bytes can change, which does not apply to a source dependency — but it is still the right default until the baseline exists.
### Principles
It converts documentation into an executable contract, which is the only form that stays true — the argument AGENTS.md already makes for one-line comments. Open/closed — dependence on upstream internals becomes an explicit, checked interface rather than an implicit one rediscovered by users. It removes the class: no future bump can silently change how the product decodes.
---
Found by a code-path audit of the dictation pipeline. Traced in the source and checked against `Docs/` by a second reader; **not reproduced at runtime**, which is why this carries no `confirmed` label. Any rule or prompt change proposed above is measured against the corpus with `make bakeoff` before it lands.
Contributor guide
Research direction
Read Package.swift, Package.resolved, and Docs/measuring-accuracy.md first, then compare the cited WhisperKit locations in Docs/speech-engines.md, Docs/speech-vocabulary-prompt.md, and Docs/silence.md. Run the existing evaluation path, including make bakeoff, before changing the dependency or prompts. Done means the documented premise and citations are corrected, the corpus baseline exists, and the proposed WhisperKitContract checks cover the stated decoder assumptions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- build-system, documentation, testing
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100