uttrflow / uttrflow/uttrflow-swift

Both recording cues land inside the buffer the recogniser reads, and nothing trims either one

Open
#168 2 comments 0 reactions 1 assignee Claimed by @naveen-bhatt View on GitHub
area:dictation bug P1
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

**P1 — accuracy.**

### What happens

With the shipping default the recogniser is given a 0.564 s system alert at the head of every recording, measured by the repo at up to -11.5 dBFS against a -25.3 dBFS quiet mean, and the onset of a second alert in the tail. The user is invited to speak over it, because the cue is the go-ahead. Whether this costs a word has never been measured; that it is inside the audio the model decodes is measured and certain.

### Why

Call order in DictationController is the only thing that decides what is in the buffer, and DictationController is a gesture object. The capture engine owns the recording but never learns that the app is making noise into it; the cue owns the noise but never learns a recording is open. The fact that both halves know lives in Docs/audio-capture.md instead of in code, which is why the stop cue is early for no benefit and the head trim has never been decided either way.

### What the code shows

Every cited line says what the finding claims. DictationController.beginListening (121-127) awaits pipeline.startRecording() and only then calls cue.playStart(), so the microphone is open for the whole 0.564 s Tink; endHold (183-185) calls cue.playStop() before await pipeline.finishRecording(), and toggleListening (114-115), finishAtTheLimit (150-151) and stopHandsFree (192-193) do the same, so every one of the four ways a dictation ends puts the stop cue's onset in the tail. SystemSoundPlayer.play returns as soon as NSSound.play() is called (RecordingCue+System.swift:26). AVAudioCaptureEngine.stop() returns .canonical(accumulator.take()) unmodified (line 63) and nothing between the accumulator and BackedSpeechEngine removes a lead-in. SettingsStore.swift:85 does default playsSoundWhenRecordingStarts to true. Docs/audio-capture.md:48-70 has already measured the bleed (+6 dB over the first 700 ms, loudest trial -11.5 dBFS against a -25.3 dBFS quiet mean) and lists trimming as an unadopted mitigation, so this is an acknowledged gap rather than a decision already taken - the only alternative the doc actually rejects is delaying the microphone until the cue finishes, which is not what is proposed. The VoiceActivity claim also holds: the cue is a >120 ms run above threshold at the head, so voicedFrames anchors `first` on it and the 200 ms margin (VoiceActivity.swift:19,44) is spent in front of the bell. What I could NOT verify is the transcript damage. The repo measures acoustic contamination, never a word error: there is no corpus run with and without the cue, and the specific examples in the finding (\"Shipped the invoice\", the missing \"no\") are invented. Downgraded from P0 to P1 accordingly - contamination on every dictation at a level the doc itself calls speech-range is a daily risk, but P0 requires demonstrated word loss and none is demonstrated.

### Where

- `Sources/UttrflowPipeline/DictationController.swift:121-127`
- `Sources/UttrflowPipeline/DictationController.swift:183-185`
- `Sources/UttrflowPipeline/DictationController.swift:114-115`
- `Sources/UttrflowPipeline/DictationController.swift:150-151`
- `Sources/UttrflowPipeline/DictationController.swift:192-193`
- `Sources/UttrflowAudio/RecordingCue+System.swift:20-27`
- `Sources/UttrflowAudio/AVAudioCaptureEngine.swift:55-64`
- `Sources/UttrflowSettings/SettingsStore.swift:85`
- `Docs/audio-capture.md:46-73`

### Fix direction

Split this into the half that is free and the half that must be measured, and do not do them the same way.

Free, and structurally right: move the stop cue off the recording. `AVAudioCaptureEngine.stop()` already stops the source as its first act, so the tail contamination exists only because `DictationController` plays the cue before it calls the pipeline. Give `AudioCaptureEngine` the `RecordingCueing` and let `stop()` play the stop cue immediately after `source.stop()` and before it drains the accumulator. The user still hears the cue within milliseconds of key-up, and the tail is clean. That is Single Responsibility applied properly: the capture engine is the only object that knows the instant the microphone actually closed, which is the only instant at which the stop cue is safe.

Not free, and the finding's fix is wrong here: do NOT have `start()` play the cue and then discard every sample before the cue interval ends. That deletes a fixed 0.564 s from the head of every recording, and the users it deletes speech from are precisely the ones the finding is about - people who press and speak. Docs/audio-capture.md's phrase "before any human has begun speaking" is an assumption about user behaviour, not a fact about the buffer, and turning it into an unconditional truncation converts a probabilistic contamination into a deterministic word loss. It also cannot be made exact: `NSSound.play()` gives no start timestamp and no completion, so the interval would be a guess.

The honest structural change for the start cue is to make the contamination a measured pipeline decision rather than an accident of call order. Add a cue-bleed pair to the transcription corpus - the same utterances captured with `playsSoundWhenRecordingStarts` on and off through `uttrflow-dev dictate` - and let the number decide between three options in this order: (1) if the bleed costs corpus cases, default the setting to off and say why in Settings, because a confirmation sound that corrupts the confirmation is worse than none; (2) if it does not, leave it and delete mitigation 2 from the doc so the gap stops reading as an open TODO; (3) only if a trim is chosen, gate it on a cue that was actually played (`playStart` already returns whether the player accepted it - surface that instead of discarding it into `awaitingStop`) and apply it identically to the accumulator and the `RecordingWriter`, so the WAV kept for retry and the buffer handed to the recogniser stay the byte-identical pair Docs/recordings.md promises. Whichever lands, the code comment at RecordingCue+System.swift:8 that currently points at the doc should point at the decision.

### Principles

SRP: the capture engine is the only object that knows when the microphone opened and closed, so it is the only object that can place a cue safely; a gesture handler should not be deciding what the recogniser hears. KISS: reordering the stop cue is arithmetic-free and removes half the problem outright, which is why it should not be bundled with the speculative head trim. Measurement before rules: the head trim changes what the model is given, so it belongs behind the corpus like every other rule change, not behind a doc sentence.

---

Found by a code-path audit of the dictation pipeline. Traced in the source and checked against `Docs/` by a second reader; **not reproduced at runtime**, which is why this carries no `confirmed` label. Any rule or prompt change proposed above is measured against the corpus with `make bakeoff` before it lands.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.