uttrflow / uttrflow/uttrflow-swift

The whole audio graph is constructed, tapped and started inside every key-down, and that cost has never been measured

Open
#214 2 comments 0 reactions 0 assignees View on GitHub
area:dictation bug claimed P2 performance
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

**P2 — latency.**

### What happens

Press the shortcut and speak at once: `startRecording` is still awaiting the construction of a new `AVAudioEngine`, an input-format query, a tap install, `prepare()` and `start()`, and no sample exists until that returns — nor does the `.recording` state or the start cue. Whether that swallows a syllable or is imperceptible is unknown, because the opening half of capture is in no stage and no document carries a figure for it.

### Why

`AVAudioEngineMicrophoneSource` ties two lifetimes together: the audio graph, which depends only on the input device and already has a configuration-change observer for when that changes, and the recording, which happens many times a day. Tying the first to the second rebuilds a graph that did not change — and the cost of doing so is invisible because `PipelineStage` names only the stages somebody chose to wrap.

### What the code shows

The code is as described. `AVAudioEngineMicrophoneSource.start` always goes through `open()` (lines 30-38, 41-79), which constructs `AVAudioEngine()`, queries `inputFormat(forBus:)`, builds a fresh `AudioResampler`, installs a tap, calls `prepare()` then `start()`, and registers a configuration-change observer; `stop()` calls `close()` (lines 89-106), which removes the tap, stops the engine and drops it, so nothing survives to the next dictation. `AVAudioCaptureEngine.start()` awaits it (AVAudioCaptureEngine.swift:33-53) and `DictationPipeline.startRecording` awaits that at line 173 before `transition(to: .recording)` at line 185, and `DictationController.beginListening` plays the start cue only after the pipeline reports listening (DictationController.swift:121-128). The opening really is unmeasured: the only capture measurement is `metrics.measuring(.capture) { capture.stop() }` at DictationPipeline.swift:202-206, and `PipelineStage` (Metrics.swift:4-17) has no case that could hold the opening. Two corrections. The doc quotes are about the wrong thing: Docs/performance.md's "Capture and insertion are not timed / No microphone is touched here by design" is describing the bakeoff harness's limits, not the product's instrumentation, and the 118 ms figure in Docs/audio-capture.md:85-87 is AppKit building a *sound output* graph, which is not evidence about `AVAudioEngine` input. So the magnitude — whether this eats a syllable or is imperceptible — is genuinely unknown, and the finding says so. Given an unmeasured cost with a plausible but unproven user effect, this is P2, not P1. I am also refuting the proposed fix outright: see below.

### Where

- `Sources/UttrflowAudio/AVAudioEngineMicrophoneSource.swift:30-38`
- `Sources/UttrflowAudio/AVAudioEngineMicrophoneSource.swift:41-79`
- `Sources/UttrflowAudio/AVAudioEngineMicrophoneSource.swift:89-106`
- `Sources/UttrflowPipeline/DictationPipeline.swift:173`
- `Sources/UttrflowPipeline/DictationPipeline.swift:202-206`
- `Sources/UttrflowCore/Support/Metrics.swift:4-17`
- `Sources/UttrflowPipeline/DictationController.swift:121-128`

### Fix direction

Measure before restructuring, because the proposed restructuring is unsafe for this product and the number that would justify it does not exist. First: give the opening a span — either a `microphoneOpen` case on `PipelineStage` or extend `.capture` to cover `start()` as well as `stop()` — and record the figure in Docs/performance.md where the gap is currently stated. `uttrflow-dev record` comparing the first captured sample's timestamp against key-down is the check. Only then decide. Do NOT ship the proposed "one engine for the source's lifetime, `pause()` between recordings": for a product whose entire claim is that dictation is local and on this Mac, an input graph that outlives the recording is a privacy change, and the finding asserts without evidence that "the macOS microphone indicator still goes out" under `AVAudioEngine.pause()` — `pause()` stops rendering and deliberately keeps resources allocated, and whether the HAL input stream and therefore the indicator go with it is exactly the kind of platform question AGENTS.md says to probe rather than reason about. If the measurement shows the opening is material, the right structural split is narrower and provable: separate what depends only on the device and does not arm it (input-format query, `AudioResampler` construction, node graph) from what does (tap install and `start()`), prewarm the first at `prepare()` time beside the speech model, and rebuild it from the existing `hardwareChanged()` path, which already exists for precisely this reason (Docs/microphone.md:26-35). The tap block must consult the sink under its existing lock either way. Whatever lands, the source itself is outside the coverage gate by its own doc comment, so the split must put the testable half in a type that carries tests.

### Principles

SRP: a source owns a device and a recording is a period during which that device is armed; two lifetimes, two states. YAGNI in reverse: the configuration-change observer already exists so the graph can outlive one recording. But KISS cuts the other way on the privacy question — the current teardown makes "the microphone is closed between dictations" true by construction, and any replacement has to prove that property rather than assume it.

---

Found by a code-path audit of the dictation pipeline. Traced in the source and checked against `Docs/` by a second reader; **not reproduced at runtime**, which is why this carries no `confirmed` label. Any rule or prompt change proposed above is measured against the corpus with `make bakeoff` before it lands.

Contributor guide

Open the contributing guide

Research direction

Read AVAudioEngineMicrophoneSource.swift, DictationPipeline.swift, Metrics.swift, and Docs/performance.md to trace the opening path and existing measurement. Run `uttrflow-dev record` and compare the first captured sample with key-down; done means the opening is recorded through an appropriate PipelineStage and the measured figure is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
audio-video-rtc, desktop-dev, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
57/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.