uttrflow / uttrflow/uttrflow-swift

Dictionary corrections never reach a word with punctuation attached, because word scores are keyed with the punctuation left on

Open
#714 0 comments 0 reactions 0 assignees View on GitHub
area:dictation bug good first issue P1
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

A dictionary correction can only replace a word the recogniser scored as doubtful. The score is looked up in `Transcription.scoredWords` (`Sources/UttrflowPipeline/TranscriptChanging.swift:25-43`), and the two sides of that lookup are keyed differently:

- the recogniser's words are keyed as `word.text.lowercased()` (`:31`), punctuation left on;
- the transcript's words are looked up as `word.trimmingCharacters(in: .punctuationCharacters).lowercased()` (`:39`).

WhisperKit merges punctuation into the word timing it belongs to (`mergePunctuations` in its `SegmentSeeker`, with `Constants.defaultAppendPunctuations` = `"'.。,,!!??::”)]}、` and the prepend set `"'“¡¿([{-`), and `WhisperKitBackend` passes `$0.word` through unchanged (`Sources/UttrflowSpeech/WhisperKitBackend.swift:118-120`); `RawTranscript+Mapping.swift:110` trims only whitespace. So a word followed by a comma, full stop or question mark, or opened by a quote or bracket, is stored under `"tarvock,"` and looked up as `"tarvock"`. The lookup misses, the word gets the fallback confidence `1` (`:41`), and the correction engine never considers it.

There is a second step waiting behind the first. `DictationCorrection.applying` (`Sources/UttrflowPipeline/DictationChanges.swift:46-50`) replaces the whole whitespace-separated token, punctuation included, with the dictionary spelling. Once the score is found, "tarvock," becomes "Tarvok" and the comma is gone.

Measured with a throwaway test in `UttrflowPipelineTests`, using word timings shaped as WhisperKit returns them:

| Input | Expected | Actual |
|---|---|---|
| words `Open`(0.99) `the`(0.99) `tarvock,`(0.2) `then`(0.99) `check`(0.99) `tarvock.`(0.3) | `tarvock,` = 0.2, `tarvock.` = 0.3 | both `1.0` |
| `applying` a correction of word 2 to `Tarvok` in `Open the tarvock, then check tarvock.` | `Open the Tarvok, then check tarvock.` | `Open the Tarvok then check tarvock.` |

The existing test `ScoredWordsTests.punctuation()` passes because its word timings are written without the punctuation while the transcript text carries it, which is not what the recogniser produces.

## Why it matters

Names and product words most often sit right before a comma or at the end of a sentence ("Thanks, Tarvok.", "Send it to Tarvok."). Those are exactly the positions where the personal dictionary silently does nothing, so someone who added a word sees it misspelled again and has no way to tell why.

## How to reproduce

In `Tests/UttrflowPipelineTests/ScoredWordsTests.swift`:

```swift
let text = "Thanks, tarvock."
let scored = Transcription(text: text, segments: [TranscriptionSegment(
text: text, start: .zero, end: .seconds(1),
words: [TranscribedWord(text: "Thanks,", confidence: 0.99),
TranscribedWord(text: "tarvock.", confidence: 0.38)])]).scoredWords
#expect(scored?.last?.confidence == 0.38) // fails today: 1.0
```

And in `DictationChangesTests.swift`, applying a correction for word 2 of `"Open the tarvock, then"` should give `"Open the Tarvok, then"`.

## Acceptance criteria

- Both sides of the score lookup reduce a word the same way (punctuation trimmed, lower-cased).
- `ScoredWordsTests` uses punctuated word timings like the recogniser's, including an opening quote or bracket.
- Applying a correction keeps the punctuation around the replaced word (leading and trailing), and undoing it (`CorrectionUndo`) still finds the words.

## Where to start

- `Sources/UttrflowPipeline/TranscriptChanging.swift:28-42` and `Sources/UttrflowPipeline/DictationChanges.swift:34-56`.
- Tests: `Tests/UttrflowPipelineTests/ScoredWordsTests.swift`, `Tests/UttrflowPipelineTests/DictationChangesTests.swift`.
- Run `make verify` (export `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer` first) and read CONTRIBUTING.md.

**Size:** S, an hour or two.

Contributor guide

Open the contributing guide

Research direction

Start with Sources/UttrflowPipeline/TranscriptChanging.swift:28-42 and DictationChanges.swift:34-56, then read the reproductions in ScoredWordsTests.swift and DictationChangesTests.swift. Run the focused tests first; done means punctuated recogniser words receive their scores, corrections preserve leading and trailing punctuation, and CorrectionUndo still works. Finish with make verify after setting DEVELOPER_DIR as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.