uttrflow / uttrflow/uttrflow-swift

With a dictionary prompt the recogniser puts a comma between two adjacent dictionary words: "Zorvane, Kelthmar will meet…"

Open
#567 0 comments 0 reactions 0 assignees View on GitHub
area:dictation bug P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

`VocabularyPrompt` joins the dictionary words with ", " inside "The words used here are …" (`Sources/UttrflowSpeech/VocabularyPrompt.swift:33`; `Docs/speech-vocabulary-prompt.md`). When two listed words are spoken next to each other, as a first and last name are, the decoder copies the list punctuation into the transcript.

`uttrflow-dev bench` on main, the `nouns-vocabulary` category (invented names, each name a separate vocabulary word), shipping and rules identical because it is in the recogniser's raw text:

| Clip | Without vocabulary (raw) | With vocabulary (raw and inserted) |
|---|---|---|
| `nouns0` US | `Zorvane Keltmer will meet…` | `Zorvane, Kelthmar will meet…` |
| `nouns0` UK | `Zorvane Kelthma will meet…` | `Zorvane, Kelthmar will meet…` |
| `nouns0` Indian | `Zorvane Keltma will meet…` | `Zorvane, Kelthmar will meet…` |
| `nouns1` US | `Ask Mervella Ostrander whether…` | `Ask Mirvella, Ostrander, whether…` |
| `nouns1` UK | `Ask Mavella Ostranda whether…` | `Ask Mirvella, Ostrander whether…` |
| `nouns1` Indian | `Ask Mervella Ostranda whether…` | `Ask Mirvella, Ostrander, whether…` |

6 of 6 clips with two adjacent dictionary words gained one or two commas; none of the 12 clips without adjacent dictionary words did. The bench's word error rate drops punctuation, so this category reports 0.0%.

## Why it matters

The dictionary is how users get their colleagues' names right. With it, every full name arrives split by a comma ("Ask Mirvella, Ostrander, whether"), which reads as addressing two people.

## How to measure

`python3 Scripts/dictation_bench.py jobs --clean-only --categories nouns,nouns-vocabulary > jobs.tsv`, run `uttrflow-dev bench jobs.tsv`, and compare the `asr` event text of each `-vocabulary` clip with its plain counterpart.

## Acceptance criteria

- Adjacent dictionary words no longer gain commas on the bench's `nouns-vocabulary` clips, with the 0.0% word error rate kept. Options to compare: a separator the decoder does not copy, grouping entries that are commonly adjacent, or removing a comma between two dictionary words that the audio has no pause for (the word timings are available).
- The prompt-token budget in `Docs/speech-vocabulary-prompt.md` is re-checked if the separator changes.
- A bench or transcription-corpus check fails if the commas come back.

Contributor guide

Open the contributing guide

Research direction

Start at Sources/UttrflowSpeech/VocabularyPrompt.swift:33 and Docs/speech-vocabulary-prompt.md. Run the listed dictation_bench.py jobs command and uttrflow-dev bench, then compare asr event text for adjacent-word and plain clips. Done means nouns-vocabulary clips retain 0.0% word error rate without inserted commas, the token budget is rechecked, and a regression check prevents recurrence.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, swift
Domain
audio-video-rtc, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.