uttrflow / uttrflow/uttrflow-swift

Add Hindi clips to the dictation bench: short replies, English-then-Hindi code-switching, lakh and crore, and a self-correction

Open
#707 0 comments 0 reactions 0 assignees View on GitHub
area:dictation enhancement good first issue P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What this is

`Scripts/dictation_bench.py` builds its synthetic corpus in `clips()` (`:138-172`). For Hindi it has only the 12 committed `TranscriptionCorpus` passages, read by the `Lekha` voice (`:169-171`), and 3 romanised Hinglish sentences read by an English voice (`HINGLISH`, `:92-96`). Each passage is 9–12 s of continuous speech in one language.

The failures a Hindi speaker actually hits are in shapes it doesn't contain. Each was measured with a throwaway run on `origin/main` 26d7bc1 and is filed separately (#699, #698, #708, #703, #700):

| category to add | what it catches | measured |
|---|---|---|
| `hi-reply` | 1–2 s Hindi replies decoded as English | 4 of 10 came back as English words |
| `code-switch` | an English sentence, a pause, then a Hindi sentence (one clip, two voices joined) | the Hindi piece was translated into invented English in 3 of 4 runs and dropped in 1 |
| `hi-lakh` | "पचास लाख", "दो करोड़ पचास लाख", "डेढ़ लाख" | लाख written as लाक or लाग in 3 of 3 clips |
| `hi-selfcorrection` | "चार बजे है, नहीं नहीं, पाँच बजे है" | correction applied in 0 of 3 |
| `hi-fillers` | "मतलब …, हाँ, अच्छा …" | Apple's model answered one with its own worked example |

## What to do

1. Add lists next to `HINGLISH`, for example `HINDI_REPLIES`, `HINDI_AMOUNTS` and `HINDI_CORRECTIONS`. Each item is a `(devanagari, romanised written form)` pair, spoken by `Lekha`. Use invented content with no real names. About 5–10 items per list is plenty.
2. In `clips()`, add them with `add(..., devanagari=...)` the way the committed passages are added, so both scripts count as correct.
3. For `code-switch`, synthesise an English sentence with `Rishi` and a Hindi sentence with `Lekha`, and join them with `read_wav`/`write_wav` (both already in the script), putting 1.5 s of silence between them. Make the English part at least 5 s (say it twice), so the pipeline cuts a piece before the Hindi starts. Its `written` is both halves.
4. Mention the new categories in the corpus section of `Docs/performance.md`.

## Acceptance criteria

- `python3 Scripts/dictation_bench.py corpus` produces the new clips, and `jobs --categories hi-reply,code-switch,hi-lakh,hi-selfcorrection,hi-fillers` lists them.
- A clip with an existing voice and text is not re-synthesised (names are hashed today; keep that).
- No real names or personal data in any text.

## Where to start

- `Scripts/dictation_bench.py:92-96` (`HINGLISH`), `:138-172` (`clips`), `:175-183` (`read_wav`, `write_wav`)
- `Docs/performance.md`, the section on re-running the bench
- If you have the model installed, run `uttrflow-dev bench` on the new jobs: `swift build -c release --product uttrflow-dev`, then `uttrflow-dev models install` if needed. Running is optional; the change is Python only. Run one bench process at a time.
- Still run `make verify` before pushing (export `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer` first).
- Read [CONTRIBUTING.md](https://github.com/uttrflow/uttrflow-swift/blob/main/CONTRIBUTING.md) first, and say on this issue that you are taking it.

**Size:** S, about 2–3 hours.

Contributor guide

Open the contributing guide

Research direction

Read CONTRIBUTING.md, then inspect Scripts/dictation_bench.py:92-96 and :138-183, especially clips(), read_wav(), and write_wav(). Run the corpus and jobs commands from the acceptance criteria while preserving existing voice/text caching. Done means the five Hindi-related categories are listed, generated clips include both scripts and the code-switch pause, Docs/performance.md mentions them, and make verify passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.