uttrflow / uttrflow/uttrflow-swift

Apple's model declines most Hindi dictation as an unsupported language, so Hindi is tidied by the rules and left in Devanagari

Open
#445 1 comment 0 reactions 0 assignees View on GitHub
area:dictation bug P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

### What happens

Apple's on-device model declines most Hindi dictation, so it is tidied by the rules and inserted in Devanagari rather than as the Hinglish the prompt asks for.

Measured on `origin/main` at 1d69ee8, Release build of `uttrflow-dev`, through `DictationPipeline` with the shipping router (`TextTransformers.router()`), macOS 26.5.1. Speech is the system `Lekha` voice reading the committed `TranscriptionCorpus` Hindi and Hinglish passages (12 clips, 9–12 s each).

| passages | tidied by Apple's model | tidied by rules |
|---|---|---|
| Hindi, 6 | 0 | 6 |
| Hinglish, 6 | 0 | 6 |

Every one of the 12 falls to the rules in about 20 ms. Calling `GenerativeTextTransformer` for `.foundationModels` directly on the same requests reports availability `available` and then throws `transformFailed(... "An unsupported language or locale was used")`.

It depends on the text, not only on the language. With `uttrflow-dev clean -e foundationModels`:

- a one-clause Devanagari sentence of 7–10 words is romanised as intended;
- a two-clause Devanagari sentence of about 20 words is declined;
- a romanised Hinglish sentence of 12 words is also declined.

So `AppleFoundationCleanupModel.verifiedBeyondApplesList` answers `available` for Hindi while the model itself refuses the ordinary length of a Hindi dictation. The router hides the refusal: the words still arrive, untidied.

One accepted answer in the same probe was a translation into English rather than a romanisation. That half is the guard limitation already tracked in #196; this issue is the refusal rate.

### Why it matters

Hindi is routed to Apple's model on the strength of a bake-off score (`Docs/bakeoff.md`), and nothing re-measures it on the pipeline. A Hindi speaker gets the floor on nearly every dictation and no record says why, because `TransformerRouter` records refusals only for `outputRejected`, not for `transformFailed`.

### Reproduce

```
say -v Lekha -o hi.wav --file-format=WAVE --data-format=LEI16@16000 ""
uttrflow-dev dictate hi.wav # "tidied by rules"
uttrflow-dev clean -e foundationModels ""
```

### Not done here

No fix is proposed in this issue. The options change product behaviour (declaring Hindi unavailable so a local model is tried first, recording `transformFailed` as a refusal, or re-running the bake-off on the current OS) and want a decision rather than a benchmark.

Contributor guide

Open the contributing guide

Research direction

Start with Docs/bakeoff.md, DictationPipeline, and TextTransformers.router(), then reproduce the Hindi and Hinglish cases with TranscriptionCorpus. Read GenerativeTextTransformer and AppleFoundationCleanupModel.verifiedBeyondApplesList to trace the availability and transformFailed paths, and inspect TransformerRouter's refusal recording. Done requires a maintainer decision on the product behavior before implementation scope is defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
ai
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.