microsoft / microsoft/WindowsAppSDK
BatchRecognition returns English text for Japanese audio with no documented language setting
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 471
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 28
Description
### Describe the bug
In my diagnostic test, `Microsoft.Windows.AI.Speech.BatchRecognition` returned English text that appears to be a translation of Japanese speech instead of a Japanese transcription.
The English output broadly follows the meaning of the Japanese input, although some details are mistranslated or omitted.
I could not find a language or transcription-vs-translation option on `SpeechRecognitionModel`, `BatchRecognition`, or `RecognizeFromFile`. The [Windows AI Speech documentation](https://learn.microsoft.com/windows/ai/apis/speech-recognition) and the [Microsoft AI Dev Gallery sample](https://github.com/microsoft/ai-dev-gallery/blob/main/AIDevGallery/Samples/WCRAPIs/SpeechRecognition.xaml.cs) use the same API shape without a language setting.
For one reproducible input, the spoken Japanese is:
> 北浜鉄道は、若葉線の新型車両を来月十五日から運行します。車内には車椅子用のスペースと、百ボルトの電源コンセントを設置しました。
The result returned by `RecognizeFromFile` is:
> Kitahama Railway transports the new model of Wakabasen from the 15th of next month. Inside the car, we installed a space for the car and a 100 volt power supply.
The WAV was generated with the Microsoft Haruka Desktop Japanese voice. A native Japanese speaker listened to the WAV and found the speech clearly intelligible and natural, without anything that sounded like non-Japanese speech. It is passed to `RecognizeFromFile` as mono, 16 kHz, 16-bit PCM. I can provide the WAV if it would help with investigation.
### Steps to reproduce the bug
1. Create a packaged MSIX application with the `systemAIModels` capability.
2. Prepare the speech recognition model with `SpeechRecognitionModel.EnsureReadyAsync()` if required.
3. Use a mono, 16 kHz, 16-bit PCM WAV containing the Japanese sentence above.
4. Run the following relevant code:
```csharp
var modelResult = await SpeechRecognitionModel.TryCreateAsync();
if (modelResult.SpeechModel is null)
{
throw new InvalidOperationException(
$"Failed to create SpeechRecognitionModel: {modelResult.ExtendedError}");
}
using var recognition = new BatchRecognition(modelResult.SpeechModel);
string text = await recognition.RecognizeFromFile(wavPath);
```
I also tested one run with both of the following applied before creating the model:
- declaring `ja-jp` in the MSIX resources;
- requesting `ja-JP` through `Windows.Globalization.ApplicationLanguages.PrimaryLanguageOverride` (the value read back inside the packaged process was normalized to `ja`).
The control and language-override runs returned exactly the same 161-character English output.
In a small single-voice diagnostic set of 12 synthetic Japanese-news WAV files, 9 results were English-dominant. All files used the same Microsoft Haruka Desktop voice at several speaking rates. The Windows culture, UI culture, and system locale were all `ja-JP`.
### Expected behavior
For Japanese speech, I expected `BatchRecognition` to return a Japanese transcription rather than English text that appears to be a translation.
If automatic language detection and translation are intentional, I would expect a documented way to select Japanese and request transcription instead of translation.
Could you clarify whether there is currently a supported way to constrain recognition to Japanese and disable translation?
### Screenshots
_No response_
### NuGet package version
2.2.2-experimental9
### Packaging type
Packaged (MSIX)
### Windows version
Windows 11 version 25H2, OS build 26200.8655 (General Availability Channel; not enrolled in Windows Insider)
### IDE
Other (`dotnet` CLI, .NET 9)
### Additional context
- `Microsoft.WindowsAppSDK.AI`: 2.2.6-experimental
- Installed speech component: `Microsoft.Windows.AI.Speech.Preview.CPU` 1.2607.112.0 (x64)
- OS architecture: x64
- The input is transcoded to the same canonical format used by the Microsoft AI Dev Gallery Speech Recognition sample: mono, 16 kHz, 16-bit PCM WAV.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the SpeechRecognition documentation and the Microsoft AI Dev Gallery sample at SpeechRecognition.xaml.cs, then inspect the SpeechRecognitionModel, BatchRecognition, and RecognizeFromFile API behavior described in the report. Reproduce the Japanese WAV case with the listed package and OS versions. Done means documenting or confirming a supported Japanese transcription setting, or clearly identifying that the current behavior cannot be configured.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai, desktop
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100