microsoft / microsoft/WindowsAppSDK
Microsoft.Windows.AI.Speech speech recognition API crashes with 0xC000001D: Illegal Instruction
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 471
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 28
Description
### Describe the bug
The Microsoft.Windows.AI.Speech crashes with `(asrmodelapi.dll): 0xC000001D: Illegal Instruction.` when calling `StreamingRecognition.StartContinuousRecognitionAsync()`.
It looks like some CPU instruction required is not supported. If my CPU is not supported, I would expect `SpeechRecognitionModel.GetReadyState()` to fail before (but it succeeded).
Please provide more documentation which hardware is expected to be supported, and if the hardware is not supported, don't crash the app. Also I would expect a RTX 2060 GPU to be more than capable to run a speech recoginition model instead of using CPU (when looking at the loaded DLLs).
CPU: Intel i5-11400; GPU: NVIDIA RTX 2060
### Steps to reproduce the bug
Execute following code:
```c#
var catalog = ExecutionProviderCatalog.GetDefault();
await catalog.EnsureAndRegisterCertifiedAsync();
var readyState = SpeechRecognitionModel.GetReadyState();
if (readyState is AIFeatureReadyState.DisabledByUser or AIFeatureReadyState.NotSupportedOnCurrentSystem)
{
throw new Exception();
}
if (readyState == AIFeatureReadyState.NotReady)
{
var ensureResult = await SpeechRecognitionModel.EnsureReadyAsync();
if (ensureResult.Status != AIFeatureReadyResultState.Success)
{
throw new Exception();
}
}
var modelResult = await SpeechRecognitionModel.TryCreateAsync();
var audioConfig = AudioConfiguration.FromAudioDevice(deviceName);
var recognition = new StreamingRecognition(audioConfig, _model);
recognition.Recognizing += OnRecognizing;
recognition.Recognized += OnRecognized;
await recognition.StartContinuousRecognitionAsync(); // <--- this will crash
```
### Expected behavior
Speech recognition works. Or, if my CPU is not supported, I would expect `SpeechRecognitionModel.GetReadyState()` to fail before (but it succeeded).
### Screenshots
_No response_
### NuGet package version
2.2.2-experimental9
### Packaging type
Packaged (MSIX)
### Windows version
Insider Build (xxxxx)
### IDE
_No response_
### Additional context
Debug output
```
'Test.exe' (Win32): Loaded 'C:\Test\bin\x64\Debug\net8.0-windows10.0.22621.0\win-x64\AppX\Microsoft.Windows.AI.Speech.Projection.dll'.
'Test.exe' (CoreCLR: clrhost): Loaded 'C:\Test\bin\x64\Debug\net8.0-windows10.0.22621.0\win-x64\AppX\Microsoft.Windows.AI.Speech.Projection.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'Test.exe' (Win32): Loaded 'C:\Test\bin\x64\Debug\net8.0-windows10.0.22621.0\win-x64\AppX\Microsoft.Windows.AI.Projection.dll'.
'Test.exe' (CoreCLR: clrhost): Loaded 'C:\Test\bin\x64\Debug\net8.0-windows10.0.22621.0\win-x64\AppX\Microsoft.Windows.AI.Projection.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'Test.exe' (Win32): Loaded 'C:\Test\bin\x64\Debug\net8.0-windows10.0.22621.0\win-x64\AppX\Microsoft.Windows.AI.MachineLearning.Projection.dll'.
'Test.exe' (CoreCLR: clrhost): Loaded 'C:\Test\bin\x64\Debug\net8.0-windows10.0.22621.0\win-x64\AppX\Microsoft.Windows.AI.MachineLearning.Projection.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\Microsoft.WindowsAppRuntime.2-experimental9_2.2.2.0_x64__8wekyb3d8bbwe\Microsoft.Windows.AI.MachineLearning.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\AppExtension.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\Windows.StateRepositoryPS.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\AppXDeploymentClient.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\Windows.Web.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Test\bin\x64\Debug\net8.0-windows10.0.22621.0\win-x64\AppX\onnxruntime.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\setupapi.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\dbghelp.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\onnxruntime_providers_openvino_plugin.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\tbb12.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\tbbmalloc.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\onnxruntime_providers_openvino_plugin_impl.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino_onnx_frontend.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino_auto_plugin.dll'. Symbol loading disabled by Include/Exclude setting.
Exception thrown at 0x00007FFDF1BB1CFA in Test.exe: Microsoft C++ exception: std::runtime_error at memory location 0x000000DF23B7DA90.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino_intel_gpu_plugin.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\OpenCL.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_12ab2876952d3f1f\nvopencl64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\nvapi64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_12ab2876952d3f1f\nvdxgdmal64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\iigd_dch.inf_amd64_92899f2b73e871b4\igdrcl64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\ws2_32.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\iigd_dch.inf_amd64_92899f2b73e871b4\igdgmm64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\iigd_dch.inf_amd64_92899f2b73e871b4\igdfcl64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\iigd_dch.inf_amd64_92899f2b73e871b4\igc64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\opengl32.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\glu32.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Unloaded 'C:\Windows\System32\glu32.dll'
'Test.exe' (Win32): Unloaded 'C:\Windows\System32\opengl32.dll'
Exception thrown at 0x00007FFDF1BB1CFA in Test.exe: Microsoft C++ exception: std::runtime_error at memory location 0x000000DF23B7D930.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino_intel_cpu_plugin.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\tbbbind_2_5.dll'. Symbol loading disabled by Include/Exclude setting.
The thread 41568 has exited with code 0 (0x0).
'Test.exe' (Win32): Unloaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino_intel_cpu_plugin.dll'
'Test.exe' (Win32): Unloaded 'C:\Windows\System32\OpenCL.dll'
The thread 28016 has exited with code 0 (0x0).
'Test.exe' (Win32): Unloaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino_intel_gpu_plugin.dll'
Exception thrown at 0x00007FFDF1BB1CFA in Test.exe: Microsoft C++ exception: ov::Exception at memory location 0x000000DF23B7E0A0.
Exception thrown at 0x00007FFDF1BB1CFA in Test.exe: Microsoft C++ exception: ov::Exception at memory location 0x000000DF23B7B6B0.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.80.0_x64__8wekyb3d8bbwe\ExecutionProvider\openvino_intel_cpu_plugin.dll'. Symbol loading disabled by Include/Exclude setting.
The thread 42676 has exited with code 0 (0x0).
Exception thrown at 0x00007FFDF1BB1CFA in Test.exe: Microsoft C++ exception: ov::NotImplemented at memory location 0x000000DF23B7E360.
Exception thrown at 0x00007FFDF1BB1CFA in Test.exe: Microsoft C++ exception: ov::Exception at memory location 0x000000DF23B7B960.
Exception thrown at 0x00007FFDF1BB1CFA in Test.exe: Microsoft C++ exception: ov::Exception at memory location 0x000000DF23B78F60.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.NVIDIA.TRT-RTX.EP.2_0.0.40.0_x64__8wekyb3d8bbwe\ExecutionProvider\onnxruntime_providers_nv_tensorrt_rtx.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\vcruntime140_1.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\nvml.dll'. Module was built without symbols.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.NVIDIA.TRT-RTX.EP.2_0.0.40.0_x64__8wekyb3d8bbwe\ExecutionProvider\cudart64_12.dll'. Module was built without symbols.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\msvcp140.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\vcruntime140.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\vcruntime140.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\vcruntime140.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Unloaded 'C:\Windows\System32\vcruntime140.dll'
'Test.exe' (Win32): Unloaded 'C:\Windows\System32\vcruntime140.dll'
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.NVIDIA.TRT-RTX.EP.2_0.0.40.0_x64__8wekyb3d8bbwe\ExecutionProvider\tensorrt_plugins.dll'. Module was built without symbols.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\nvcuda.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\gpapi.dll'. Symbol loading disabled by Include/Exclude setting.
File 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.NVIDIA.TRT-RTX.EP.2_0.0.40.0_x64__8wekyb3d8bbwe\ExecutionProvider\tensorrt_rtx_1_5.dll' is signed by NVIDIA Corporation.
[TensorRT Plugin] Library 'tensorrt_rtx_1_5.dll' signature verified successfully.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.NVIDIA.TRT-RTX.EP.2_0.0.40.0_x64__8wekyb3d8bbwe\ExecutionProvider\tensorrt_rtx_1_5.dll'. Module was built without symbols.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_12ab2876952d3f1f\nvcuda64.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\Microsoft.WindowsAppRuntime.2-experimental9_2.2.2.0_x64__8wekyb3d8bbwe\Microsoft.Windows.AI.Speech.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\mfreadwrite.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\mfplat.dll'. Symbols loaded without source information.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\RTWorkQ.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\Microsoft.Windows.AI.Speech.Preview.CPU_1.2607.112.0_x64__8wekyb3d8bbwe\winml\asrmodelapi.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\Microsoft.Windows.AI.Speech.Preview.CPU_1.2607.112.0_x64__8wekyb3d8bbwe\winml\vadapi.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Program Files\WindowsApps\Microsoft.Windows.AI.Speech.Preview.CPU_1.2607.112.0_x64__8wekyb3d8bbwe\winml\asrapi.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\winmmbase.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\wdmaud.drv'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\wdmaud2.drv'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\avrt.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\AudioSes.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\msacm32.drv'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\msacm32.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\midimap.dll'. Symbol loading disabled by Include/Exclude setting.
'Test.exe' (Win32): Loaded 'C:\Windows\System32\ResourcePolicyClient.dll'. Symbol loading disabled by Include/Exclude setting.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Exception thrown at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
Unhandled exception at 0x00007FFBB883D82F (asrmodelapi.dll) in Test.exe: 0xC000001D: Illegal Instruction.
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the supplied C# repro, especially SpeechRecognitionModel.GetReadyState(), EnsureReadyAsync(), and StreamingRecognition.StartContinuousRecognitionAsync(). Review the debug output around the OpenVINO and NVIDIA execution providers; done means the unsupported hardware path no longer crashes and the supported hardware requirements or failure state are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, csharp
- Domain
- ai, audio-video-rtc, desktop
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100