microsoft / microsoft/foundry-local
[Bug] Whisper Base Small model with accuracy issue and service not stable issue
- Dominant language
- C++
- Stars
- 2.6k
- Forks
- 369
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 39
Description
### Describe the issue
One ISV team is developing local AI features based on FL Whisper model, they want to know whether we can help them on the fixing and improvement.
Model : Whisper Base /Small
1. We often observe word segmentation issues in the transcription results, where unnecessary spaces are inserted. For example, "2006年" may be transcribed as "200 6年". Do you have any recommended optimization approaches for improving tokenization and text formatting accuracy?
2. During testing, we occasionally encounter a situation where the model service becomes unresponsive (typically after repeatedly starting and stopping the feature). Once this occurs, the service can no longer be accessed successfully until the PC is rebooted.
o Is there an API available to forcibly restart the LLM service?
o Is there a recommended minimum interval between service startup and shutdown operations to avoid this issue?
### To reproduce
1.We often observe word segmentation issues in the transcription results, where unnecessary spaces are inserted. For example, "2006年" may be transcribed as "200 6年". Do you have any recommended optimization approaches for improving tokenization and text formatting accuracy?
2. During testing, we occasionally encounter a situation where the model service becomes unresponsive (typically after repeatedly starting and stopping the feature). Once this occurs, the service can no longer be accessed successfully until the PC is rebooted.
### Urgency
_No response_
### Platform and architecture
Windows X64
### OS Version
Windows 11 25H2 x64 - 26200.9168
### Installation type
Released package/binary
### Foundry Local version
SDK1.2.3
### API or surface area
C# SDK
### Hardware acceleration/backend
CPU
### Backend/runtime version
_No response_
Contributor guide
Research direction
Start with the C# SDK's Whisper Base/Small service lifecycle and transcription API on Windows x64; reproduce the spacing error and repeated start/stop failure using SDK 1.2.3 on CPU. Done means a defined mitigation for the "2006年" segmentation and a reliable restart or lifecycle recommendation that avoids requiring a reboot.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai, api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100