Significant Korean accuracy degradation with Whisper inputs shorter than 30 seconds
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
CTranslate2 added support for Whisper inputs shorter than 30 seconds.
Does this feature only allow shorter input shapes, or is it expected
to preserve transcription accuracy and timestamp behavior compared
with padding the same audio to 3000 Mel frames?
I tested large-v3-turbo with approximately 100–350 Mel frames.
Inference became significantly faster, but Korean transcription
accuracy decreased substantially, especially for short telephone
utterances.
When the exact same audio file is used repeatedly, the transcription
result is consistent. However, when speakers repeat the same sentence
over a telephone connection, recognition results frequently differ
or become incorrect.
Is there a recommended minimum input length or padding strategy for
short inputs? Are additional changes required for timestamp decoding
when the encoder input is shorter than 3000 Mel frames?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the large-v3-turbo results with 100–350 Mel frames and with the same audio padded to 3000 Mel frames. Compare Korean transcription accuracy and timestamp behavior across repeated files and telephone utterances, then determine whether a minimum input length, padding strategy, or timestamp-decoding change is required.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100