OpenNMT / OpenNMT/CTranslate2

Significant Korean accuracy degradation with Whisper inputs shorter than 30 seconds

Open
#2,100 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

CTranslate2 added support for Whisper inputs shorter than 30 seconds.

Does this feature only allow shorter input shapes, or is it expected
to preserve transcription accuracy and timestamp behavior compared
with padding the same audio to 3000 Mel frames?

I tested large-v3-turbo with approximately 100–350 Mel frames.
Inference became significantly faster, but Korean transcription
accuracy decreased substantially, especially for short telephone
utterances.

When the exact same audio file is used repeatedly, the transcription
result is consistent. However, when speakers repeat the same sentence
over a telephone connection, recognition results frequently differ
or become incorrect.

Is there a recommended minimum input length or padding strategy for
short inputs? Are additional changes required for timestamp decoding
when the encoder input is shorter than 3000 Mel frames?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the large-v3-turbo results with 100–350 Mel frames and with the same audio padded to 3000 Mel frames. Compare Korean transcription accuracy and timestamp behavior across repeated files and telephone utterances, then determine whether a minimum input length, padding strategy, or timestamp-decoding change is required.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.