OpenNMT / OpenNMT/CTranslate2

Allow no-timestamp Whisper decoding beyond 224 tokens

Open Beginner friendly
#2,074 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

Problem

WhisperReplica::generate currently limits decoding with:

decoding_options.max_length = std::min(total_max_length / 2, total_max_length - start_step);

This applies the original 224-token sampling limit even when the prompt ends in <|notimestamps|>. For languages that rely heavily on Whisper's byte-level fallback, 224 tokens can be exhausted by a short audio clip even though the model has a 448-position text context.

Reproduction

Using faster-whisper with large-v3, vad_filter=False, and without_timestamps=True, both an Armenian and a Georgian sample stopped at exactly 224 re-tokenized visible-text tokens. One output ended with the Unicode replacement character because generation stopped mid-byte sequence.

The same audio with timestamp decoding used more than 224 visible-text tokens because timestamp boundaries allow faster-whisper to continue the transcription loop, but disabling timestamps produces one decode for the complete audio window.

Proposed behavior

When the task prompt ends with <|notimestamps|>, allow decoding to use the remaining text context:

total_max_length - start_step

Keep the existing half-context limit for timestamped decoding. This preserves the standard timestamp path while allowing explicit no-timestamp callers to use the model's remaining decoder positions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at WhisperReplica::generate and inspect how decoding_options.max_length is computed from the task prompt and start_step. Reproduce with no-timestamp decoding and verify that prompts ending in <|notimestamps|> can use the remaining text context, while timestamped decoding keeps the existing half-context limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.