Allow no-timestamp Whisper decoding beyond 224 tokens
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Problem
WhisperReplica::generate currently limits decoding with:
decoding_options.max_length = std::min(total_max_length / 2, total_max_length - start_step);
This applies the original 224-token sampling limit even when the prompt ends in <|notimestamps|>. For languages that rely heavily on Whisper's byte-level fallback, 224 tokens can be exhausted by a short audio clip even though the model has a 448-position text context.
Reproduction
Using faster-whisper with large-v3, vad_filter=False, and without_timestamps=True, both an Armenian and a Georgian sample stopped at exactly 224 re-tokenized visible-text tokens. One output ended with the Unicode replacement character because generation stopped mid-byte sequence.
The same audio with timestamp decoding used more than 224 visible-text tokens because timestamp boundaries allow faster-whisper to continue the transcription loop, but disabling timestamps produces one decode for the complete audio window.
Proposed behavior
When the task prompt ends with <|notimestamps|>, allow decoding to use the remaining text context:
total_max_length - start_step
Keep the existing half-context limit for timestamped decoding. This preserves the standard timestamp path while allowing explicit no-timestamp callers to use the model's remaining decoder positions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at WhisperReplica::generate and inspect how decoding_options.max_length is computed from the task prompt and start_step. Reproduce with no-timestamp decoding and verify that prompts ending in <|notimestamps|> can use the remaining text context, while timestamped decoding keeps the existing half-context limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100