ml-explore / ml-explore/mlx-examples
[whisper] Feature Request: Add `carry_initial_prompt` parameter to `transcribe()`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Summary
I'd like to request the addition of a carry_initial_prompt parameter to
transcribe(). I'm happy to submit a PR if the maintainers are open to it.
Problem
Currently, initial_prompt is only applied to the first 30-second chunk.
For longer audio files (e.g. meeting recordings, lectures), the vocabulary
and style hints provided via initial_prompt have no effect on subsequent
chunks. This makes it difficult to reliably guide transcription of
domain-specific terms, proper nouns, or formatting style throughout an
entire file.
Prior Art
This has already been addressed in the two other major Whisper implementations:
- openai/whisper — merged in PR #2343 (~1 year ago)
- whisper.cpp — requested in Issue #2564,
implemented in PR #3395,
and released in v1.8.1
mlx-whisper is currently the only major implementation without this feature.
Proposed Change
Add carry_initial_prompt: bool = False to the transcribe() signature:
# mlx_whisper/transcribe.py
def transcribe(
audio,
*,
...
initial_prompt: Optional[str] = None,
carry_initial_prompt: bool = False, # new
...
)
When carry_initial_prompt=True, prepend initial_prompt_tokens to the
prompt of every chunk in the transcription loop, left-slicing to fit within
the 223-token limit — mirroring the upstream implementation:
https://github.com/openai/whisper/blob/main/whisper/transcribe.py
Backward Compatibility
The default value is False, so existing behavior is fully preserved.
Note
As mentioned in the original openai/whisper PR, setting
carry_initial_prompt=True may increase the risk of looping and could
reduce the effectiveness of condition_on_previous_text. Users should be
aware of this trade-off.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in mlx_whisper/transcribe.py at the transcribe() signature and transcription loop, then compare the linked upstream Whisper implementation. Add the optional parameter while preserving the default behavior, carry the initial prompt across chunks within the stated token limit, and verify both enabled and default-disabled behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100