ml-explore / ml-explore/mlx-examples

[whisper] Feature Request: Add `carry_initial_prompt` parameter to `transcribe()`

Open
#1,410 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

Summary

I'd like to request the addition of a carry_initial_prompt parameter to
transcribe(). I'm happy to submit a PR if the maintainers are open to it.

Problem

Currently, initial_prompt is only applied to the first 30-second chunk.
For longer audio files (e.g. meeting recordings, lectures), the vocabulary
and style hints provided via initial_prompt have no effect on subsequent
chunks. This makes it difficult to reliably guide transcription of
domain-specific terms, proper nouns, or formatting style throughout an
entire file.

Prior Art

This has already been addressed in the two other major Whisper implementations:

  • openai/whisper — merged in PR #2343 (~1 year ago)
  • whisper.cpp — requested in Issue #2564,
    implemented in PR #3395,
    and released in v1.8.1

mlx-whisper is currently the only major implementation without this feature.

Proposed Change

Add carry_initial_prompt: bool = False to the transcribe() signature:

# mlx_whisper/transcribe.py
def transcribe(
    audio,
    *,
    ...
    initial_prompt: Optional[str] = None,
    carry_initial_prompt: bool = False,  # new
    ...
)

When carry_initial_prompt=True, prepend initial_prompt_tokens to the
prompt of every chunk in the transcription loop, left-slicing to fit within
the 223-token limit — mirroring the upstream implementation:

https://github.com/openai/whisper/blob/main/whisper/transcribe.py

Backward Compatibility

The default value is False, so existing behavior is fully preserved.

Note

As mentioned in the original openai/whisper PR, setting
carry_initial_prompt=True may increase the risk of looping and could
reduce the effectiveness of condition_on_previous_text. Users should be
aware of this trade-off.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in mlx_whisper/transcribe.py at the transcribe() signature and transcription loop, then compare the linked upstream Whisper implementation. Add the optional parameter while preserving the default behavior, carry the initial prompt across chunks within the stated token limit, and verify both enabled and default-disabled behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.