openai / openai/codex

Bring composer dictation to the CLI/TUI

Open
#46,698 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CLI enhancement TUI
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What variant of Codex are you using?

CLI (TUI), main @ 7498521d2, Windows 11

What feature would you like to see?

Bring composer dictation to the CLI/TUI, so speech can be transcribed into the composer as an editable draft the user reviews and submits themselves.

Dictation already exists in the Codex desktop app — issues like #46403, #45065, #46452, and #37623 all reference a "Start Dictation" shortcut and composer/global dictation there. The CLI has no equivalent. /voice starts a full realtime conversation (the model listens, replies with audio, drives its own turns), which is a different mode of working from using voice as an input method for the composer.

This matters most for longer prompts. Describing a refactor or a bug takes a paragraph, and speaking one is faster than typing it, but the text still needs reviewing before anything is sent.

Text should land in the composer, not auto-submit. Transcription reliably mangles code identifiers — codex-rs becomes "codex RS", underscores vanish, unwrap splits into "un wrap". Since Codex executes shell commands and edits files, auto-submitting unreviewed transcription would be risky. Keeping the final keystroke with the user makes misrecognition a non-event.

Suggested behavior:

  • A configurable toggle binding starts and stops capture.
  • Partial transcripts stream into the composer and self-correct as the backend revises them.
  • A waveform driven by real mic amplitude, so it is visibly clear the mic is live.
  • Esc discards the audio and restores the composer exactly as it was.
  • Text inserts at the cursor, so typing and dictation compose.
Additional information

A terminal-specific constraint worth flagging, since it does not apply to the app.

Hold-to-talk depends on the terminal reporting key-release events. codex-rs/tui/src/tui/keyboard_modes.rs:238 deliberately omits REPORT_EVENT_TYPES on Ghostty, iTerm2, and tmux without extended-keys-format = csi-u, because those terminals leak or drop release events. On them a key-up never arrives, so a hold-to-talk binding would open the mic and never close it.

A plain toggle avoids this entirely. If hold-to-talk is wanted, it would need capability gating (the probe at codex-rs/tui/src/terminal_probe.rs:465 already detects the active flags) plus an unconditional timeout as a backstop.

Worth noting that #46403 reports the opposite problem in the app — a user who wants toggle and got push-to-talk — which suggests toggle is the safer default in the terminal too.

Existing building blocks. Much of what this needs is already in the tree:

  • RealtimeSessionMode::Transcription (codex-rs/codex-api/src/endpoint/realtime_websocket/methods_v2.rs:145) is configured for exactly this: transcription model, no TTS, no server VAD, no tools.
  • codex-rs/voice-host/ handles mic capture, Opus, and WebRTC.
  • RealtimeWebrtcSessionHandle::take_microphone_peak() exposes live mic amplitude.
  • codex-rs/tui/src/bottom_pane/voice_strip.rs renders a waveform.
  • conversation.item.input_audio_transcription.delta / .completed carry the transcript.

One gap: transcription mode is selectable only as a global config setting (realtime.type = "transcription", read at codex-rs/core/src/realtime_conversation.rs:1478). Neither ThreadRealtimeStartParams nor ConversationStartParams carries a session-mode field, so a session cannot request transcription mode while leaving /voice conversational. A per-session sessionMode would let dictation and voice conversation coexist.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with codex-rs/tui/src/bottom_pane/voice_strip.rs, the terminal capability probe in codex-rs/tui/src/terminal_probe.rs, and the realtime session definitions in codex-rs/codex-api/src/endpoint/realtime_websocket/methods_v2.rs. Trace how transcription deltas and microphone peaks could reach the composer while keeping /voice conversational. Done means toggle-based capture inserts editable text at the cursor, supports cancellation and waveform feedback, and does not auto-submit.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
audio-video-rtc, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.