anthropics / anthropics/claude-code

[FEATURE] Configurable audio transcription endpoint for dictation in Cowork on 3P

Abierto
#92,822 0 comentarios 0 reacciones 0 asignados Ver en GitHub
area:cowork area:self-hosted-environments enhancement
Lenguaje dominante
Python
Estrellas
145k
Forks
23.1k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

### Preflight Checklist

- [x] I have searched [existing requests](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20label%3Aenhancement) and this feature hasn't been requested yet
- [x] This is a single feature request (not multiple features)

### Problem Statement

Dictation is unavailable in Claude Desktop when running in third-party inference (3P) mode. The Help Center states dictation is available in Claude Cowork, but in a 3P deployment there is no way to use it.

The underlying reason appears structural rather than a bug. In 3P mode:

User identity is local device identity only — there is no Anthropic account credential to authenticate a call to an Anthropic-hosted transcription service.
The only redirectable surface is inference. Per the gateway documentation, the required contract is POST /v1/messages with streaming and tool use, plus an optional GET /v1/models. The Anthropic Messages API has no transcription endpoint.
The configuration reference has no key for a speech-to-text endpoint. inferenceProvider, inferenceGatewayBaseUrl, inferenceCustomHeaders and the rest all describe inference only.

This is arguably correct default behavior — the stated 3P security posture is that prompts, responses, files and tool outputs go only to the configured inference endpoint. Sending microphone audio to Anthropic would break exactly that guarantee. But the result is that dictation is simply unavailable to 3P users, with no supported path to enable it against infrastructure they control.

For regulated deployments this is the group most likely to want dictation routed through their own stack, and least able to use the first-party path.

### Proposed Solution

Add managed-configuration keys for a transcription endpoint, parallel to the existing gateway inference keys — for example:



Key | Type | Description
-- | -- | --
transcriptionProvider | enum | none (default) or gateway
transcriptionBaseUrl | string | Base URL of an OpenAI-compatible transcription endpoint
transcriptionApiKey | string | Credential for that endpoint
transcriptionAuthScheme | enum | bearer (default) or x-api-key
transcriptionModel | string | Model ID the endpoint expects


Contract: POST /v1/audio/transcriptions, OpenAI-compatible multipart (file, model). Default off, so existing 3P deployments are unchanged and no audio leaves the device unless an admin opts in.


This is a small surface because the OpenAI transcription shape is already near-universal. LiteLLM — named in the gateway docs as a supported gateway — already exposes /v1/audio/transcriptions with OpenAI, Azure, Fireworks AI, Groq, Deepgram, Mistral (Voxtral) and OVHcloud backends, plus per-request fallbacks. Portkey and other gateways in this category are similar. Operators would not need to change anything server-side; the gap is purely that the client has no way to be pointed at an endpoint that already exists.


It also fits the existing security model cleanly: audio would go to the same operator-controlled infrastructure as inference, keeping the "only your configured endpoint" property intact, and would be auditable in the same gateway logs.

### Alternative Solutions

OS-level dictation (macOS Fn Fn, Windows Win+H) — works today and injects into the Cowork composer. This is the current workaround, but accuracy and language coverage are weaker than a Whisper-class model, it varies by platform, and it can't be centrally configured or audited by an admin.

Local Whisper with a hotkey (whisper.cpp, MacWhisper) — better accuracy, but per-user setup with no MDM story, and duplicates capacity the org's gateway may already provide.

Routing transcription through Anthropic in 3P mode — rejected, for the data-residency reasons above.
Reusing inferenceGatewayBaseUrl and assuming the gateway also serves /v1/audio/transcriptions — simpler, but conflates two services that are often separate deployments, and would break gateways serving only the Messages API. Separate keys seem safer.

### Priority

Critical - Blocking my work

### Feature Category

Interactive mode (TUI)

### Use Case Example

_No response_

### Additional Context

_No response_

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Start with the configuration reference and the existing inferenceProvider, inferenceGatewayBaseUrl, inferenceCustomHeaders, and related managed-configuration handling. Trace how Cowork dictation currently selects its transcription service, then define the integration boundary for the proposed keys and POST /v1/audio/transcriptions contract. Done means opt-in configuration, supported authentication and model settings, and no audio routing by default.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Área
api, backend-api-design
Tipo de issue
Nueva funcionalidad
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Activo
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.