anthropics / anthropics/claude-code
[FEATURE] Configurable audio transcription endpoint for dictation in Cowork on 3P
- 主要言語
- Python
- スター
- 145k
- フォーク
- 23.1k
- PR マージ指標
- PR 指標を取得中
説明
### Preflight Checklist
- [x] I have searched [existing requests](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20label%3Aenhancement) and this feature hasn't been requested yet
- [x] This is a single feature request (not multiple features)
### Problem Statement
Dictation is unavailable in Claude Desktop when running in third-party inference (3P) mode. The Help Center states dictation is available in Claude Cowork, but in a 3P deployment there is no way to use it.
The underlying reason appears structural rather than a bug. In 3P mode:
User identity is local device identity only — there is no Anthropic account credential to authenticate a call to an Anthropic-hosted transcription service.
The only redirectable surface is inference. Per the gateway documentation, the required contract is POST /v1/messages with streaming and tool use, plus an optional GET /v1/models. The Anthropic Messages API has no transcription endpoint.
The configuration reference has no key for a speech-to-text endpoint. inferenceProvider, inferenceGatewayBaseUrl, inferenceCustomHeaders and the rest all describe inference only.
This is arguably correct default behavior — the stated 3P security posture is that prompts, responses, files and tool outputs go only to the configured inference endpoint. Sending microphone audio to Anthropic would break exactly that guarantee. But the result is that dictation is simply unavailable to 3P users, with no supported path to enable it against infrastructure they control.
For regulated deployments this is the group most likely to want dictation routed through their own stack, and least able to use the first-party path.
### Proposed Solution
Add managed-configuration keys for a transcription endpoint, parallel to the existing gateway inference keys — for example:
Key | Type | Description
-- | -- | --
transcriptionProvider | enum | none (default) or gateway
transcriptionBaseUrl | string | Base URL of an OpenAI-compatible transcription endpoint
transcriptionApiKey | string | Credential for that endpoint
transcriptionAuthScheme | enum | bearer (default) or x-api-key
transcriptionModel | string | Model ID the endpoint expects
Contract: POST /v1/audio/transcriptions, OpenAI-compatible multipart (file, model). Default off, so existing 3P deployments are unchanged and no audio leaves the device unless an admin opts in.
This is a small surface because the OpenAI transcription shape is already near-universal. LiteLLM — named in the gateway docs as a supported gateway — already exposes /v1/audio/transcriptions with OpenAI, Azure, Fireworks AI, Groq, Deepgram, Mistral (Voxtral) and OVHcloud backends, plus per-request fallbacks. Portkey and other gateways in this category are similar. Operators would not need to change anything server-side; the gap is purely that the client has no way to be pointed at an endpoint that already exists.
It also fits the existing security model cleanly: audio would go to the same operator-controlled infrastructure as inference, keeping the "only your configured endpoint" property intact, and would be auditable in the same gateway logs.
### Alternative Solutions
OS-level dictation (macOS Fn Fn, Windows Win+H) — works today and injects into the Cowork composer. This is the current workaround, but accuracy and language coverage are weaker than a Whisper-class model, it varies by platform, and it can't be centrally configured or audited by an admin.
Local Whisper with a hotkey (whisper.cpp, MacWhisper) — better accuracy, but per-user setup with no MDM story, and duplicates capacity the org's gateway may already provide.
Routing transcription through Anthropic in 3P mode — rejected, for the data-residency reasons above.
Reusing inferenceGatewayBaseUrl and assuming the gateway also serves /v1/audio/transcriptions — simpler, but conflates two services that are often separate deployments, and would break gateways serving only the Messages API. Separate keys seem safer.
### Priority
Critical - Blocking my work
### Feature Category
Interactive mode (TUI)
### Use Case Example
_No response_
### Additional Context
_No response_
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
Start with the configuration reference and the existing inferenceProvider, inferenceGatewayBaseUrl, inferenceCustomHeaders, and related managed-configuration handling. Trace how Cowork dictation currently selects its transcription service, then define the integration boundary for the proposed keys and POST /v1/audio/transcriptions contract. Done means opt-in configuration, supported authentication and model settings, and no audio routing by default.
索引モデルが issue の本文から書いたものです。
評価
- 領域
- api, backend-api-design
- issue の種類
- 機能追加
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 活発
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100