anthropics / anthropics/claude-code

Third-party gateway mode: no capability discovery (1M silently capped at 200k), stale model selection, settings.json env ignored/mangled

Aperta
#88,345 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area:desktop area:model area:providers bug platform:macos
Lingua principale
Python
Stelle
145k
Fork
23.1k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

## Summary

Using Claude Code / Claude Desktop against a third-party LLM gateway (`ANTHROPIC_BASE_URL` pointing at an OpenAI-compatible corporate gateway, TrueFoundry-hosted) works for inference, but the surrounding "gateway mode" experience has several defects that each required reverse-engineering undocumented config to work around.

## Environment

- macOS (Darwin 25.5), Claude Desktop app in third-party-inference mode (`~/Library/Application Support/Claude-3p/`), also reproducible with Claude Code CLI
- Gateway: OpenAI-compatible `/v1/models` endpoint behind `ANTHROPIC_BASE_URL` (base URL includes a path component, e.g. `https://gateway.example.com/api/llm`)
- Models served: `claude-fable-5`, `claude-opus-5`, `claude-haiku-4-5`

## Problem 1: No way for gateway users to get 1M context — capability discovery silently caps everything at 200k

A gateway's `/v1/models` is a bare OpenAI-style list with no `supports_1m_context` capability field, so model auto-discovery gives the app no 1M info and **every model defaults to a 200k window**, even models that support 1M and even when the gateway passes `anthropic-beta: context-1m-2025-08-07` through correctly (verified with a live request — the gateway is not the limiter).

The only workaround we found was hand-editing an **undocumented** config: adding `inferenceModels` entries with `supports1m: true` (and `prefer1m: true`), plus explicitly setting `modelDiscoveryEnabled: true` — because a present model list can otherwise cause discovery to be skipped entirely, which is itself surprising.

**Ask:** a documented, supported way to declare per-model capabilities (context window, 1M support) for gateway deployments — or honor a standard capability field if the gateway provides one.

## Problem 2: `modelPrefer1mContext` never applies to an existing saved selection

`modelPrefer1mContext: true` only changes which picker row a user with *no saved model selection* starts on, and only if a 1M variant already exists. It never upgrades a saved selection and never creates 1M variants. In practice: after fixing capability config, the user must know to manually re-pick the 1M row in the model picker once after restart. Nothing in the UI or docs indicates this.

**Ask:** either re-evaluate the preference when new 1M variants appear, or document the "re-pick once after config change" requirement.

## Problem 3: Desktop app ignores `settings.json` gateway env and mangles the base URL it does inject

With `ANTHROPIC_BASE_URL` / `ANTHROPIC_AUTH_TOKEN` set in `~/.claude/settings.json` → `env`, the desktop app does not pass them through — it injects its own gateway configuration from its host-credentials store. Observably, sandboxed shells inside a session see `ANTHROPIC_BASE_URL` set to the gateway **origin with the path component stripped** (`https://gateway.example.com` instead of `https://gateway.example.com/api/llm`). For gateways mounted under a path (common for corporate gateways), anything that trusts that env var gets a broken endpoint.

**Ask:** define and document the precedence between `settings.json` env and the desktop app's own gateway config, and preserve the full base URL (including path) wherever it's propagated.

## Why this matters

Enterprise/gateway deployments are presumably exactly the users Anthropic wants on first-party tooling. Right now each of the three items above fails silently (200k cap, stale selection, stripped URL) and the fixes live in undocumented config keys discovered by trial and error.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start by reproducing the three gateway-mode cases using the `/v1/models` endpoint, the model picker, `~/.claude/settings.json` `env`, and the desktop host-credentials store. Trace capability discovery, saved model selection, and gateway environment propagation. Done means gateway users can declare or receive model capabilities, preferences handle new 1M variants, and the full base URL and documented environment precedence are preserved.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
api, cli, cloud, desktop
Tipo di issue
Bug
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.