OpenRouter: use provider-reported context_length at session time instead of relying solely on the build-time canonical catalog
- Vorherrschende Sprache
- Rust
- Sterne
- 54.2k
- Forks
- 6.2k
- Ø Merge
- 3 T. 2 Std.
- Gemergte PRs (30 T.)
- 262
Beschreibung
## Summary
Goose resolves a model's context window almost entirely from the **canonical catalog generated from models.dev at build time** (embedded via `include_str!` in `goose-provider-types/src/canonical/data/canonical_models.json`). Models added to models.dev / OpenRouter **after** a release are unknown, so sessions fall back to `DEFAULT_CONTEXT_LIMIT` (128_000). Meanwhile the OpenRouter integration already calls an API that exposes the real number — but throws the metadata away.
The only workaround today is the global `GOOSE_CONTEXT_LIMIT` env/config value, which applies to **every** provider/model and must be manually removed when switching to a smaller-context model.
## Evidence (repro: v1.46.0 desktop, provider `openrouter`, model `stealth/ox-alpha`)
- `GET https://openrouter.ai/api/v1/models` reports `context_length: 1048576`, `top_provider.max_completion_tokens: 131072` for `stealth/ox-alpha`
- models.dev (`https://models.dev/api.json`) lists it today with `limit.context: 1048576`
- Compiled catalogs in **v1.46.0, v1.47.0, and current main** do not contain the model → sessions resolve `context_limit: 128000`
- `crates/goose/src/providers/openrouter.rs::fetch_recommended_models` hits `api/v1/models` but only extracts model ids/names; `context_length` is discarded
- `provider_inventory_models.context_limit` exists in storage but stays `NULL` for OpenRouter models; inventory enrichment (`providers/inventory/mod.rs`) does `canonical.or(fallback)` with `fallback = None`
- Setting `GOOSE_CONTEXT_LIMIT=1048576` works (verified via persisted `sessions.model_config_json`), but it's global, not per-model/per-provider
## Impact
- Stealth / newly listed OpenRouter models are common and rotate quickly; they will systematically miss release-cadence catalog rebuilds
- Wrong limit ⇒ premature compaction; in this case ~87% of usable context is thrown away (1M clamped to 128k)
- Conversely, a forgotten global override produces raw API errors once the user switches to a smaller-context model
## Proposed precedence
1. Explicit user config (existing `GOOSE_CONTEXT_LIMIT`)
2. Canonical catalog entry
3. **Provider-reported metadata** fetched during inventory refresh (OpenRouter `context_length`; also populate the existing `provider_inventory_models.context_limit` column while at it)
4. `DEFAULT_CONTEXT_LIMIT` fallback
Minimal-change option: thread OpenRouter's `context_length` through the existing `fetch_recommended_models` / inventory enrichment path and consume it in model-config resolution whenever `context_limit` is still `None`.
Nice-to-have: a per-model override in `config.yaml` (under the provider entry) so users don't need one global env var.
Happy to test patches against v1.46/v1.47.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.