aaif-goose / aaif-goose/goose

OpenRouter: use provider-reported context_length at session time instead of relying solely on the build-time canonical catalog

オープン
#11,530 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Rust
スター
54.2k
フォーク
6.2k
平均マージ
3日 4時間
マージ済み PR(30日)
240

説明

## Summary

Goose resolves a model's context window almost entirely from the **canonical catalog generated from models.dev at build time** (embedded via `include_str!` in `goose-provider-types/src/canonical/data/canonical_models.json`). Models added to models.dev / OpenRouter **after** a release are unknown, so sessions fall back to `DEFAULT_CONTEXT_LIMIT` (128_000). Meanwhile the OpenRouter integration already calls an API that exposes the real number — but throws the metadata away.

The only workaround today is the global `GOOSE_CONTEXT_LIMIT` env/config value, which applies to **every** provider/model and must be manually removed when switching to a smaller-context model.

## Evidence (repro: v1.46.0 desktop, provider `openrouter`, model `stealth/ox-alpha`)

- `GET https://openrouter.ai/api/v1/models` reports `context_length: 1048576`, `top_provider.max_completion_tokens: 131072` for `stealth/ox-alpha`
- models.dev (`https://models.dev/api.json`) lists it today with `limit.context: 1048576`
- Compiled catalogs in **v1.46.0, v1.47.0, and current main** do not contain the model → sessions resolve `context_limit: 128000`
- `crates/goose/src/providers/openrouter.rs::fetch_recommended_models` hits `api/v1/models` but only extracts model ids/names; `context_length` is discarded
- `provider_inventory_models.context_limit` exists in storage but stays `NULL` for OpenRouter models; inventory enrichment (`providers/inventory/mod.rs`) does `canonical.or(fallback)` with `fallback = None`
- Setting `GOOSE_CONTEXT_LIMIT=1048576` works (verified via persisted `sessions.model_config_json`), but it's global, not per-model/per-provider

## Impact

- Stealth / newly listed OpenRouter models are common and rotate quickly; they will systematically miss release-cadence catalog rebuilds
- Wrong limit ⇒ premature compaction; in this case ~87% of usable context is thrown away (1M clamped to 128k)
- Conversely, a forgotten global override produces raw API errors once the user switches to a smaller-context model

## Proposed precedence

1. Explicit user config (existing `GOOSE_CONTEXT_LIMIT`)
2. Canonical catalog entry
3. **Provider-reported metadata** fetched during inventory refresh (OpenRouter `context_length`; also populate the existing `provider_inventory_models.context_limit` column while at it)
4. `DEFAULT_CONTEXT_LIMIT` fallback

Minimal-change option: thread OpenRouter's `context_length` through the existing `fetch_recommended_models` / inventory enrichment path and consume it in model-config resolution whenever `context_limit` is still `None`.

Nice-to-have: a per-model override in `config.yaml` (under the provider entry) so users don't need one global env var.

Happy to test patches against v1.46/v1.47.

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

問題は OpenRouter provider 統合にあります。まず `crates/goose/src/providers/openrouter.rs::fetch_recommended_models` を調べて、`context_length` がどこで破棄されているかを確認してください。次に、`providers/inventory/mod.rs` のインベントリ補完ロジックと `provider_inventory_models.context_limit` 列を確認してください。目的は、provider が報告したコンテキスト長をインベントリの更新処理を通じて渡し、正規カタログにモデルがない場合の model-config 解決でそれを使用することです。`stealth/ox-alpha` のようなモデルでテストし、`sessions.model_config_json` にある解決後の上限を確認して検証してください。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
rust
領域
ai-infra-agents, backend
issue の種類
機能追加
難易度
3/5
見積もり時間
1〜2日
活発さ
活発
明瞭さ
明確に書かれている
初心者へのやさしさ
65/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。