aaif-goose / aaif-goose/goose

Make provider model metadata (context limits, reasoning) live-first instead of build-time-catalog-only

Open
#11,558 2 comments 0 reactions 1 assignee Assigned to @jamadeo View on GitHub
Dominant language
Rust
Stars
54.2k
Forks
6.2k
Avg merge
3d 4h
Merged PRs (30d)
240

Description

### Describe the improvement

Goose's model metadata (context window, max output tokens, reasoning support) currently comes from a build-time snapshot of models.dev, refreshed only when goose is released. For OpenRouter specifically, `/api/v1/models` already exposes `context_length`, `top_provider.max_completion_tokens`, and reasoning support via `supported_parameters` at runtime — but the inventory refresh keeps only model IDs, so any model missing from the bundled catalog falls back to a hardcoded 128k context limit and never shows a thinking control.

This tracking issue covers making provider model metadata **live-first**: fetch it from the provider when available, store it in the inventory cache, and treat the build-time catalog as a per-field fallback rather than the source of truth.

### Why is this needed?

Day-zero model releases (stealth/private/org-aliased models) are common on OpenRouter. Today those sessions silently run with wrong compaction thresholds and no reasoning support until a new goose release ships an updated catalog.

### Scope / related work

- [x] Initial implementation: #11531 (OpenRouter live metadata → inventory → session config + canonical-info endpoint)
- [ ] Extend metadata fetching to other providers with rich models APIs
- [ ] Mid-session convergence after a manual inventory refresh (currently applies at session prep and model switch)
- [ ] Decide on provenance tracking for `ModelConfig` fields so explicit user overrides vs catalog-derived defaults can be distinguished structurally instead of by value comparison

### Additional context

Other OpenRouter-related changes in flight may touch `crates/goose/src/providers/openrouter.rs` and the inventory refresh path (`providers/inventory/`) — coordinating here to avoid collisions.

Contributor guide

Open the contributing guide

Research direction

Start by reading the initial implementation PR #11531 to understand the changes to `crates/goose/src/providers/openrouter.rs` and the inventory refresh path (`providers/inventory/`). Examine how model metadata is currently fetched from `/api/v1/models` and cached. The goal is to extend this live-first approach to other providers, ensure mid-session updates work, and track field provenance. Running existing tests for the inventory and provider modules will help verify changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai-infra-agents, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.