Provider request: OpenLLM (local BYOK/subscription gateway with fallback chains)
- Dominant language
- Rust
- Stars
- 54.2k
- Forks
- 6.2k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 262
Description
**What problem would this solve?**
Users who route their LLM traffic through [OpenLLM](https://openllm.sh) — a local gateway (`127.0.0.1:8787`) that fans out to Anthropic, OpenAI, Google, Alibaba, AWS Bedrock, OpenCode and custom endpoints, plus subscription plans (Claude, ChatGPT, Kimi, Grok, Cursor), with fallback chains — currently have no first-class way to use it from Goose. They must hand-write a `custom_providers/` JSON and it never appears in the provider picker alongside the other local backends (LM Studio, llama-swap, oMLX). The daemon already holds the credentials, so asking for an API key in Goose is also confusing.
**What would a good outcome look like?**
OpenLLM is discoverable in the provider picker like the other keyless local providers. Selecting it needs no API key, and the model list reflects whatever the user has configured in the daemon (`GET /v1/models`, which also reports `meta.n_ctx` for context-limit discovery).
**Possible approaches**
Same shape as the bundled local keyless providers: a declarative definition with `engine: "openai"`, `OPENLLM_HOST` defaulting to `http://127.0.0.1:8787`, `requires_auth: false`, `dynamic_models: true`, and `skip_canonical_filtering` so the gateway's curated list shows through verbatim; static `ultra` / `plus` / `lite` tier aliases as fallback when the daemon is unreachable.
Proposed files:
- `crates/goose-providers/src/declarative/definitions/openllm.json`
- `crates/goose-providers/src/declarative.rs` (`expose_declarative_providers!` list)
- `documentation/docs/getting-started/providers.md` (table row + setup)
- Optionally, docs for the OpenLLM MCP server (`documentation/docs/mcp/openllm-mcp.md`, `documentation/static/servers.json`) — happy to split this into a separate issue/PR if the maintainers prefer the provider change to stand alone.
Verification: build, run `goose configure`, pick OpenLLM with the daemon running, confirm the dynamic model list and a chat turn complete; then stop the daemon and confirm the static tier aliases still appear.
**Additional context**
A PR implementing this is already open as #11669; per CONTRIBUTING I'm filing this issue so it can be triaged to Ready first, and I've converted the PR to a draft in the meantime. Disclosure: I'm on the OpenLLM team.
- [x] I have verified this does not duplicate an existing feature request
Contributor guide
Assessment
This issue has not been assessed yet.