aaif-goose / aaif-goose/goose

Provider request: OpenLLM (local BYOK/subscription gateway with fallback chains)

Đang mở
#11,791 2 bình luận 0 reaction 1 người được giao Được @0xxmemo nhận Xem trên GitHub
Ngôn ngữ chính
Rust
Star
54.2k
Fork
6.2k
Merge trung bình
3 ngày 4 giờ
Pull request đã merge (30 ngày)
240

Mô tả

**What problem would this solve?**

Users who route their LLM traffic through [OpenLLM](https://openllm.sh) — a local gateway (`127.0.0.1:8787`) that fans out to Anthropic, OpenAI, Google, Alibaba, AWS Bedrock, OpenCode and custom endpoints, plus subscription plans (Claude, ChatGPT, Kimi, Grok, Cursor), with fallback chains — currently have no first-class way to use it from Goose. They must hand-write a `custom_providers/` JSON and it never appears in the provider picker alongside the other local backends (LM Studio, llama-swap, oMLX). The daemon already holds the credentials, so asking for an API key in Goose is also confusing.

**What would a good outcome look like?**

OpenLLM is discoverable in the provider picker like the other keyless local providers. Selecting it needs no API key, and the model list reflects whatever the user has configured in the daemon (`GET /v1/models`, which also reports `meta.n_ctx` for context-limit discovery).

**Possible approaches**

Same shape as the bundled local keyless providers: a declarative definition with `engine: "openai"`, `OPENLLM_HOST` defaulting to `http://127.0.0.1:8787`, `requires_auth: false`, `dynamic_models: true`, and `skip_canonical_filtering` so the gateway's curated list shows through verbatim; static `ultra` / `plus` / `lite` tier aliases as fallback when the daemon is unreachable.

Proposed files:
- `crates/goose-providers/src/declarative/definitions/openllm.json`
- `crates/goose-providers/src/declarative.rs` (`expose_declarative_providers!` list)
- `documentation/docs/getting-started/providers.md` (table row + setup)
- Optionally, docs for the OpenLLM MCP server (`documentation/docs/mcp/openllm-mcp.md`, `documentation/static/servers.json`) — happy to split this into a separate issue/PR if the maintainers prefer the provider change to stand alone.

Verification: build, run `goose configure`, pick OpenLLM with the daemon running, confirm the dynamic model list and a chat turn complete; then stop the daemon and confirm the static tier aliases still appear.

**Additional context**

A PR implementing this is already open as #11669; per CONTRIBUTING I'm filing this issue so it can be triaged to Ready first, and I've converted the PR to a draft in the meantime. Disclosure: I'm on the OpenLLM team.

- [x] I have verified this does not duplicate an existing feature request

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

The work involves adding a new provider definition in `crates/goose-providers/src/declarative/definitions/openllm.json` and exposing it in `crates/goose-providers/src/declarative.rs`. Start by examining the existing local keyless provider definitions to understand the pattern. Verify the change by building, running `goose configure`, and testing with the OpenLLM daemon running and stopped to confirm dynamic and static model lists work.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
rust
Lĩnh vực
backend, cli, tooling
Loại issue
Tính năng
Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Đặc tả rõ ràng
Mức phù hợp với người mới
65/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.