aaif-goose / aaif-goose/goose

Support local model runners that distribute models as OCI artifacts

Đang mở
#11,692 10 bình luận 0 reaction 1 người được giao Được @ericcurtin nhận Xem trên GitHub
Ngôn ngữ chính
Rust
Star
54.2k
Fork
6.2k
Merge trung bình
3 ngày 4 giờ
Pull request đã merge (30 ngày)
240

Mô tả

**What problem would this solve?**

Goose's built-in local options today are Ollama and, via the OpenAI-compatible path, servers like vLLM and LM Studio. A separate class of local runner has appeared that distributes model weights as **OCI artifacts** pulled from a normal container registry, rather than through a bespoke model library. [llmman](https://github.com/llmmanorg/llmman) is one; the packaging format is the [CNCF ModelPack spec](https://github.com/modelpack/model-spec).

Who this affects: people in environments where the container registry is already the governed distribution channel — mirrored, scanned, access-controlled, air-gapped. For them, "pull the model the same way you pull an image" is the difference between a model being usable and not. Right now they have to configure Goose's generic OpenAI-compatible provider by hand, which works but means the runner isn't discoverable in `goose configure`, and its default port and no-API-key behaviour aren't known to Goose.

The narrower version of the problem: Goose has no first-class local provider whose model identifiers are registry references.

**What would a good outcome look like?**

Someone running such a server locally can select it during `goose configure` and start a session without hand-editing config or knowing which generic provider to bend into shape. Its defaults — port, no API key required — are known to Goose, and model identifiers are whatever the runner resolves against its registry rather than a list Goose maintains.

Beyond that I'd rather not prescribe the shape. There's a real design question about whether this deserves its own provider or is better served by improving the generic OpenAI-compatible path, and I don't have a strong view on which is right for the project.

**Possible approaches**

Constraints and trade-offs worth weighing in the design discussion:

- These runners serve an OpenAI-compatible API, so no new wire format is needed. That makes the marginal implementation cost low — but also raises the fair question of whether a distinct provider earns its long-term maintenance cost versus documenting the generic path better.
- Model lists can't be static. Available models are whatever the user has pulled, so discovery has to come from the server's `/v1/models` or be left open.
- Some of these servers also speak the Anthropic Messages API, which may or may not be interesting to Goose.
- There is more than one such runner, so if this is worth doing it's probably worth designing for the category rather than one product.

**Additional context**

I have a working implementation on a local branch (a provider definition in the existing pattern; `cargo check` and `cargo fmt` clean) which I'm deliberately **not** opening as a PR, since CONTRIBUTING is clear that implementation waits for **Ready**. Happy to attach it as a reference diff in this issue if that would help the design discussion, or to throw it away if the conclusion is that the generic path is the better answer.

Disclosure: I used an AI assistant to research the codebase and draft this. I've reviewed it and can speak to any of it.

- [x] I have verified this does not duplicate an existing feature request

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Look at the existing provider definitions in the codebase, likely under a `providers` or `local` module. Examine how Ollama and the generic OpenAI-compatible provider are implemented. The design discussion centers on whether to create a new provider or extend the generic one. Start by understanding the configuration flow in `goose configure` and how model discovery works. 'Done' means a user can select this runner type during configuration without manual edits.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
docker, kubernetes, rust
Lĩnh vực
ai-infra-agents, backend-api-design, cli
Loại issue
Tính năng
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
45/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.