aaif-goose / aaif-goose/goose

Support local model runners that distribute models as OCI artifacts

未关闭
#11,692 10 条评论 0 个 reaction 已指派 1 人 已被 @ericcurtin 认领 在 GitHub 查看
主要语言
Rust
星标
54.2k
派生
6.2k
平均合并
3 天 2 小时
30 天内合并 PR
262

描述

**What problem would this solve?**

Goose's built-in local options today are Ollama and, via the OpenAI-compatible path, servers like vLLM and LM Studio. A separate class of local runner has appeared that distributes model weights as **OCI artifacts** pulled from a normal container registry, rather than through a bespoke model library. [llmman](https://github.com/llmmanorg/llmman) is one; the packaging format is the [CNCF ModelPack spec](https://github.com/modelpack/model-spec).

Who this affects: people in environments where the container registry is already the governed distribution channel — mirrored, scanned, access-controlled, air-gapped. For them, "pull the model the same way you pull an image" is the difference between a model being usable and not. Right now they have to configure Goose's generic OpenAI-compatible provider by hand, which works but means the runner isn't discoverable in `goose configure`, and its default port and no-API-key behaviour aren't known to Goose.

The narrower version of the problem: Goose has no first-class local provider whose model identifiers are registry references.

**What would a good outcome look like?**

Someone running such a server locally can select it during `goose configure` and start a session without hand-editing config or knowing which generic provider to bend into shape. Its defaults — port, no API key required — are known to Goose, and model identifiers are whatever the runner resolves against its registry rather than a list Goose maintains.

Beyond that I'd rather not prescribe the shape. There's a real design question about whether this deserves its own provider or is better served by improving the generic OpenAI-compatible path, and I don't have a strong view on which is right for the project.

**Possible approaches**

Constraints and trade-offs worth weighing in the design discussion:

- These runners serve an OpenAI-compatible API, so no new wire format is needed. That makes the marginal implementation cost low — but also raises the fair question of whether a distinct provider earns its long-term maintenance cost versus documenting the generic path better.
- Model lists can't be static. Available models are whatever the user has pulled, so discovery has to come from the server's `/v1/models` or be left open.
- Some of these servers also speak the Anthropic Messages API, which may or may not be interesting to Goose.
- There is more than one such runner, so if this is worth doing it's probably worth designing for the category rather than one product.

**Additional context**

I have a working implementation on a local branch (a provider definition in the existing pattern; `cargo check` and `cargo fmt` clean) which I'm deliberately **not** opening as a PR, since CONTRIBUTING is clear that implementation waits for **Ready**. Happy to attach it as a reference diff in this issue if that would help the design discussion, or to throw it away if the conclusion is that the generic path is the better answer.

Disclosure: I used an AI assistant to research the codebase and draft this. I've reviewed it and can speak to any of it.

- [x] I have verified this does not duplicate an existing feature request

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。