Support local model runners that distribute models as OCI artifacts
- 主要语言
- Rust
- 星标
- 54.2k
- 派生
- 6.2k
- 平均合并
- 3 天 2 小时
- 30 天内合并 PR
- 262
描述
**What problem would this solve?**
Goose's built-in local options today are Ollama and, via the OpenAI-compatible path, servers like vLLM and LM Studio. A separate class of local runner has appeared that distributes model weights as **OCI artifacts** pulled from a normal container registry, rather than through a bespoke model library. [llmman](https://github.com/llmmanorg/llmman) is one; the packaging format is the [CNCF ModelPack spec](https://github.com/modelpack/model-spec).
Who this affects: people in environments where the container registry is already the governed distribution channel — mirrored, scanned, access-controlled, air-gapped. For them, "pull the model the same way you pull an image" is the difference between a model being usable and not. Right now they have to configure Goose's generic OpenAI-compatible provider by hand, which works but means the runner isn't discoverable in `goose configure`, and its default port and no-API-key behaviour aren't known to Goose.
The narrower version of the problem: Goose has no first-class local provider whose model identifiers are registry references.
**What would a good outcome look like?**
Someone running such a server locally can select it during `goose configure` and start a session without hand-editing config or knowing which generic provider to bend into shape. Its defaults — port, no API key required — are known to Goose, and model identifiers are whatever the runner resolves against its registry rather than a list Goose maintains.
Beyond that I'd rather not prescribe the shape. There's a real design question about whether this deserves its own provider or is better served by improving the generic OpenAI-compatible path, and I don't have a strong view on which is right for the project.
**Possible approaches**
Constraints and trade-offs worth weighing in the design discussion:
- These runners serve an OpenAI-compatible API, so no new wire format is needed. That makes the marginal implementation cost low — but also raises the fair question of whether a distinct provider earns its long-term maintenance cost versus documenting the generic path better.
- Model lists can't be static. Available models are whatever the user has pulled, so discovery has to come from the server's `/v1/models` or be left open.
- Some of these servers also speak the Anthropic Messages API, which may or may not be interesting to Goose.
- There is more than one such runner, so if this is worth doing it's probably worth designing for the category rather than one product.
**Additional context**
I have a working implementation on a local branch (a provider definition in the existing pattern; `cargo check` and `cargo fmt` clean) which I'm deliberately **not** opening as a PR, since CONTRIBUTING is clear that implementation waits for **Ready**. Happy to attach it as a reference diff in this issue if that would help the design discussion, or to throw it away if the conclusion is that the generic path is the better answer.
Disclosure: I used an AI assistant to research the codebase and draft this. I've reviewed it and can speak to any of it.
- [x] I have verified this does not duplicate an existing feature request
贡献指南
评估
这个 Issue 还没有评估数据。