aaif-goose / aaif-goose/goose

Support local model runners that distribute models as OCI artifacts

オープン
#11,692 コメント 10 件 リアクション 0 件 担当者 1 名 @ericcurtin が担当を希望しています GitHub で見る
主要言語
Rust
スター
54.2k
フォーク
6.2k
平均マージ
3日 4時間
マージ済み PR(30日)
240

説明

**What problem would this solve?**

Goose's built-in local options today are Ollama and, via the OpenAI-compatible path, servers like vLLM and LM Studio. A separate class of local runner has appeared that distributes model weights as **OCI artifacts** pulled from a normal container registry, rather than through a bespoke model library. [llmman](https://github.com/llmmanorg/llmman) is one; the packaging format is the [CNCF ModelPack spec](https://github.com/modelpack/model-spec).

Who this affects: people in environments where the container registry is already the governed distribution channel — mirrored, scanned, access-controlled, air-gapped. For them, "pull the model the same way you pull an image" is the difference between a model being usable and not. Right now they have to configure Goose's generic OpenAI-compatible provider by hand, which works but means the runner isn't discoverable in `goose configure`, and its default port and no-API-key behaviour aren't known to Goose.

The narrower version of the problem: Goose has no first-class local provider whose model identifiers are registry references.

**What would a good outcome look like?**

Someone running such a server locally can select it during `goose configure` and start a session without hand-editing config or knowing which generic provider to bend into shape. Its defaults — port, no API key required — are known to Goose, and model identifiers are whatever the runner resolves against its registry rather than a list Goose maintains.

Beyond that I'd rather not prescribe the shape. There's a real design question about whether this deserves its own provider or is better served by improving the generic OpenAI-compatible path, and I don't have a strong view on which is right for the project.

**Possible approaches**

Constraints and trade-offs worth weighing in the design discussion:

- These runners serve an OpenAI-compatible API, so no new wire format is needed. That makes the marginal implementation cost low — but also raises the fair question of whether a distinct provider earns its long-term maintenance cost versus documenting the generic path better.
- Model lists can't be static. Available models are whatever the user has pulled, so discovery has to come from the server's `/v1/models` or be left open.
- Some of these servers also speak the Anthropic Messages API, which may or may not be interesting to Goose.
- There is more than one such runner, so if this is worth doing it's probably worth designing for the category rather than one product.

**Additional context**

I have a working implementation on a local branch (a provider definition in the existing pattern; `cargo check` and `cargo fmt` clean) which I'm deliberately **not** opening as a PR, since CONTRIBUTING is clear that implementation waits for **Ready**. Happy to attach it as a reference diff in this issue if that would help the design discussion, or to throw it away if the conclusion is that the generic path is the better answer.

Disclosure: I used an AI assistant to research the codebase and draft this. I've reviewed it and can speak to any of it.

- [x] I have verified this does not duplicate an existing feature request

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Look at the existing provider definitions in the codebase, likely under a `providers` or `local` module. Examine how Ollama and the generic OpenAI-compatible provider are implemented. The design discussion centers on whether to create a new provider or extend the generic one. Start by understanding the configuration flow in `goose configure` and how model discovery works. 'Done' means a user can select this runner type during configuration without manual edits.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
docker, kubernetes, rust
領域
ai-infra-agents, backend-api-design, cli
issue の種類
機能追加
難易度
4/5
見積もり時間
3〜5日
活発さ
活発
明瞭さ
おおむね明確
初心者へのやさしさ
45/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。