block / block/buzz

Desktop goose ACP agents silent or flaky with custom OpenAI-compatible provider (hosted models)

Open
#4,233 7 comments 2 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

### Summary

We run Buzz Desktop agents on **goose ACP** against a **custom OpenAI-compatible provider** and **self-hosted models** (not stock cloud providers). In that configuration we hit a multi-layer failure mode that looks like “agent is online but never replies in channel,” even when goose completes the turn successfully.

This is an **edge case** relative to default Buzz/goose setups that use first-party providers and models that reliably emit tool calls. It is a real class for internal proxies and self-hosted OpenAI-compatible endpoints. We are filing before related PRs so the product gap is documented for maintainers.

### Environment (reproducible class)

| Piece | Our setup |
|-------|-----------|
| Host | Buzz Dev Desktop (`tauri dev` / managed agents) |
| Runtime | `goose acp` (**goose 1.44.0** measured) via `buzz-acp` harness |
| Provider | Custom OpenAI-compatible endpoint (example: `custom_actvite` → self-hosted OpenAI API) |
| Models | e.g. deepseek-class and qwen-coder-class on our infra |
| Agent access | Often `respond_to=owner-only` |
| Publish path | Channel visibility requires a successful `buzz messages send` (tool **or** harness content-delivery fallback) |

CLI `goose` against the same custom provider/models can work from an interactive login shell while Desktop-spawned ACP children fail or “succeed silently.”

### Symptoms

1. Agent harness is **up** (presence online, pool ready, subscribed to channel).
2. Owner `@mention` produces typing/activity (or nothing visible).
3. **No kind:9 channel reply** (thread or top-level).
4. Goose session may still show a full assistant turn (text / `write` tools) in session DB.
5. Harness logs may show either:
- `agent_returned` / JSON-RPC **-32603 Internal error** with goose data **`Provider not set`**
(verbatim class: `Error getting agent reply: Provider not set`), or
- turn completion **without** any publish tool and **without** harness content-delivery fallback (pre-S1 / clobbered binaries).

At default info logging, **successful** turns often leave **no** `created session` lines in `buzz_acp` logs — “log frozen after membership” is **not** a falsifier for “turn never ran.” Watch for new `-32603` or channel kind:9 instead.

### Expected

- Desktop-spawned goose ACP resolves custom provider secrets the same way CLI goose does (or fails loudly in UI).
- When a channel/DM mention turn ends with user-visible assistant text and **no** successful publish tool call, the harness still delivers **exactly one** channel/DM reply (model-agnostic).
- Managed-agent harness binary choice is **stable across Desktop restarts** (not clobbered by cargo `target/debug` rebuilds in dev).

### Actual

#### A. Custom provider secrets on Desktop spawn (goose + Desktop env)

Desktop-spawned `goose acp` children inherit a thin environment. With `GOOSE_PROVIDER=` set but without the host’s keyring/secrets path:

- **`GOOSE_DISABLE_KEYRING=1`** is the **required** switch so goose can load `~/.config/goose/secrets.yaml` when the system keyring is unavailable under GUI/Desktop spawn.
- The provider’s expected API key env (class: `CUSTOM_*_API_KEY`; our measured name: `CUSTOM_ACTVITE_API_KEY`, often mapped from a shell-only name like `GOOSE_ACTVITE_API_KEY` in login bashrc) is a useful **belt**, not always mandatory if secrets.yaml is complete **and** keyring is disabled.

Without that setup, goose ACP fails with **-32603 / Provider not set**. Interactive CLI works because the login shell already sets keyring-disable (and often key mapping).

**Measured falsifier (goose 1.44.0):** secrets.yaml only + **no** `GOOSE_DISABLE_KEYRING` → Provider not set class; `GOOSE_DISABLE_KEYRING=1` alone → `stopReason: end_turn` OK.

**Workaround we applied (ops, not product):** persist those env vars on the managed agent (and/or global agent config) and respawn the agent process.

#### B. Self-hosted models that complete without tool calls (Buzz publish contract)

Buzz channel UX only shows kind:9. Success path historically depended on the model invoking a publish tool (e.g. goose-namespaced `buzz_publish__buzz_messages_send` / shelling `buzz messages send`).

With a **self-hosted deepseek-class model** + **toolshim already enabled** (`GOOSE_TOOLSHIM=true`):

- Tool path **sometimes** works (MCP is fine when called).
- On short creative turns the model often returns **assistant text only** (`toolRequest` count = 0) despite VISIBILITY / “must publish” Context in the user message.
- A sibling model (qwen-coder-class) more often called the publish tool under the same contract.

Result: goose “succeeds,” channel stays silent. **Prompt-forced wording is not an acceptable product fix.**

**Product direction that worked for us (S1):** `buzz-acp` **content-delivery fallback** — on end of turn, if no successful publish tool fired, post undelivered assistant text once as a channel reply (log class: `content-delivery fallback: posted undelivered agent content as channel reply`). Model-agnostic; preserves PUBLISH ONCE / no double-post when the tool already succeeded.

#### C. Dev Desktop harness path clobber (aggravating)

With bare `acp_command: "buzz-acp"`, Desktop workspace resolution (`resolve_workspace_command` in `desktop/src-tauri/src/managed_agents/discovery.rs`) prefers cargo workspace `target/debug/buzz-acp` over `~/.local/bin`. Full Desktop / `tauri dev` restart **rebuilds/relinks** that path and can replace a known-good harness (with content-delivery fallback / VISIBILITY Context) with an older debug binary that lacks it. Agents look healthy; publish path regresses.

**Measured example:** pre-S1 SPAWN build had **0** `content-delivery fallback` strings; good user-installed S1 build had them. Live agents after full restart ran the pre-S1 binary until re-pinned.

**Workaround we applied:** pin affected agents’ `acp_command` to an **absolute** path outside cargo target (e.g. `$HOME/.local/bin/buzz-acp`) so the next cargo clobber cannot move those agents back onto pre-fallback binaries. Absolute paths skip workspace resolution when the path exists (`command_looks_like_path`).

**Scope note:** Layer C is primarily a **dev / tauri cargo workspace** clobber; packaged Desktop may pin differently.

### Minimal reproduction sketch (custom provider edge case)

1. Configure a Buzz managed agent: runtime goose ACP, **custom** OpenAI-compatible provider, self-hosted model (ideally one weak at tool calls, e.g. deepseek-class).
2. Do **not** set `GOOSE_DISABLE_KEYRING=1` (and omit provider API key env if secrets depend on keyring); start from Desktop (not login-shell goose).
3. Observe: turn error / `-32603` Provider not set **or** fix secrets and continue.
4. With secrets fixed but **without** harness content-delivery fallback: owner `@Agent write a short sentence` → session has assistant text → **zero** channel kind:9.
5. With content-delivery fallback binary: same ordinary wording → **exactly one** kind:9.

### What we believe should change in product

1. **Document** Desktop goose + custom provider requirements (`GOOSE_DISABLE_KEYRING`, provider key env names, secrets.yaml vs keyring under GUI).
2. **Harden Desktop spawn** so custom providers get the same secret resolution path as CLI (or fail loudly in UI with “provider not configured” instead of silent/emoji-only turns). Default-injecting `GOOSE_DISABLE_KEYRING=1` for goose-runtime agents on hosts without a working system keyring would cover the common edge case.
3. **Ship content-delivery fallback** in `buzz-acp` as the default for channel/DM mention turns so self-hosted / non-tool-calling models are not second-class for channel visibility.
4. **Spawn resolution (dev):** prefer absolute/user-installed harness, or stop resolving bare `buzz-acp` exclusively to workspace `target/debug` in a way that silently reverts product fixes after every full Desktop rebuild. Absolute `acp_command` should remain supported and recommended for local product binaries.

### Out of scope / not claiming

- This is **not** “goose cannot use custom providers at all” — CLI and correctly-env’d ACP both work.
- This is **not** fixed by swapping every agent to a different model; we intentionally split models.
- Origin timeouts / CF **524** on a single self-hosted model route are capacity issues, distinct from Provider-not-set and publish-fallback classes.
- Goose steer / hung-turn classes (deadline extensions with no channel egress) are related operational noise but not required to reproduce A/B/C.

### Acceptance criteria (for a fix PR)

- [ ] Custom provider agent started **only** from Desktop UI can complete a turn without manual shell keyring env (or UI blocks with a clear config error).
- [ ] Owner ordinary mention (no forced “you must publish” poetry) produces **exactly one** kind:9 when the model returns text and does not call publish tools.
- [ ] Same agent with tool-based publish still produces **exactly one** kind:9 (no double post).
- [ ] Full Desktop restart does not silently drop agents onto a harness build missing content-delivery fallback when an absolute pin or equivalent product default is set.

---

### Suggested labels (if available)

`bug`, `desktop`, `goose`, `acp`, `documentation`

Contributor guide

Open the contributing guide

Research direction

Start with desktop/src-tauri/src/managed_agents/discovery.rs and the buzz-acp spawn and publish entry points. Reproduce the Desktop-only custom-provider failure with goose 1.44.0, then trace provider environment handling, content-delivery fallback, and bare-command resolution. Done means clear provider errors, exactly one channel reply with or without a publish tool, and no silent regression to an older harness after restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, tauri
Domain
api, desktop, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.