block / block/buzz

Proposal: capability-driven Agent avatar generation with Codex as a first adapter

Open
#3,089 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

## Motivation

The Add Agent avatar picker currently supports uploading or dropping an image, pasting a URL, and choosing an emoji. It does not offer a way to generate a purpose-built avatar, so users who want something more distinctive need to leave Buzz and bring an asset back manually.

I would like to explore whether Buzz can add an optional **Generate** source without making the core product depend on a new image-generation vendor, API-key setting, or externally hosted avatar URL.

I searched the current issues and PRs and did not find an existing proposal for Agent avatar generation. The closest related work is the merged Emoji picker (#1576), avatar upload lifecycle hardening (#2277), and the open external-avatar import work (#2476).

## Directional proposal

Model avatar generation as an optional Agent Harness capability rather than a Codex-specific frontend feature:

- The Rust runtime catalog remains the source of truth for whether generation is available and why it may be unavailable.
- Frontend code asks for the projected capability; it does not check `runtime.id === "codex"`.
- Codex would be the first adapter evaluated because Buzz already integrates it through `codex-acp` and Codex can expose built-in image generation.
- Since the target Agent does not exist yet, the Add Agent flow would use a disposable ACP session for asset generation rather than creating a hidden managed Agent or relay identity.
- The disposable session would receive only the user's image prompt and avatar composition constraints—no Agent private key, relay credentials, persona environment, or system prompt.
- Buzz would consume standard ACP image content, preview it locally, and upload the selected bytes through the existing media path.
- Only the resulting current-relay media URL would be saved as the Agent avatar; provider URLs, inline data, and temporary paths would remain non-durable.
- Unsupported, logged-out, policy-disabled, or indeterminate states would fail closed and leave the existing Image and Emoji flows unchanged.

For an initial experiment, I would avoid Pollinations, GIPHY, Memegen, direct OpenAI Image API calls, or any new provider credential UI. I would also avoid a silent `OPENAI_API_KEY` fallback: the Generate source should be available only if Buzz can prove that built-in Codex image generation is exposed under the user's existing Codex authentication and workspace policy.

## Why Codex looks technically plausible

Buzz currently launches Codex through `@agentclientprotocol/codex-acp`. The adapter has already merged terminal image-generation completion with ACP image content:

- https://github.com/agentclientprotocol/codex-acp/pull/262

An additional adapter PR proposes exposing the generated file path as a formal ACP location, but the proposed Buzz seam would use inline ACP image content and should not depend on that unreleased change:

- https://github.com/agentclientprotocol/codex-acp/pull/331

OpenAI documents built-in Codex image generation through `$imagegen`, while noting that availability and usage depend on plan and workspace settings:

- https://learn.chatgpt.com/docs/image-generation.md

These are feasibility signals, not proof that the complete Buzz Add Agent flow works today. A local spike would still need to verify authentication mode, tool exposure, cancellation, result extraction, temporary-file behavior, relay upload, and exact compatible versions.

## Questions for maintainers

1. Is capability-driven Agent avatar generation directionally in scope for Buzz?
2. Is Codex an acceptable first adapter if the UI and runtime contract remain provider-agnostic?
3. Would you prefer generation inside Add Agent, or only as a post-create Edit Avatar action to avoid an ephemeral pre-create harness session?
4. Should the first version require built-in ChatGPT-authenticated Codex generation and explicitly reject API-key fallback?
5. Would a non-default preview gate be the appropriate boundary for an initial implementation?
6. Are there ongoing avatar or harness changes beyond #2476 that this proposal should wait for or build on?

## Contribution plan

I have not started a production implementation and am not assuming this belongs on the project roadmap. If the direction makes sense, I am willing to run a local spike first and report exact-version evidence before opening a PR.

If the spike succeeds and maintainers want the feature, I would propose one focused, end-to-end Draft PR behind a preview gate: runtime capability and preflight, disposable Codex ACP generation, the Generate picker state, relay media materialization, failure handling, and mock E2E coverage. I would not submit separate Buzz PRs containing unused capability plumbing or a non-functional UI. If a released `codex-acp` change is required, I would treat that as an upstream prerequisite in its own repository before opening the Buzz PR.

## Alternatives considered

- **Direct third-party generation provider:** simpler to call, but adds vendor selection, credentials, billing, privacy, moderation, and availability policy to Buzz core.
- **Ask the newly created Agent to generate its own avatar:** avoids a pre-create session but makes the creation flow multi-step and mutates identity after deployment.
- **Direct OpenAI Image API integration:** technically straightforward but introduces explicit API-key storage and separately billed usage, which is a different product decision from using an existing Agent Harness capability.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the Rust runtime catalog, the Add Agent flow, the existing media path, and Buzz's @agentclientprotocol/codex-acp integration. Run the proposed local Codex spike to verify authentication, capability exposure, cancellation, image extraction, temporary-file behavior, and relay upload. Done means an exact-version feasibility report and maintainer agreement on scope, placement, authentication, and preview gating.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, backend-api-design, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.