block / block/buzz

Buzz Desktop managed-agent publisher replays stale signed kind:30177 events indefinitely

Open
#4,967 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

We run a self-hosted Buzz relay (`ghcr.io/block/buzz:main`) with one Buzz Desktop client (0.5.4 per the client's binary version resource and uninstall registry; Windows). The desktop's managed-agent re-assert loop republishes the signed kind:30177 event persisted at instance creation instead of re-signing, so once that artifact ages past the relay's ±900s ingest window, every re-assert is rejected — permanently, every 30 seconds, with no recovery path. Measured on our relay: 15,462 rejected writes across a 47-hour log window (a floor — the storm predates the window), about a third of the client's bridge traffic, and no agent-definition update can reach the relay again.

Full diagnosis below: mechanism with timestamps, a reproduction that needs only an agent and fifteen minutes of waiting, and a suggested fix — re-sign at publish, and invalidate the retained artifact on rejection. Cross-verified independently from the relay side and the client host.

---

# Buzz Desktop: managed-agent publisher replays a stored signed event instead of re-signing

**Status:** diagnosis complete, cross-verified from both sides (relay + client host).
**Written:** 2026-08-05. Investigators: Calliope (relay side, Delphi), Athena (client side, Aurora).
**Intended use:** upstream bug report to the Buzz maintainers.

---

## Summary

Buzz Desktop's managed-agent publisher re-asserts each agent definition by
**republishing the signed kind:30177 event it stored on first publish**, rather
than re-signing at publish time. Because a Nostr event id commits to
`created_at`, the artifact's timestamp never moves. Once the stored event ages
past the relay's ingest window (±900s), every subsequent re-assert is rejected —
permanently, on a 30-second cycle, with no path to recovery short of deleting
the artifact.

A rejection does not trigger a re-sign, so the loop is closed: the definition
can never be updated on the relay again, and the client never learns.

## Impact, measured

Measured on one relay (Delphi, `ghcr.io/block/buzz:main`) against one desktop:

| Metric | Value |
|---|---|
| Rejected writes in retained log window (2026-08-03T21:32Z → 2026-08-05T20:56Z, 47.4h) | **15,462** as of 2026-08-05T20:56Z — still climbing at the rate below |
| Rate while the client is running | **360/hour** — exactly 3 events per 30s cycle |
| Average across the window | 326/hour — the shortfall is client downtime, not a varying rate (see below) |
| Share of all HTTP bridge traffic in a sample hour | 360 of 1,125 requests (**32%**) |
| Stored kind:30177 rows, all time | 6 — three from 2026-07-30, three from 2026-08-02. **None since.** |
| Storm began | ~2026-08-03T00:15Z — 15 minutes after signing, when the artifact aged past the window |

Every rejection is a full request: TLS, auth, signature verification, and a
`WARN` log line, discarded at the timestamp gate
(`MAX_TIMESTAMP_DRIFT_SECS = 900`, `crates/buzz-relay/src/handlers/ingest.rs`).
The relay has accepted no managed-agent definition from this client since
2026-08-02T23:59Z.

The retained log window opens after the storm began, so 15,462 is a floor, not
a total.

**The cadence never adapts.** Scanning the whole retained log for interruptions
in the rejection stream — threshold set at 45s, i.e. any single missed 30s cycle
— finds ten, of which two are substantial and both fall on 2026-08-04:

```
07:29:02Z → 11:43:01Z 4.23 h client host rebooted (Windows Update); the
desktop does not auto-start
11:43:01Z → 11:48:55Z 5.9 min app restart cycle immediately after
```

The other eight are isolated one-or-two-cycle skips of 47–74s, scattered across
the three days, each followed immediately by a return to the exact 30s cadence.
Three of them cluster in the 12:07–12:27Z hour on 08-04, the tail of that
morning's restart sequence.

**The distinction that matters for diagnosis: none of these is backoff.** A
client reacting to rejection would show growing intervals; these are flat ~60s
one-offs against an otherwise metronomic 30s, with no trend in either
direction. The 4.23h reboot accounts for the difference between the 360/hour
running rate and the 326/hour window average. Nothing in the client ever slows
down, backs off, or gives up: ~15,000 identical rejections have produced no
change in its behaviour at all.

Client build version is not recorded here because the relay does not log a
client version or user-agent on this path — anyone citing one should source it
from the desktop host.

## Reproduction

1. Create a managed agent in Buzz Desktop; observe the kind:30177 event publish
and be accepted.
2. Wait longer than the relay's ingest window (15 minutes).
3. Observe every subsequent 30-second re-assert rejected with
`invalid: event timestamp too far from server time`, indefinitely.

No further action is required to reproduce — step 2 is just waiting.

## Mechanism, established

1. **Definitions are signed once, at instance creation.** On 2026-08-02 the
three agents' store entries were written at `23:59:48.663`, `.775`, `.880`,
and the resulting events carry `created_at` of `1785715188`, `1785715189`,
`1785715189` — that is, `:48`, `:49`, `:49`.

All three entry stamps fall inside second `:48`, so the event stamp is not
derived from the entry field: truncation would give `:48,:48,:48` and
rounding `:49,:49,:49`. Neither reproduces the observed split. What does fit
is signing with `now()` shortly after each entry write, with the run crossing
the second boundary between the first and second agent — consistent with any
write-to-sign delay between roughly 0.23s and 0.34s. **The delay is inferred
from that constraint, not measured**; the load-bearing conclusion is only
that the stamp comes from signing time, not from a stored field.

2. **The signed event is persisted verbatim.**
`%APPDATA%\xyz.block.buzz.app\agents\retention\.db` — SQLite, mtime
frozen at `2026-08-03T00:00:00Z` (~11s after signing, i.e. after the relay
accepted), untouched since. It holds complete signed events — `id`, `sig`,
`created_at`, `tags`, `content` — plus the serialized projection string used
as the change comparator.

3. **The periodic re-assert replays that artifact unconditionally.** Publish
success *was* recorded on 08-02, and the projection has not changed, yet the
client still re-asserts every 30 seconds — so this is not merely
"suppression keyed to a success that never arrives." It replays regardless,
and a rejection never marks the artifact stale.

4. **Nothing else can move the timestamp.** App restarts do not (three on
08-04). Local store edits do not — they touch `updated_at`, not the stored
event. A UI edit would not either: it changes content, but the publisher
still replays the retained artifact.

## Cross-verification

The two sides were checked independently and agree exactly:

- Relay-side: the three stored rows' NIP-01 ids recompute from their own fields
(`sha256([0,pubkey,created_at,kind,tags,content])`) — all three match.
- Client-side: today's live store values, run through the documented
`agent_event_content()` projection, reproduce the stored content strings
byte-for-byte and therefore the same three ids.
- The retention DB's stored event for Fizz matches the relay's stored row field
for field, including the omitted `respond_to_allowlist` key on an empty vec.

So the client is emitting a byte-identical event with a byte-identical id every
cycle, and has been for three days.

## Note on the payload projection

`ManagedAgentEventContent` declares nine fields, but
`agent_event_content()` (`desktop/src-tauri/src/managed_agents/agent_events.rs`)
nulls `system_prompt`, `model`, `provider` and `persona_source_version` whenever
`persona_id.is_some()` — the documented "slimmed projection" — and all four are
`skip_serializing_if = "Option::is_none"`. Definition-linked instances
therefore emit exactly five fields. This is recorded because a struct-shape
reading of the binary suggests nine and leads to the wrong conclusion.

## Suggested fix

**Re-sign at publish time.** The re-assert path should build and sign a fresh
event with `now()` rather than replaying the retained artifact; the retention
row should serve as the change comparator (its evident purpose) and not as the
thing that gets sent.

Secondary, defence in depth: a rejected publish should invalidate the retained
artifact so the next cycle re-signs, rather than replaying it indefinitely.

## Local workaround (untested, destructive-ish)

Full app quit including tray → rename the retention DB → relaunch, expecting the
publisher to find no artifact and sign fresh. Reversible by renaming back.
Rebuild behaviour is unproven. Not run: the fix belongs upstream, and this
touches app data.

**Do not** attempt to fix it by editing `managed-agents.json` `created_at` — the
publisher does not read that field for the event stamp.

## Relay-side mitigation (optional, does not fix the client)

At the ingest timestamp gate (`crates/buzz-relay/src/handlers/ingest.rs`), a
verbatim re-send of an event the relay already stores could be answered
`duplicate:` instead of `invalid:`. Verified applicable: the ids are stable, so
the lookup would match.

Two caveats: put the lookup **inside the rejection branch**, not before the
gate, so normal ingest pays no extra database round-trip; and note that this
silences the only signal that surfaces the client defect. It should not be
treated as closing this report.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.