block / block/buzz

Channel sections silently revert to an older state after a relay restart

Open
#6,302 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

# Channel sections silently revert to an older state after a relay restart

**Component:** desktop + relay · **Desktop:** 0.5.10 (Linux) · **Relay:** self-hosted,
`ghcr.io/block/buzz@sha256:bf73e1f7…` (`main`, image created 2026-08-07)

## Summary

A relay restart causes the desktop client to replace its local `buzz-channel-sections.v1`
state with an older server-side version, **silently discarding section changes the relay had
already acknowledged**. There is no conflict, no warning, and no merge — the newer local
state is simply overwritten.

I lost 3 sections and 18 channel assignments this way. I have before/after captures because
I happened to be polling localStorage at the time.

## Timeline (one incident, fully instrumented)

Sections were created and channels assigned interactively from ~13:00. Each change was
written locally and the sync watermark advanced within ~2s, i.e. **the relay acknowledged
every one**:

```
13:00:04 section "section-C" created → local write
13:00:06 buzz-sync-watermark.v1:channel-sections … = 13:00:04 (acked)
13:01:37 section "section-D" created
13:01:39 watermark = 13:01:37 (acked)
…continued to 5 sections / 28 assignments; final watermark 13:04:12
```

Roughly 13 hours later the relay container restarted (routine host backup stops and starts
containers). The client logged:

```
02:27:42 Buzz: WebSocket disconnected; retrying in 1.0s:
received 1012 (service restart) relay restarting; then sent 1012 (service restart)
02:27:43 server rejected WebSocket connection: HTTP 502 (×4, backing off 1→2→4→8s)
02:27:46 buzz-mesh: status report after heartbeat failed: relay returned 502 Bad Gateway
02:27:47 relay container started
02:28:12 sections 5 → 2, assignments 28 → 10
```

The surviving state is exactly the state from **before** that afternoon's work.

## Before / after

`buzz-channel-sections.v1::wss%3A%2F%2F` — `"version": 1` in both.

| | before 02:28:12 | after |
|---|---|---|
| `section-A` `f1175c91-…` | 6 channels | 6 channels |
| `section-B` `bea6391a-…` | 5 channels | 4 channels |
| `section-C` `4a2c18e0-…` | 6 channels | **gone** |
| `section-D` `e286c48d-…` | 6 channels | **gone** |
| `section-E` `47d59364-…` | 5 channels | **gone** |
| **totals** | **5 sections / 28 assignments** | **2 / 10** |

Full JSON of both states available on request.

## Expected

A relay restart should not lose writes it acknowledged. Failing that, the client should not
overwrite newer local state with an older server snapshot without merging or warning — the
sync watermark is already there and could gate it.

## Actual

Acknowledged section state is absent after the relay restarts, and the client adopts the
older server state, discarding local work permanently.

## Reproduction

1. Self-host a relay. In the desktop client create 2–3 channel sections and assign channels.
2. Confirm the sync watermark advances (`buzz-sync-watermark.v1:channel-sections:…`).
3. Restart the relay container.
4. On reconnect, observe sections revert to their pre-change state.

## Notes / possibly related

- Local persistence is **not** the problem: the sections were present in the client's
localStorage DB independently of the WAL, so nothing was lost client-side before the sync.
- The same identity holds two communities on two relays. Their section keys are separate,
but **their `assignments` maps contain the same channel IDs** — every one of the 10
assignments on relay B also appeared on relay A, while relay B only *owns* 6 of those
channels. Sections created against one community appear to leak into the other's section
state. Possibly a separate bug; mentioned in case the two share a root cause.
- Relay is tracking the rolling `main` tag rather than a pinned release.

## Environment

- Desktop 0.5.10, Linux (Arch/EndeavourOS, KDE Plasma 6.7 Wayland)
- Self-hosted relay in Docker on Unraid, reached over Tailscale and via a public hostname
- Two communities/relays on one identity

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.