Slack migration parity: four gaps found completing a real workspace migration on a hosted relay
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Summary
We migrated a company's Slack workspace into a hosted Buzz community and finished it — ~1,700 messages and threaded replies across 15 channels, 233 file attachments, 559 reactions, every message signed by a distinct identity per original author, reconciled against the Slack Web API by message id in both directions.
So the question is not "can it be done." It can, entirely from client-side primitives, and the result is good. This issue is about the gap between "good" and **1:1**: the specific things a Slack migration cannot reproduce on a hosted relay today, what each one costs the operator, and what a supported import path would need to expose.
Filed as a field report rather than a design, because the design already exists and is sitting in this repo's queue. Related, and deliberately not duplicated:
- **#2704** — a full Slack→Buzz importer PR with a two-party attribution model and a narrow relay import exemption. Most of what is asked for below is in it.
- **#3306** — RFC for bulk historical backfill and an `imported_from` provenance convention. We shipped that tag shape in production; it works and it should be the convention.
- **#3307** — NIP-IM, the docs half of the same idea.
- **#4119** — live connectors for a gradual cutover. Complementary, different problem: that one is about *not* migrating yet.
Three independent parties have now arrived at the same four gaps from different directions. That seems worth a maintainer position, even if the position is "won't fix, here is why."
---
## What is already 1:1 today
Worth stating, because it is more than we expected going in, and none of it needed a fork:
| Dimension | Status |
|---|---|
| Messages and threaded replies | 1:1 — kind 9, replies `e`-tagged to their parent |
| Original author attribution | 1:1 *in appearance* — one nostr identity per Slack user, each signing its own messages |
| File bytes | 1:1 — original bytes into Blossom, attached as NIP-92 `imeta` |
| Reactions | 1:1 — real kind 7 events signed by whoever left them in Slack |
| Relative order | 1:1 — events stamped one second apart inside the drift window |
| Provenance | `["imported_from","slack","//"]` on every event (#3306's shape) |
| Retiring the import identities afterwards | NIP-IA (kinds 9035/9036) already does exactly this, and does it well |
NIP-IA deserves specific credit: archived pubkeys keep full attribution on their history but drop out of mention autocomplete, search, DM pickers and add-member pickers. It is the reason a client stopped objecting to "a pile of fake accounts in my workspace." It shipped before anyone tried to import into it, and it happens to be the right primitive.
---
## Gap 1 — Original timestamps are unreachable, and it is three layers deep
The one that makes the archive read wrong. Every imported message carries the import date; the original Slack date has to be rendered into the message *text* as a first line, which is a workaround a reader can see.
Backdating is blocked at three independent layers, and it is worth writing them down because relaxing only the first produces a `500` *after* an apparently successful accept:
1. **Ingest** — `MAX_TIMESTAMP_DRIFT_SECS = 900` (`crates/buzz-relay/src/handlers/ingest.rs:1976`), a hardcoded const with no config hook, applied to every event kind.
2. **Database** — `migrations/0021_created_at_fence_floor.sql`, a `DEFERRABLE INITIALLY DEFERRED` constraint trigger that re-evaluates `clock_timestamp()` inside COMMIT and aborts any `channel_id IS NOT NULL` row older than the floor.
3. **Push** — `EVENT_USEFUL_SECS` in `crates/buzz-relay/src/push_runtime.rs`, which drops old events from notification delivery. Cosmetic next to the other two, but it means an import is silent even if the first two are cleared.
**Layer 2 is not a knob and should not be treated as one.** Its own comment is explicit: the replica-fence proof requires that once a replica has replayed past a sampled writer LSN, no transaction can later commit a below-fence row, or keyset pagination goes silently incomplete. Enforcement is opt-in per session via the `buzz.created_at_floor` GUC and a no-op when unset — deliberately, so `pg_restore` and backfills keep working — and there is **no in-band bypass for channel-bearing rows** by design.
The consequence is the part worth surfacing: **historical import is architecturally an operator-plane operation, not a client one.** It has to run on a connection outside the relay's writer pool, with the operator holding the replica breaker closed from before the transaction opens until its WAL replays. No client, no CLI, and no amount of correct signing can do it from outside — which means on a hosted relay it is unavailable to the customer who is migrating, permanently, unless this repo provides a path.
#2704's approach — an authorized-import exemption gated on an owner/admin `import` tag — is the right shape. The thing we would flag for review: an ingest-level exemption alone does not clear layer 2, because the relay's writer pool arms the GUC on every connection. The exemption has to be designed at the writer-pool/breaker level or the accepted event dies at COMMIT.
## Gap 2 — Attribution has no answer for people who will never hold a key
#2704 requires both halves of a binding: an owner/admin attestation (kind 30623) *and* the subject's own self-claim (kind 30624), and history attributes only when `attestation.p == claim.signer`. As a security model for **current employees** that is right, and the anti-unilateral-takeover property is worth the friction.
But a real Slack archive is mostly written by people who cannot participate in that handshake:
- employees who left two years ago
- contractors and agency staff whose accounts are deactivated
- **Slack Connect users from other workspaces**, who appear as authors and are not members of anything you control
- `USLACKBOT`, which is not a person but carries tombstones for messages Slack no longer exposes
On our run, a meaningful share of authors fell into these categories. With a claim-only model their history renders unattributed or collapses onto the importer, which is exactly the "wall of messages from one bot" outcome the whole exercise exists to avoid.
What we actually did: derived one key per Slack user as `HMAC-SHA256(master_seed, slack_user_id)`, published a kind 0 per identity, marked each archived via NIP-IA, then destroyed the seed. It works and it is honest about what it is — but it means that during the migration **one operator-held seed can forge a message as any archived person**, and the only control on that is procedural. That is a worse security position than #2704's, and it is the position every operator is forced into by the absence of an alternative.
The ask: an **owner-attested archive-author** class that needs no self-claim, is structurally distinguishable from a claimed identity, and renders as such — "Jordan Reyes (archived, unclaimed)". Then a departed employee's history is attributed without anyone pretending it was consented to, and a returning employee can upgrade it to a real binding later via the #2704 path.
## Gap 3 — Relay membership has no API, so any importer depends on a human in a GUI
Every identity that signs anything needs **relay** membership, and the only way to create it is an invite code minted by the workspace owner in Buzz Desktop. `BUZZ_AUTH_TAG` is not a bearer pass — a fresh key plus a valid auth tag still gets `restricted: not a relay member`. Codes expire in roughly a week. There is no CLI or API to add a member, remove a member, or list them.
Practical consequences we hit, all of them mid-run:
- Onboarding N archive identities means N claims against one owner-issued code, and the owner has to be awake.
- A reactor discovered late — someone who left an emoji but never posted — stalls the run until the owner mints another code. (`/api/invites/claim` also answers `200 already_member` for keys that are **not** members, so that response cannot be used to detect it; you have to publish a kind 0 and read it back.)
- Retiring the identities at teardown is Desktop-only and owner-only, so the operator cannot complete their own cleanup.
- Two things sit in front of the claim endpoint that look like relay rejections and are not: Cloudflare 403s a `Python-urllib` User-Agent with body `error code: 1010`, and a workspace with a join policy answers `403 join_policy_required` (`GET /api/join-policy` → `POST /api/invites/accept-policy` → `policy_receipt`). Both cost a round trip to diagnose.
An import is a bulk-identity operation by nature. If #2704 or anything like it lands, it will need a programmatic way to provision and retire the identities it creates.
## Gap 4 — Membership and system events render as timeline rows, and an import generates them in bulk
This one produced the only defect our client complained about, and it is cheap to fix.
Buzz Desktop draws a timeline row per membership event. An import that touches channel membership therefore writes a visible block of rows into the middle of a freshly restored archive, and:
- **deleting them does not help** — a delete publishes a `message_deleted` tombstone that also renders, so the count never drops;
- **patching the client does not help** — "nobody should see these" is a data requirement, and a rendering change only fixes screens that install it.
We got out of it by discovering that `check_channel_membership` (`ingest.rs:632`) returns `Ok` if the pubkey is a member **or** `visibility == "open"` — so archive identities can publish into an open channel they have never joined, and the memberships never need to exist. That works, but it is a side effect being used as a feature, it does not extend to private channels without a create-open-publish-flip dance, and nothing documents it.
Two small things would close this properly: a channel-scoped way to suppress membership rows for events tagged as imported, and an explicit statement that publishing into an open channel does not require channel membership — so the next person does not find it by reading Rust at 2am.
---
## The ask
Not asking for our approach to be adopted — the tooling is ours to maintain. What would help everyone importing into Buzz:
1. **A maintainer position on #2704**, or on the import exemption in it specifically. It is the only proposal that addresses Gap 1, and if the answer is "not this shape", the shape you would take is the useful information.
2. **`imported_from` blessed as convention** (#3306 / #3307). It costs nothing, it is already GIN-indexed, it gives idempotency for free, and it is the only way to tell restored history from native history after the fact. We shipped it; it needs no code, only a decision.
3. **Guidance on Gap 2** — whether an unclaimed, owner-attested archive author is acceptable in the model, or whether operator-derived keys are considered the expected answer. Right now every operator invents this alone, holding a seed that can forge as anyone.
4. **A membership API** (Gap 3), even a minimal owner-authorized add/remove/list.
5. **A line in the docs for the open-channel publish rule** (Gap 4).
Happy to write any of this up as a PR, contribute the reconciliation tooling, or supply the reference dataset — a real workspace already migrated end to end, with both directions of the diff, is a decent test fixture for whatever lands.
One adjacent inconsistency, unrelated to parity but noticed along the way: `buzz-cli` enforces an upload **allow**-list of `jpeg,png,gif,webp,mp4` (`crates/buzz-cli/src/client.rs`) while Desktop enforces a **deny**-list of `html/xhtml/svg/javascript`/executables (`desktop/src-tauri/src/commands/media.rs`). The relay accepts everything Desktop does, so a PDF uploads from Desktop and is refused by the CLI. Happy to file separately if it is worth a ticket.
Contributor guide
Research direction
Start with the referenced proposal in #2704, then read crates/buzz-relay/src/handlers/ingest.rs, migrations/0021_created_at_fence_floor.sql, crates/buzz-relay/src/push_runtime.rs, and check_channel_membership at ingest.rs:632. Map each requested gap to an existing entry point or constraint before proposing scope. Done requires a maintainer decision on the import model, documented conventions, and clearly bounded API or behavior changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, rust
- Domain
- authentication, backend-api-design, databases, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100