CI never runs the buzz-db replica-routing Postgres fixtures — enforce them before enabling BUZZ_REPLICA_READ_MAX_AGE_MS
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Summary
None of the Postgres-backed replica-routing fixtures added in #3268 — nor most of the 34 `#[ignore]`d tests in `crates/buzz-db/src/lib.rs` — are ever executed by CI. They were verified exhaustively by hand at `dd26caa9f` (see the PR body), but a future regression on the routed-read path can merge green.
Found by Dawn during her #3268 sign-off amendment; independently confirmed by Max and Eva at the same SHA.
## Evidence (at `dd26caa9f`)
- `Unit Tests` runs `just test-unit` → `cargo nextest run -p buzz-db --lib` (`Justfile:279-285`). Ignored tests are excluded by design, and the Justfile comment says so.
- The only `--run-ignored ignored-only` nextest invocations in `.github/workflows/` are `ci.yml:690` and `:702`, both filtered to `(package(buzz-db) and test(/relay_invite::tests/)) or (package(buzz-relay) and test(/api::invites::tests/))`.
- Therefore, of the 34 `#[ignore]`d buzz-db tests, CI runs only the `relay_invite::tests` subset. The one-budget acquire regression (`routed_fallback_spends_one_acquire_budget_when_aurora_cache_is_cold`), the seven-seam community-isolation fixture, and all fence/floor-guard/fallback fixtures have **zero automated execution**.
## Why this matters
- The one-budget fixture exists precisely because a "fix is present" claim survived review without a measurement (#3268 thread, Dawn's Pass 11 postmortem). An `#[ignore]`d test no job selects is a comment, not a gate.
- Max's rollout condition: **do not set `BUZZ_REPLICA_READ_MAX_AGE_MS` on bb-block until this is fixed.** Merge itself is safe (new seams are dark by default), but enabling the new seams in production without CI enforcement of their regressions is not.
## Constraints on the fix
1. **Ordering dependency on #3619**: the moment any filter widening selects `test_usage_metrics_lock_has_single_owner_and_releases_on_drop`, the shared-DB advisory-lock collision fires in CI. #3619 (scratch-DB isolation for that test) must land first, or the selection must explicitly exclude it.
2. **A naive regex is insufficient.** Dawn's first-draft filter `test(/tests::(routed|replica|reader|fence|floor_guard)/)` misses at least: `query_events_routed_*`, `count_events_routed_*`, `lazy_reader_pool_*`, `head_fetch_routes_by_configured_budget`, `channel_window_routes_*`, `thread_replies_cursor_pages_*`, `read_session_degrades_*`. A partially-covering filter recreates this bug one level down — green CI, unenforced invariants.
3. Several fixtures assume scratch DBs and serial execution; the selection should run serially against the existing `backend-integration` Postgres archive (that job already has the infra).
## Suggested shape (per Max)
An **explicit, auditable selection** rather than a clever regex: either move the replica-routing fixtures into a dedicated test module (e.g. `tests::replica_routing::`) and filter on that module, or maintain an explicit nextest filter list in `ci.yml` with a comment stating it must cover every `#[ignore]`d routing fixture. Initially cover the exact routing matrix verified by hand in #3268. Keep `test_usage_metrics_lock...` excluded until #3619 lands.
## Acceptance
- The one-budget fixture and the seven-seam isolation fixture demonstrably run in CI (visible in the job log at least once).
- A revert of only the #3268 production fix fails CI (the 320ms defang check, now automated).
- No selection of the usage-metrics lock test until #3619 is closed.
Refs: #3268, #3619.
Contributor guide
Research direction
Start by reading .github/workflows/ci.yml:690-702, Justfile:279-285, and the ignored tests in crates/buzz-db/src/lib.rs. Run the existing backend-integration Postgres job or inspect its nextest selection, then verify that the one-budget and seven-seam fixtures appear in its log while test_usage_metrics_lock_has_single_owner_and_releases_on_drop remains excluded. Done means the listed routing matrix is explicitly covered and the acceptance checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, postgresql, rust
- Domain
- ci-cd, databases, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100