fix(signals): the enablement backfill fans the five newest issues into separate reports and PRs within minutes
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Problem
Enabling error tracking as a signal source runs BackfillErrorTrackingWorkflow (products/signals/backend/temporal/backfill_error_tracking.py), which emits the five most recently created issues from the last 30 days as issue_created signals in one go (BACKFILL_ISSUE_LIMIT = 5, BACKFILL_WINDOW_DAYS). Each signal then walks the normal path: grouping, research, and, when the report is immediately actionable and the team's autostart settings allow it, an implementation task and a PR.
On one customer project the backfill fired at 00:23, four separate reports existed by 00:24, two implementation tasks started at 00:30, and two PRs changing the same file for the same reason were open by 00:49. Twenty-six minutes after turning the product on, the customer's first impression was duplicate reports and duplicate PRs. The five backfilled issues were, by construction, whatever happened to be newest, which on a project with local dev servers reporting into production means dev noise.
Fixes
- Hold auto-start for backfilled signals. Tag backfill-originated signals (
extra={"backfill": true}inemit_backfill_signal_activity) and havemaybe_autostart_implementation_taskskip auto-start for reports whose signals are all backfill, with askip_reasonthe inbox can show. The reports still generate and notify; the customer presses Create PR if they want one. - Let the batch settle before research. Run grouping for the five backfilled signals as one batch (it may already be one batch; confirm) and delay research on them by a short window so the sibling overlap check and any later grouping matches can apply before PRs open.
- Prefer issues with production impact. Instead of the five newest issues, pick by occurrence count and distinct users in the window, or at least exclude issues whose events all come from local development hosts once #90178 makes that visible.
- Same posture for other backfills (health checks, replay clusters) if they emit on enable.
- Tests: backfill-tagged report does not auto-start; a report with one backfill and one live signal behaves as today; the issue selection query prefers impact over recency.
Why
The first hour after enablement is when the product is judged. A damper on auto-start and a smarter pick of seed issues costs nothing in coverage and removes the worst-case first impression.
From this Slack thread; the investigation is written up in the self-driving-debugging skill in the skills store.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with products/signals/backend/temporal/backfill_error_tracking.py and trace emit_backfill_signal_activity into maybe_autostart_implementation_task; confirm whether grouping already batches backfilled signals. Done means backfill-only reports do not auto-start, mixed reports retain current behavior, and selection favors production impact rather than recency, with matching tests added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100