Outcomes end at merge: add a post-merge window so "accepted" is joined to "held"
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 71
- Forks
- 64
- Avg merge
- 15h 38m
- Merged PRs (30d)
- 66
Description
What the collector measures today
services/api/src/watchtower/outcomes.ts joins each agent PR to fate, acceptance (human squash-merge with evidence), review rounds, and fixup commits. oneShot = merged with zero change requests and zero human commits after the first bot commit on the PR branch.
Everything stops at merged_at. There is no signal for what happened to the change afterwards.
What that misses
- A revert of the merge commit (the sdlc page reports "0 production reverts"; nothing here measures it).
- A human commit or PR touching the same paths within N days of merge. Fixups landed in a separate PR are invisible to
fixupCommits, so one-shot is overstated. - A platform issue / CI-doctor fingerprint whose first appearance follows the merge and touches the same files.
The ROADMAP's definition of done, item 8, says deployment and production signals are joined back to the PR and outcome. The outcome record has nowhere to hold them.
Proposal
Add a post-merge evaluation to the nightly outcomes job:
- New columns on
outcomes:held(bool, null until the window closes),post_merge_window_days(project setting, default 7),reverted_at,post_merge_human_commits(count of non-bot commits on the default branch touching the PR's changed paths inside the window),linked_incident_fingerprint. held = merged && !reverted && post_merge_human_commits == 0 && no linked incident.- Analytics: report
heldnext toacceptedandoneShotinanalytics.tsand the project page, withunassessedwhile the window is open. - Keep the evidence path independent: read the default branch and revert commits from the GitHub API, never from Facility's own events, per the watchtower's rule 1.
Why
Acceptance measures whether a reviewer said yes. Held measures whether they were right. The learning agent's "silence outranks everything" rule applies here: a change that quietly needed a follow-up fix is the failure class most worth clustering, and today it never turns anything red.
I can open a PR against outcomes.ts and analytics.ts with tests if the shape is agreed. Filing first per CONTRIBUTING.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the proposed changes in services/api/src/watchtower/outcomes.ts and analytics.ts, then read ROADMAP item 8 and CONTRIBUTING for the project’s constraints. Define the post-merge fields and window behavior, add tests as proposed, and verify that analytics and the project page report held or unassessed alongside accepted and oneShot.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, typescript
- Domain
- analytics, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100