stablyai / stablyai/orca

[Bug]: PR list check badge counts historical failed runs, not current check status

Open
#9,864 1 comment 1 reaction 1 assignee Claimed by @brennanb2025 View on GitHub
bug has_repro os:linux
Dominant language
TypeScript
Stars
71.3k
Forks
4.7k
Avg merge
14h 54m
Merged PRs (30d)
520

Description

### Operating system

Linux

### Orca version

v1.4.146

### Details

Short summary:
In the PR list, the checks badge shows "N failing" even when every check currently passes on GitHub (merge box green). N appears to equal the number of *previously failed* workflow runs on the head commit, not the number of currently failing checks.

What happened?

We run a label-gated required check (fails on PR open, passes after a label is added — same head SHA). After the check passes, GitHub's merge box is green and `gh pr checks` shows no failures, but Orca's PR list badge permanently shows "1 failing". A PR that went through the fail→pass cycle twice shows "2 failing". The badge only clears when a new commit changes the head SHA.

How can we reproduce it?
1. Add a workflow triggered on `pull_request: [opened, labeled]` whose job fails when a label is absent and passes when it is present.
2. Open a PR → the run fails.
3. Add the label → a new run on the same head SHA passes. GitHub now shows all checks green.
4. Orca's PR list still shows "1 failing". Remove and re-add the label to repeat the cycle and the badge becomes "2 failing".

What we think is going on:

Each event-triggered workflow run creates a new check suite on the same head commit, and `statusCheckRollup` keeps the check runs of *all* suites. Verified on an affected PR (`gh pr view --json statusCheckRollup`):

```
name conclusion completedAt
merge-label FAILURE 02:08 <- run triggered by "opened"
merge-label SUCCESS 02:21 <- run triggered by "labeled"
(all other checks SUCCESS/SKIPPED)
```

`deriveWorkItemCheckSummary()` in `src/main/github/client.ts` counts every rollup entry without deduplicating by check name, so the stale FAILURE entry keeps the summary in `failure` state forever. GitHub's own UI shows only the latest run per check, which is why the two disagree. The details-panel path (`mapGraphQLPRChecksResponse`) also keeps all check-run contexts, so the same stale entries can show up there.

Expected: the badge reflects the latest conclusion per check context (e.g. dedupe rollup entries by name, keeping the most recent), matching GitHub's merge box.

Happy to provide more repro details if useful.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.