Some branch pushes never appear in the push log
- Dominant language
- JavaScript
- Stars
- 12
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
Some Calypso PR branches end up in a weird state:
- the branch has a valid PR and CircleCI checks were triggered for it
- the CircleCI task `icfy-stats` finished successfully and sent a hook request to `api.iscalypsofastyet.com`
- the `circle_builds` table has a record for the given SHA and build number -- the build is known to ICFY (can be verified on `icfy.com/buildlog` frontend URL and also directly in the MySQL table)
- the `pushes` table, however, doesn't have a record for the push SHA. ICFY is not aware of the GitHub push.
**How ICFY learns about new pushes:**
The process happens inside the [queuer microservice](https://github.com/Automattic/icfy/blob/master/server/queuer.js).
It **doesn't** use GitHub webhooks yet. Instead, we're periodically polling the [`/events`](https://github.com/Automattic/icfy/blob/master/server/github.js#L5-L6) endpoint that provides a paginated list of recent repo events. There we search for the interesting ones (pushes with new commits) until we reach the "last known push". If the result of this process are any new pushes, we queue them.
There seems to be a bug in this process. Are we saving an incorrect "last known push"? Are we ignoring some important pushes, like some with no commits? Should we switch from `PushEvent`s to some other events, e.g., associated with pull requests? These are all open questions.
Cc: @blowery and @flootr who might be interested in a nontrivial opportunity to dig into ICFY.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in server/queuer.js and server/github.js around the /events polling code. Trace how pagination, PushEvent filtering, and the last-known push are handled, then compare the pushes and circle_builds records for an affected SHA. Done means identifying why qualifying pushes are skipped and verifying that the polling process records them reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, mysql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100