Automattic / Automattic/icfy

Some branch pushes never appear in the push log

Open
#12 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
JavaScript
Stars
12
Forks
4
PR merge metrics
No merged PRs in 30d

Description

Some Calypso PR branches end up in a weird state:
- the branch has a valid PR and CircleCI checks were triggered for it
- the CircleCI task `icfy-stats` finished successfully and sent a hook request to `api.iscalypsofastyet.com`
- the `circle_builds` table has a record for the given SHA and build number -- the build is known to ICFY (can be verified on `icfy.com/buildlog` frontend URL and also directly in the MySQL table)
- the `pushes` table, however, doesn't have a record for the push SHA. ICFY is not aware of the GitHub push.

**How ICFY learns about new pushes:**
The process happens inside the [queuer microservice](https://github.com/Automattic/icfy/blob/master/server/queuer.js).

It **doesn't** use GitHub webhooks yet. Instead, we're periodically polling the [`/events`](https://github.com/Automattic/icfy/blob/master/server/github.js#L5-L6) endpoint that provides a paginated list of recent repo events. There we search for the interesting ones (pushes with new commits) until we reach the "last known push". If the result of this process are any new pushes, we queue them.

There seems to be a bug in this process. Are we saving an incorrect "last known push"? Are we ignoring some important pushes, like some with no commits? Should we switch from `PushEvent`s to some other events, e.g., associated with pull requests? These are all open questions.

Cc: @blowery and @flootr who might be interested in a nontrivial opportunity to dig into ICFY.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in server/queuer.js and server/github.js around the /events polling code. Trace how pagination, PushEvent filtering, and the last-known push are handled, then compare the pushes and circle_builds records for an affected SHA. Done means identifying why qualifying pushes are skipped and verifying that the polling process records them reliably.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, mysql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.