huggingface / huggingface/serge
Let a task push its outcome to a GitHub issue (nightly triage table goes stale the moment the CI job exits)
- Dominant language
- Python
- Stars
- 51
- Forks
- 10
- Avg merge
- 11h 32m
- Merged PRs (30d)
- 36
Description
## Problem
The nightly integration-failure triage (`huggingface/transformers-ci`) opens one tracking issue per
run — e.g. [transformers#48050](https://github.com/huggingface/transformers/issues/48050) — with a
row per dispatched group that is supposed to show the outcome. Serge cannot update it, so the table
is only ever as fresh as the dispatching GitHub Actions job:
- the job renders the table at creation (all `(pending)`), then polls `GET /tasks/…/status` in a
reconcile loop capped at `ITF_RECONCILE_TIMEOUT` (1200s) and exits;
- once the Action exits nothing writes to the issue again, so a task finishing after that window is
invisible until the next nightly run re-renders the table;
- Serge has no idea the issue exists. The triage payload passes `tracking_issue` only *inside the
prompt* ("include `Relates to #N` in the PR body"); there is no `tracking_issue` field anywhere in
`reviewbot/`. Serge's only outbound push on completion is Slack (`slack_notify_pr_created` /
`slack_notify_task_finished`), and `/status` is auth-only, so nothing outside a token-holding CI job
can even read the outcome.
Result: a `no_fix` or `error` that Serge knows about at 11:30 shows up on the issue tomorrow.
## Proposal
Add a GitHub-issue completion notification, symmetric to the Slack one:
```yaml
notifications:
github_issue:
repo: huggingface/transformers # defaults to the task's repo
number: 48050
marker: itf-9b5330bbfa6e # the caller's row key (group fingerprint)
```
On terminal state, hook it in next to `_notify_task_finished` (`reviewbot/webapp.py:1763`) and post
the outcome — status, one-line reason, PR link when there is one, `job_id`, and the normalizer error
when that is what stopped it. `GitHubClient.post_issue_comment`
(`reviewbot/github_client.py:269`) already exists and the endpoint is the same for issues, so a
comment path needs no new client code; if we want the *table row* itself updated in place, that needs
a small `PATCH /issues/{n}` body helper plus the `marker` above to find and rewrite exactly one row
idempotently (re-delivery must not duplicate).
Notes / constraints:
- Same failure-tolerance as the Slack path: best-effort, never fail the task on a notification error.
- The App token must carry `issues: write` for the target repo (the triage workflow already requests
it for itself).
- The target repo/issue must be validated against the task's own repo — do not let an arbitrary
caller make Serge write to any issue it has a token for.
- Keep it opt-in per task, off by default, so only the triage caller uses it.
## Why not solve it on the caller side
The CI job cannot outlive its runner, and the `fingerprint → job_id` map exists only in that job's
memory, so a periodic reconciler workflow would first need that state persisted (e.g. in a hidden
comment in the issue body). A push from Serge is the only path that covers tasks finishing after the
job exits. Tracked from the caller side in
huggingface/transformers-ci#79 and huggingface/transformers-ci#80, which fix the two staleness bugs
that are the caller's own fault.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in reviewbot/webapp.py around _notify_task_finished at line 1763 and inspect the existing Slack completion path, then read GitHubClient.post_issue_comment in reviewbot/github_client.py around line 269. Trace how task notification settings are represented and determine whether completion should add an idempotent issue comment or update a marked table row. Done means an opt-in terminal notification reports the outcome safely, validates the target repository, and cannot make notification failures fail the task.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100