huggingface / huggingface/transformers-ci

triage: re-running the daily triage clobbers the prior run's Outcome recap in the tracking issue

Open
#43 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
7
Avg merge
1h 30m
Merged PRs (30d)
39

Description

## Problem

`integration_failure_triage.py` names its tracking issue per **day** (`[serge] integration failure triage - YYYY-MM-DD`) and **refreshes that issue's body in place** as tasks reconcile. Re-running the triage the same day reuses the same issue and regenerates the body from the *new* run's state — where the groups have just been re-dispatched and have no outcome yet — so the previous run's completed **`## Outcome recap`** section (the per-group `no_fix`/`error` reason + token spend) is **overwritten and lost from the body**.

The recap is the only place the "why did this group open no PR" explanation is surfaced outside the Serge dashboard, so losing it mid-day is confusing and destroys useful analysis.

## Evidence

huggingface/transformers#47563 (2026-07-26) edit history (`userContentEdits`):

| time | editor | body size | note |
|---|---|---|---|
| 19:26 | github-actions | 3497 | created |
| 19:50 | github-actions | **5016** | run 1 reconcile adds `## Outcome recap` (owlvit/florence2 reasons + tokens) |
| 20:24 | github-actions | 3666 | run 2 (re-trigger) regenerates body → **recap gone** |

The recap content was only recoverable via the GitHub edit-history API; the body itself no longer had it.

## Impact

- Analysis (per-group no-PR reasons + token spend) silently disappears when the triage is re-run the same day.
- Only transiently "recovers" — the new run rewrites the recap when *its* tasks finish, but with different content (different run), so the prior run's record is permanently gone from the issue body.

## Root cause

Per-day issue reuse + in-place body refresh in `src/transformersci/agentic/integration_failure_triage.py` (the recap builder — "Recap lines for groups that produced NO PR", ~L1006 — and the tracking-issue body/reconcile path). The recap is derived only from the *current* run's finished tasks and rendered into the mutable body.

## Possible fixes (not prescriptive)

1. Write the `## Outcome recap` as a **comment** (durable; body-refresh doesn't touch comments) instead of in the mutable body.
2. **Don't reuse** the same-day issue on a re-run — mint a fresh per-run issue and mark the prior "superseded" (append, don't overwrite).
3. **Preserve/append** prior recaps in the body rather than regenerating from scratch.

Surfaced while validating the serge GPU verify loop (the owlvit recap that was lost is exactly the truncated-traceback case fixed by #42).

🤖 Filed via [Claude Code](https://claude.com/claude-code)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in src/transformersci/agentic/integration_failure_triage.py, reading the recap builder near ~L1006 and the tracking-issue body/reconcile path. Reproduce the same-day rerun behavior and trace how the prior Outcome recap is replaced; done means an earlier run's recap remains durably available after a later rerun.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.