repo-assist: Task 8 (Monthly Activity Summary) does not run deterministically despite being marked mandatory

Aperta
#372 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
github, github-actions
Ambito
ci-cd, devops

Direzione di ricerca

Inizia con repo-assist.md Task 8, quindi confronta le sette esecuzioni del workflow collegate, le relative tracce, i manifesti di safe-outputs e state.json con la cronologia dell’issue #50956. Determina dove l’aggiornamento obbligatorio può essere saltato o eliminato silenziosamente; completato significa che Task 8 viene eseguito in modo deterministico e che un output non riuscito o mancante viene rilevato o ritentato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

In repo-assist.md, Task 8 (Update Monthly Activity Summary Issue) is documented as mandatory every run:

"Always do Task 8 (Update Monthly Activity Summary Issue) every run."

In practice, on the tenstorrent/tt-metal deployment, it is not running deterministically. Over the last 7 daily scheduled runs (2026-07-24 through 2026-07-30, all completed successfully), the monthly summary issue (#50956) was only actually updated on 2 of them.

Evidence

Date Run Task 8 landed?
07-24 30072973467 ❌ Called update_issue; safe-outputs manifest logged it as processed, but no edit ever appears on the issue's timeline. Silently dropped.
07-25 30147764461 add_comment landed
07-26 30191579550 ❌ Trace shows it read the issue and planned "Task 8: Update monthly activity summary," then never called it
07-27 30244126524 add_comment landed
07-28 30335618590 state.json memory records "task8_monthly_summary" as done; actual add_comment calls that run targeted other issues/PRs (51294, 51296, 51024) instead
07-29 30429064534 ❌ Same pattern — reads #50956, reasons through "Task 8," then spends its comment budget on stale-PR nudges
07-30 30520369864 ❌ Same again

Issue #50956's updated_at has been stuck at 2026-07-27T07:02:56Z since — three consecutive daily runs produced zero visible output on it, despite each run's own reasoning trace explicitly citing "Task 8" as a planned step.

Root cause (best guess)

"Always do Task 8" is a prose instruction competing for attention/turn budget against the round-robin task selection in the same prompt, not a structurally-guaranteed step in the compiled workflow. Nothing fails or flags the run if the write doesn't happen. The agent's own repo-memory (state.json) isn't validated against ground truth either — it records task8_monthly_summary: done even on runs where no comment/edit actually reached the issue. On 07-24, the update_issue safe-output specifically appears to fail silently (logged as processed, but no visible effect and no error).

Question

Is there a supported pattern for making a task like this deterministic — e.g.:

  • Splitting it into its own workflow/job that always runs, decoupled from the round-robin agent turn
  • A safe-outputs guarantee/verification step that fails the run (or retries) if a declared "always do" output type wasn't actually emitted
  • Reconciling agent-reported state.json progress against actual GitHub state rather than trusting the agent's self-report

Happy to help test a fix if there's a preferred approach.

Lingua principale
Makefile
Stelle
959
Fork
141
Merge medio
6g 10h
PR unite (30g)
2

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di githubnext/agentics

Tutte le issue di githubnext/agentics

Issue simili

Altre issue su DevOps

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.