githubnext / githubnext/agentics

repo-assist: Task 8 (Monthly Activity Summary) does not run deterministically despite being marked mandatory

Abierto
#372 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Makefile
Estrellas
952
Forks
139
Merge medio
6 d 10 h
PR fusionados (30 d)
2

Descripción

## Summary

In `repo-assist.md`, Task 8 (Update Monthly Activity Summary Issue) is documented as mandatory every run:

> "Always do Task 8 (Update Monthly Activity Summary Issue) every run."

In practice, on the `tenstorrent/tt-metal` deployment, it is **not** running deterministically. Over the last 7 daily scheduled runs (2026-07-24 through 2026-07-30, all completed successfully), the monthly summary issue ([#50956](https://github.com/tenstorrent/tt-metal/issues/50956)) was only actually updated on 2 of them.

## Evidence

| Date | Run | Task 8 landed? |
|---|---|---|
| 07-24 | [30072973467](https://github.com/tenstorrent/tt-metal/actions/runs/30072973467) | ❌ Called `update_issue`; safe-outputs manifest logged it as processed, but no edit ever appears on the issue's timeline. Silently dropped. |
| 07-25 | [30147764461](https://github.com/tenstorrent/tt-metal/actions/runs/30147764461) | ✅ `add_comment` landed |
| 07-26 | [30191579550](https://github.com/tenstorrent/tt-metal/actions/runs/30191579550) | ❌ Trace shows it read the issue and planned "Task 8: Update monthly activity summary," then never called it |
| 07-27 | [30244126524](https://github.com/tenstorrent/tt-metal/actions/runs/30244126524) | ✅ `add_comment` landed |
| 07-28 | [30335618590](https://github.com/tenstorrent/tt-metal/actions/runs/30335618590) | ❌ `state.json` memory records `"task8_monthly_summary"` as done; actual `add_comment` calls that run targeted other issues/PRs (51294, 51296, 51024) instead |
| 07-29 | [30429064534](https://github.com/tenstorrent/tt-metal/actions/runs/30429064534) | ❌ Same pattern — reads #50956, reasons through "Task 8," then spends its comment budget on stale-PR nudges |
| 07-30 | [30520369864](https://github.com/tenstorrent/tt-metal/actions/runs/30520369864) | ❌ Same again |

Issue #50956's `updated_at` has been stuck at `2026-07-27T07:02:56Z` since — three consecutive daily runs produced zero visible output on it, despite each run's own reasoning trace explicitly citing "Task 8" as a planned step.

## Root cause (best guess)

"Always do Task 8" is a prose instruction competing for attention/turn budget against the round-robin task selection in the same prompt, not a structurally-guaranteed step in the compiled workflow. Nothing fails or flags the run if the write doesn't happen. The agent's own repo-memory (`state.json`) isn't validated against ground truth either — it records `task8_monthly_summary: done` even on runs where no comment/edit actually reached the issue. On 07-24, the `update_issue` safe-output specifically appears to fail silently (logged as processed, but no visible effect and no error).

## Question

Is there a supported pattern for making a task like this deterministic — e.g.:
- Splitting it into its own workflow/job that always runs, decoupled from the round-robin agent turn
- A safe-outputs guarantee/verification step that fails the run (or retries) if a declared "always do" output type wasn't actually emitted
- Reconciling agent-reported `state.json` progress against actual GitHub state rather than trusting the agent's self-report

Happy to help test a fix if there's a preferred approach.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Start with repo-assist.md Task 8, then compare the seven linked workflow runs, their traces, safe-outputs manifests, and state.json against issue #50956's timeline. Determine where the mandatory update can be skipped or silently dropped; done means Task 8 runs deterministically and failed or missing output is detected or retried.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
github, github-actions
Área
ci-cd, devops
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Tranquilo
Claridad
Bastante claro
Aptitud para principiantes
52/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.