githubnext / githubnext/agentics
repo-assist: Task 8 (Monthly Activity Summary) does not run deterministically despite being marked mandatory
- Dominant language
- Makefile
- Stars
- 952
- Forks
- 139
- Avg merge
- 6d 10h
- Merged PRs (30d)
- 2
Description
## Summary
In `repo-assist.md`, Task 8 (Update Monthly Activity Summary Issue) is documented as mandatory every run:
> "Always do Task 8 (Update Monthly Activity Summary Issue) every run."
In practice, on the `tenstorrent/tt-metal` deployment, it is **not** running deterministically. Over the last 7 daily scheduled runs (2026-07-24 through 2026-07-30, all completed successfully), the monthly summary issue ([#50956](https://github.com/tenstorrent/tt-metal/issues/50956)) was only actually updated on 2 of them.
## Evidence
| Date | Run | Task 8 landed? |
|---|---|---|
| 07-24 | [30072973467](https://github.com/tenstorrent/tt-metal/actions/runs/30072973467) | ❌ Called `update_issue`; safe-outputs manifest logged it as processed, but no edit ever appears on the issue's timeline. Silently dropped. |
| 07-25 | [30147764461](https://github.com/tenstorrent/tt-metal/actions/runs/30147764461) | ✅ `add_comment` landed |
| 07-26 | [30191579550](https://github.com/tenstorrent/tt-metal/actions/runs/30191579550) | ❌ Trace shows it read the issue and planned "Task 8: Update monthly activity summary," then never called it |
| 07-27 | [30244126524](https://github.com/tenstorrent/tt-metal/actions/runs/30244126524) | ✅ `add_comment` landed |
| 07-28 | [30335618590](https://github.com/tenstorrent/tt-metal/actions/runs/30335618590) | ❌ `state.json` memory records `"task8_monthly_summary"` as done; actual `add_comment` calls that run targeted other issues/PRs (51294, 51296, 51024) instead |
| 07-29 | [30429064534](https://github.com/tenstorrent/tt-metal/actions/runs/30429064534) | ❌ Same pattern — reads #50956, reasons through "Task 8," then spends its comment budget on stale-PR nudges |
| 07-30 | [30520369864](https://github.com/tenstorrent/tt-metal/actions/runs/30520369864) | ❌ Same again |
Issue #50956's `updated_at` has been stuck at `2026-07-27T07:02:56Z` since — three consecutive daily runs produced zero visible output on it, despite each run's own reasoning trace explicitly citing "Task 8" as a planned step.
## Root cause (best guess)
"Always do Task 8" is a prose instruction competing for attention/turn budget against the round-robin task selection in the same prompt, not a structurally-guaranteed step in the compiled workflow. Nothing fails or flags the run if the write doesn't happen. The agent's own repo-memory (`state.json`) isn't validated against ground truth either — it records `task8_monthly_summary: done` even on runs where no comment/edit actually reached the issue. On 07-24, the `update_issue` safe-output specifically appears to fail silently (logged as processed, but no visible effect and no error).
## Question
Is there a supported pattern for making a task like this deterministic — e.g.:
- Splitting it into its own workflow/job that always runs, decoupled from the round-robin agent turn
- A safe-outputs guarantee/verification step that fails the run (or retries) if a declared "always do" output type wasn't actually emitted
- Reconciling agent-reported `state.json` progress against actual GitHub state rather than trusting the agent's self-report
Happy to help test a fix if there's a preferred approach.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with repo-assist.md Task 8, then compare the seven linked workflow runs, their traces, safe-outputs manifests, and state.json against issue #50956's timeline. Determine where the mandatory update can be skipped or silently dropped; done means Task 8 runs deterministically and failed or missing output is detected or retried.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, github-actions
- Domain
- ci-cd, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100