githubnext / githubnext/agentics

repo-assist: Task 8 (Monthly Activity Summary) does not run deterministically despite being marked mandatory

未关闭
#372 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Makefile
星标
952
派生
139
平均合并
6 天 10 小时
30 天内合并 PR
2

描述

## Summary

In `repo-assist.md`, Task 8 (Update Monthly Activity Summary Issue) is documented as mandatory every run:

> "Always do Task 8 (Update Monthly Activity Summary Issue) every run."

In practice, on the `tenstorrent/tt-metal` deployment, it is **not** running deterministically. Over the last 7 daily scheduled runs (2026-07-24 through 2026-07-30, all completed successfully), the monthly summary issue ([#50956](https://github.com/tenstorrent/tt-metal/issues/50956)) was only actually updated on 2 of them.

## Evidence

| Date | Run | Task 8 landed? |
|---|---|---|
| 07-24 | [30072973467](https://github.com/tenstorrent/tt-metal/actions/runs/30072973467) | ❌ Called `update_issue`; safe-outputs manifest logged it as processed, but no edit ever appears on the issue's timeline. Silently dropped. |
| 07-25 | [30147764461](https://github.com/tenstorrent/tt-metal/actions/runs/30147764461) | ✅ `add_comment` landed |
| 07-26 | [30191579550](https://github.com/tenstorrent/tt-metal/actions/runs/30191579550) | ❌ Trace shows it read the issue and planned "Task 8: Update monthly activity summary," then never called it |
| 07-27 | [30244126524](https://github.com/tenstorrent/tt-metal/actions/runs/30244126524) | ✅ `add_comment` landed |
| 07-28 | [30335618590](https://github.com/tenstorrent/tt-metal/actions/runs/30335618590) | ❌ `state.json` memory records `"task8_monthly_summary"` as done; actual `add_comment` calls that run targeted other issues/PRs (51294, 51296, 51024) instead |
| 07-29 | [30429064534](https://github.com/tenstorrent/tt-metal/actions/runs/30429064534) | ❌ Same pattern — reads #50956, reasons through "Task 8," then spends its comment budget on stale-PR nudges |
| 07-30 | [30520369864](https://github.com/tenstorrent/tt-metal/actions/runs/30520369864) | ❌ Same again |

Issue #50956's `updated_at` has been stuck at `2026-07-27T07:02:56Z` since — three consecutive daily runs produced zero visible output on it, despite each run's own reasoning trace explicitly citing "Task 8" as a planned step.

## Root cause (best guess)

"Always do Task 8" is a prose instruction competing for attention/turn budget against the round-robin task selection in the same prompt, not a structurally-guaranteed step in the compiled workflow. Nothing fails or flags the run if the write doesn't happen. The agent's own repo-memory (`state.json`) isn't validated against ground truth either — it records `task8_monthly_summary: done` even on runs where no comment/edit actually reached the issue. On 07-24, the `update_issue` safe-output specifically appears to fail silently (logged as processed, but no visible effect and no error).

## Question

Is there a supported pattern for making a task like this deterministic — e.g.:
- Splitting it into its own workflow/job that always runs, decoupled from the round-robin agent turn
- A safe-outputs guarantee/verification step that fails the run (or retries) if a declared "always do" output type wasn't actually emitted
- Reconciling agent-reported `state.json` progress against actual GitHub state rather than trusting the agent's self-report

Happy to help test a fix if there's a preferred approach.

贡献指南

这个仓库没有索引到贡献指南

调研方向

Start with repo-assist.md Task 8, then compare the seven linked workflow runs, their traces, safe-outputs manifests, and state.json against issue #50956's timeline. Determine where the mandatory update can be skipped or silently dropped; done means Task 8 runs deterministically and failed or missing output is detected or retried.

由索引模型根据 Issue 内容生成。

评估

技术栈
github, github-actions
领域
ci-cd, devops
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
冷清
描述清晰度
基本清楚
新手友好度
52/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。