redhat-developer / redhat-developer/rhdh-plugins
Add release workflow and issue-scope context to agent review instructions
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 48
- Forks
- 120
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 337
Description
What happened
On PR #4417, the review agent produced two categories of false positives across 4 review rounds (~$28 total). First, in review round 2, the agent surfaced a [medium] scope-completeness finding claiming the PR could not close issue #4046 because npm publish and CHANGELOG tasks were incomplete. The human explained that npm publish is handled by the repo's automated Version Packages workflow, not individual PRs, and the changeset entry already existed. Second, across all 4 rounds, the agent repeatedly flagged stale references in documentation files (design.md, proposal.md, annotation-scheme/spec.md) that were outside the issue's authorized scope. The human documented a clear scoping rule: only files cited by issue #4046 should be updated in this PR.
What could go better
The review agent lacks two pieces of repo-specific context that caused repeated noise. (1) It does not know the repo uses changeset-based automated releases via Version Packages PRs, so it incorrectly flags missing npm publish steps as incomplete scope. (2) It does not understand that PRs may intentionally limit changes to files and components cited in the linked issue, so it flags stale references in uncited files as defects rather than informational observations. Confidence: high for the release workflow gap (clearly documented in the human's triage). Moderate for the scope convention — this may be a team preference rather than a hard rule, and different PRs may have different scope policies. These findings re-appeared across rounds, which is separately tracked by fullsend-ai/fullsend#3073 and #3515.
Proposed change
Add two context paragraphs to .fullsend/AGENTS.md. (1) Under a 'Release workflow' heading: "This repo uses changesets for versioning. Individual PRs include a .changeset/*.md entry declaring the bump level. npm publish is handled automatically by the Version Packages workflow — do not flag PRs for missing publish or CHANGELOG steps." (2) Under a 'Review scope' heading: "When a PR references a specific issue (Closes #XXXX), changes may be intentionally limited to files and components cited in that issue. Stale references or inconsistencies in uncited files should be noted as informational observations, not flagged as defects requiring fixes in the current PR."
Validation criteria
The next 5 review agent runs on PRs in this repo that reference specific issues should produce zero false positives about missing npm publish steps or Version Packages workflow, and should distinguish in-scope defects from out-of-scope informational observations in their findings.
Generated by retro agent from https://github.com/redhat-developer/rhdh-plugins/pull/4417
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Edit .fullsend/AGENTS.md, adding the proposed Release workflow and Review scope context paragraphs. Check the existing instruction structure first, then validate the change against the stated criteria: the next five review-agent runs should avoid false positives about npm publishing and distinguish out-of-scope observations from defects.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 85/100