redhat-developer / redhat-developer/rhdh-plugin-export-overlays
Add AGENTS.md review guidance for .fullsend/ harness configuration changes
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 9
- Forks
- 72
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 143
Description
What happened
PR #3523 attempted to fix broken ci-diagnose and e2e-triage agent harnesses by committing a local copy of .fullsend/env/gcp-vertex.env — a scaffold file that fullsend v0.33 deliberately deleted. The review agent (run 33637056214) recommended approval after verifying no secrets were committed, path mappings were correct, and env var expansion was sound. However, it did not flag that the approach was architecturally wrong: fullsend v0.33 removed this file as part of migrating env var delivery to env.sandbox (ADR 0055). The human reviewer (durandom) caught this ~1.5 hours later and opened the correct fix in PR #3527, which removes the host_files reference and passes Vertex env vars via env.sandbox instead.
What could go better
The review agent lacks context about fullsend's harness architecture — specifically, that .fullsend/env/ files are scaffold-managed artifacts that should not be committed locally, and that env.sandbox is the current mechanism for passing environment variables to custom harness sandboxes. Without this context, the review agent cannot distinguish a correct harness config change from one that fights against the platform's architectural direction.
This is a high-confidence finding: the review agent's comment explicitly verified implementation correctness (paths, secrets, expansion) but had zero awareness of the scaffold lifecycle or the env.sandbox migration. The gap is purely informational — the agent had no way to know that restoring this file was wrong.
Note: platform-level fixes for this class of issue exist as open proposals (fullsend-ai/fullsend#1472, #3476), but those are longer-term. Repo-specific AGENTS.md guidance can close this gap immediately for this repo's review agent.
Proposed change
Add a new section to AGENTS.md (after the existing 'E2E Nightly Fix Conventions' section) covering .fullsend/ harness configuration review guidance. The section should include:
- Scaffold vs local files: Files under
.fullsend/env/are scaffold-managed by the fullsend CLI. Do not commit new env files there or restore deleted ones — if upstream fullsend removed a file, the removal was intentional. - env.sandbox is the preferred mechanism: Custom harnesses (
ci-diagnose,e2e-triage) should pass environment variables to the sandbox via theenv.sandboxsection in harness YAML (per ADR 0055), not viahost_filesreferencing scaffold env files. - Review checklist for
.fullsend/changes: When reviewing PRs that add or modify files under.fullsend/, verify the change follows fullsend's current architecture rather than reverting to deprecated patterns. Check whetherhost_filesentries reference scaffold paths that may have been removed upstream.
This guidance is complementary to the existing prohibited-modifications list in 'E2E Nightly Fix Conventions' (which lists .fullsend/ as off-limits for E2E fix PRs but doesn't explain the harness architecture).
Validation criteria
The next review agent run on a PR that modifies .fullsend/ harness configuration should reference env.sandbox or ADR 0055 in its review when the change involves environment variable delivery. Specifically: if a PR adds a host_files entry referencing a scaffold env file, the review agent should flag it as potentially using a deprecated pattern. Validate on the next 3 PRs touching .fullsend/rhdh/harness/ files.
Generated by retro agent from https://github.com/redhat-developer/rhdh-plugin-export-overlays/pull/3523
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading AGENTS.md, especially the existing “E2E Nightly Fix Conventions” section, then review the .fullsend/rhdh/harness/ configuration and ADR 0055 references described in the issue. Add the new guidance after that section, covering scaffold-managed env files, env.sandbox, host_files, and the review checklist; done means the documented validation criteria are explicit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- markdown, yaml
- Domain
- devtools, documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 86/100