redhat-developer / redhat-developer/rhdh-plugin-export-overlays

Add AGENTS.md review guidance for .fullsend/ harness configuration changes

Open Beginner friendly
#3,531 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation non-workspace-changes ready-for-triage ready-to-code
Dominant language
TypeScript
Stars
9
Forks
72
Avg merge
3d 11h
Merged PRs (30d)
143

Description

What happened

PR #3523 attempted to fix broken ci-diagnose and e2e-triage agent harnesses by committing a local copy of .fullsend/env/gcp-vertex.env — a scaffold file that fullsend v0.33 deliberately deleted. The review agent (run 33637056214) recommended approval after verifying no secrets were committed, path mappings were correct, and env var expansion was sound. However, it did not flag that the approach was architecturally wrong: fullsend v0.33 removed this file as part of migrating env var delivery to env.sandbox (ADR 0055). The human reviewer (durandom) caught this ~1.5 hours later and opened the correct fix in PR #3527, which removes the host_files reference and passes Vertex env vars via env.sandbox instead.

What could go better

The review agent lacks context about fullsend's harness architecture — specifically, that .fullsend/env/ files are scaffold-managed artifacts that should not be committed locally, and that env.sandbox is the current mechanism for passing environment variables to custom harness sandboxes. Without this context, the review agent cannot distinguish a correct harness config change from one that fights against the platform's architectural direction.

This is a high-confidence finding: the review agent's comment explicitly verified implementation correctness (paths, secrets, expansion) but had zero awareness of the scaffold lifecycle or the env.sandbox migration. The gap is purely informational — the agent had no way to know that restoring this file was wrong.

Note: platform-level fixes for this class of issue exist as open proposals (fullsend-ai/fullsend#1472, #3476), but those are longer-term. Repo-specific AGENTS.md guidance can close this gap immediately for this repo's review agent.

Proposed change

Add a new section to AGENTS.md (after the existing 'E2E Nightly Fix Conventions' section) covering .fullsend/ harness configuration review guidance. The section should include:

  1. Scaffold vs local files: Files under .fullsend/env/ are scaffold-managed by the fullsend CLI. Do not commit new env files there or restore deleted ones — if upstream fullsend removed a file, the removal was intentional.
  2. env.sandbox is the preferred mechanism: Custom harnesses (ci-diagnose, e2e-triage) should pass environment variables to the sandbox via the env.sandbox section in harness YAML (per ADR 0055), not via host_files referencing scaffold env files.
  3. Review checklist for .fullsend/ changes: When reviewing PRs that add or modify files under .fullsend/, verify the change follows fullsend's current architecture rather than reverting to deprecated patterns. Check whether host_files entries reference scaffold paths that may have been removed upstream.

This guidance is complementary to the existing prohibited-modifications list in 'E2E Nightly Fix Conventions' (which lists .fullsend/ as off-limits for E2E fix PRs but doesn't explain the harness architecture).

Validation criteria

The next review agent run on a PR that modifies .fullsend/ harness configuration should reference env.sandbox or ADR 0055 in its review when the change involves environment variable delivery. Specifically: if a PR adds a host_files entry referencing a scaffold env file, the review agent should flag it as potentially using a deprecated pattern. Validate on the next 3 PRs touching .fullsend/rhdh/harness/ files.


Generated by retro agent from https://github.com/redhat-developer/rhdh-plugin-export-overlays/pull/3523

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading AGENTS.md, especially the existing “E2E Nightly Fix Conventions” section, then review the .fullsend/rhdh/harness/ configuration and ADR 0055 references described in the issue. Add the new guidance after that section, covering scaffold-managed env files, env.sandbox, host_files, and the review checklist; done means the documented validation criteria are explicit.

Written by the indexing model from the issue text.

Assessment

Tech stack
markdown, yaml
Domain
devtools, documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.