redhat-developer / redhat-developer/rhdh-plugin-export-overlays
Add AGENTS.md guidance for test.runOnce key scoping in multi-project workspaces
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 9
- Forks
- 72
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 143
Description
What happened
PR #3318 added a bulk-import-app-next Playwright project that runs the same spec file as the existing bulk-import project. The spec used test.runOnce("bulk-import-rhdh-setup", ...) with a hardcoded key. Because runOnce flags are process-scoped and keyed by string alone in a shared /tmp/ directory, whichever project ran second would silently skip configure() + deploy(), leaving no RHDH deployment in that namespace. All 9 tests would fail with a misleading "missing heading" error.
The review agent caught this on the first review (commit edae03f, $1.21, 5m50s) and requested changes. The author fixed it in 3 minutes by scoping the key with ${rhdh.deploymentConfig.namespace}, matching the pattern deploy() uses internally.
The PR author explicitly noted: "Worth noting for the remaining migrations: none of the five already-migrated workspaces use runOnce, which is why no earlier NFS migration surfaced this. Any workspace that does — and runs one spec in both shells — has the same trap."
What could go better
The review agent caught this bug, which is the right backstop — but a code agent performing a future NFS migration would likely reproduce the same mistake if it copies from the pre-fix version of this spec or another workspace that uses runOnce with a literal key. AGENTS.md already documents test.runOnce usage patterns (including the deploy() internal guard and when runOnce is vs. isn't needed), but it does not warn about the key collision trap when the same spec runs in multiple projects.
Confidence: high. The AGENTS.md E2E section is actively consumed by both code and review agents (evidenced by the review agent's cross-workspace comparison in this very PR), and similar guidance issues have been filed and implemented before (#3112, #2997, #3216).
Proposed change
Add a subsection under the existing test.runOnce documentation in AGENTS.md (after the "Key rules" bullet list) warning about key collisions in multi-project workspaces:
Content to add:
- When the same spec file runs in multiple Playwright projects (e.g.
bulk-importandbulk-import-app-next), a literalrunOncekey creates a collision — the first project's flag satisfies the second, silently skipping its deployment. - Always scope
runOncekeys by namespace:`workspace-setup-${rhdh.deploymentConfig.namespace}`. This mirrors whatdeploy()does internally (deploy-${namespace}). - The failure mode is silent: all tests in the second project fail on a missing UI element with no error pointing to the skipped deployment.
- Reference PR #3318 as the case study.
File: AGENTS.md, in the E2E Testing > test.runOnce section.
Validation criteria
The next code agent PR that adds an -app-next project to a workspace using test.runOnce should scope the key by namespace from the start, without requiring a review agent catch-and-fix cycle. Measurable over the next 3 NFS migration PRs that involve runOnce.
Generated by retro agent from https://github.com/redhat-developer/rhdh-plugin-export-overlays/pull/3318
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open AGENTS.md and locate the E2E Testing > test.runOnce section, then review PR #3318 for the multi-project collision case. Done means the section explains namespace-scoped keys, the silent skipped-deployment failure mode, and references PR #3318 so future migrations avoid the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, typescript
- Domain
- documentation, testing-qa
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 92/100