redhat-developer / redhat-developer/rhdh-plugins

Add openspec-implementation consistency verification step to boost AGENTS.md

Open Beginner friendly
#4,698 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation ready-for-triage ready-to-code Review effort 1/5 workspace/boost
Dominant language
TypeScript
Stars
48
Forks
120
Avg merge
2d 6h
Merged PRs (30d)
337

Description

What happened

On PR #4574, the code agent authored both an openspec spec file (openspec/specs/ogx-entity-provider/spec.md) and the implementation (OgxModelEntityProvider.ts) in the same commit. The spec stated that invalid caData should "still apply the configured caData and issue the request" (spec line 119), but the implementation returned undefined on PEM validation failure (code line 200), falling back to system CAs.

This spec-implementation divergence was the #1 finding across all 6 review rounds, flagged as HIGH severity in the first review (run 34520203260) and again in the second review (run 34524473245). The fix agent attempted to resolve it in iteration 1, but gabemontero later deliberately chose the fail-open behavior (commit 4d0d2db) without updating the spec, so the divergence persisted through all subsequent review rounds and into the merge.

The existing AGENTS.md has openspec guidance on scenario step discipline (lines 48–81) and cancellation checklists (lines 83–105), but no guidance on verifying that implementation matches spec before committing.

What could go better

A self-verification step — comparing each behavioral contract in the spec against the actual code before committing — would have caught this inconsistency at the source. The code agent would have either aligned the code with the spec or updated the spec to match its implementation choice, preventing the finding from appearing in the first place.

This was the most persistent and costly finding: it appeared in every review round, required fix agent work, and was ultimately resolved by human intervention rather than the automated workflow. Preventing it at the code-agent stage would have reduced the review churn significantly.

Confidence: moderate. The code agent may have made a deliberate implementation choice that differed from the spec text it wrote — a verification step would flag the inconsistency but cannot guarantee the agent resolves it correctly. However, even flagging it would prompt reconciliation before submission. The boost AGENTS.md already contains detailed openspec guidance that the code agent follows, so adding a verification step to the same section is the natural location.

Related: fullsend-ai/fullsend#3497 proposes general documentation-vs-code verification at the code agent definition level. This proposal adds repo-specific guidance for the openspec pattern used in the boost workspace.

Proposed change

In workspaces/boost/AGENTS.md, at the end of the "Specification-driven development" section (after line 81, before the cancellation subsection), add:

### Spec-implementation consistency check

After implementing code that has a corresponding openspec spec file
(under `openspec/specs/`), verify consistency before committing:

1. Re-read each behavioral contract in the spec (GIVEN/WHEN/THEN
   scenarios, prose requirements) and confirm the implementation
   matches. Pay special attention to error paths, fallback behavior,
   and edge cases — these are the most common sources of divergence.
2. If the implementation deliberately diverges from the spec (e.g., a
   different error-handling strategy), update the spec to match the
   actual behavior. The spec is the behavior source of truth and must
   not describe behavior the code does not implement.
3. If creating a new spec alongside new code, write the spec after
   the implementation is finalized, or re-read and reconcile the spec
   against the final code before committing.

Validation criteria

The next 3 code agent PRs that modify both an openspec/specs/ file and its corresponding plugin source should not have spec-implementation divergence flagged by the review agent in the first review round. Specifically, no HIGH-severity findings about spec-code contradiction should appear for PRs that touch both spec and implementation files.


Generated by retro agent from https://github.com/redhat-developer/rhdh-plugins/pull/4574

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Edit workspaces/boost/AGENTS.md at the end of the “Specification-driven development” section, before the cancellation subsection. Add the proposed spec-implementation consistency guidance, then review the surrounding openspec instructions for consistency. Done means the guidance is present and clearly explains reconciling openspec specs with implementation before committing.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
90/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.