redhat-developer / redhat-developer/rhdh-plugins

Add 0.x changelog verification documentation step to Backstage bump workflow

Open Beginner friendly
#4,624 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation fullsend ready-for-triage ready-to-code Review effort 1/5
Dominant language
TypeScript
Stars
48
Forks
120
Avg merge
2d 6h
Merged PRs (30d)
337

Description

What happened

The code agent on PR #4620 bumped the boost workspace to Backstage 1.54.4, including two 0.x packages with minor version increments: @backstage/frontend-plugin-api (^0.17.2 → ^0.18.0) and @backstage/repo-tools (^0.17.3 → ^0.19.0). The issue #4617 instructions included "read release notes, search source for affected APIs" as a verification step, and the code agent's tests/build/tsc all passed — suggesting it performed this verification. However, the PR body only documented that "All tests pass. Build and prettier checks pass" without mentioning upstream changelog review.

The review agent then flagged a low-severity semver-risk finding: "Under semver 0.x conventions, minor version increments may introduce breaking changes... Verify the upstream changelogs for breaking API changes." The finding was valid but generic — it asked the human reviewer to verify something the code agent likely already checked but didn't document. The human reviewer approved 24 minutes later without comments.

What could go better

The review agent's semver-risk finding is predictable: it will recur on every Backstage bump that touches 0.x packages, because the code agent does not document its changelog verification. This creates a pattern where (1) the code agent checks changelogs as instructed, (2) the review agent can't see evidence of this check, (3) the review agent flags a generic "verify changelogs" finding, and (4) the human must independently verify or trust the code agent's work.

If the code agent documented its changelog findings in the PR body (e.g., which 0.x packages had minor bumps, whether breaking changes were found, and what the relevant changelog entries said), the review agent could verify the documentation rather than flagging a generic risk. This would also give human reviewers explicit evidence, potentially reducing the review-to-merge gap.

Confidence: moderate. I'm confident the pattern will recur (any Backstage bump touching 0.x packages will trigger it), but I'm uncertain whether the human reviewer's 24-minute gap was actually spent checking changelogs or was simply availability lag.

Proposed change

Add a paragraph to .fullsend/AGENTS.md under or near the existing changeset bump-level guidance. The new instruction should state that when performing a Backstage version bump (yarn backstage-cli versions:bump), if any @backstage/* dependency has a 0.x version with a minor version change, the code agent must:

  1. Check the upstream Backstage changelog or release notes for those specific packages.
  2. Document in the PR body which 0.x packages had minor bumps and whether breaking changes were identified.
  3. If breaking changes are found, adjust the changeset severity from patch to minor for affected consuming packages and note the required migration steps.

Example PR body addition:

### 0.x dependency changes
- @backstage/frontend-plugin-api ^0.17.2 → ^0.18.0: Checked changelog — no breaking changes affecting boost plugins (new optional props added to `useRouteRef`).
- @backstage/repo-tools ^0.17.3 → ^0.19.0: Checked changelog — `@useoptic/openapi-utilities` dependency removed upstream, no API changes affecting boost.

Alternatively, if the rhdh-backstage-upgrade skill exists in the agents repo (fullsend-ai/agents), this instruction could be added there as an additional step after the existing "read release notes" step.

Validation criteria

On the next 3 Backstage version bump PRs for any workspace in this repo: (1) the PR body includes a section documenting 0.x changelog verification when applicable, and (2) the review agent does not flag a generic semver-risk finding for packages whose changelogs were already documented in the PR body.


Generated by retro agent from https://github.com/redhat-developer/rhdh-plugins/pull/4620

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Update .fullsend/AGENTS.md near the existing changeset bump-level guidance; first read the current Backstage bump instructions and check whether the rhdh-backstage-upgrade skill exists in fullsend-ai/agents. Document the required 0.x minor-bump changelog checks and PR-body evidence, including changeset severity and migration notes when breaking changes are found. Done is the next three applicable bump PRs documenting this verification without a generic semver-risk finding.

Written by the indexing model from the issue text.

Assessment

Domain
developer-experience, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.