bevyengine / bevyengine/bevy

Auto-detect breaking changes to ensure migration guides exist

Open
#11,990 1 comment 1 reaction 0 assignees View on GitHub
A-Build-System A-Cross-Cutting C-Feature
Dominant language
Rust
Stars
48.2k
Forks
4.8k
Avg merge
3d 16h
Merged PRs (30d)
171

Description

We chatted on Mastodon about breaking changes: https://mastodon.gamedev.place/@alice_i_cecile/111960897354647011

I believe the key points were:
- It's very important that each breaking change comes with a migration guide for users.
- Currently, pull requests are manually triaged for breaking changes. PRs with breaking changes are labeled `C-Breaking-Change`, and there's a CI job that ensures the existence of a migration guide in the PR.
- There are situations where an additional way of detecting breaking changes may be helpful. For example, if a complex PR contains multiple breaking changes, they should all come with instructions in the migration guide. This is also done via human review.
- Research shows that despite our best efforts, we humans miss things quite a lot: more than 1 in 6 of the top 1000 Rust crates has accidentally shipped a breaking change at least once, and over 3% of all releases we tested had breakage that wasn't supposed to be there: https://predr.ag/blog/semver-violations-are-common-better-tooling-is-the-answer/

If you're interested, I believe it to be possible to use `cargo-semver-checks` to independently analyze a PR's contents and produce a list of the breaking changes it found. This list may be incomplete — we still have many not-yet-implemented lints — but together with the reviewer's judgment will likely catch many instances of breaking changes and missing migration guides that otherwise may have slipped through uncaught.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing CI job that checks for migration guides on pull requests, then investigate how cargo-semver-checks can analyze PR contents and report breaking changes. The work is done when the automated analysis is integrated with the existing review flow and helps identify breaking changes that lack migration guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ci-cd, devtools, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.