Ecosystem reports can use a stale baseline
- Dominant language
- Rust
- Stars
- 49.6k
- Forks
- 2.4k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 458
Description
Ecosystem reports on pull requests can include "false" changes from a recently committed change.
The way this works is:
- We download the baseline Ruff binary from the last CI build on `main`
- We use the pull request Ruff binary for comparison which is building on a temporary branch that is _merged_ with `main`
- If the last commit on `main` has not finished CI (i.e. a 20 minute gap or whatever) then the commit will be present in the comparison but not the baseline.
We could solve this by building the baseline from the base commit instead of downloading it from a finished CI run. This would significantly increase the runtime of ecosystem checks though. We may want to just leave it as-is.
Contributor guide
Research direction
Trace the ecosystem report checks and the CI logic that downloads the baseline Ruff binary from the last completed main build. Compare that behavior with using the pull request's base commit, and determine whether the stale-baseline false changes can be prevented without an unacceptable runtime increase.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ci-cd
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100