dependabot / dependabot/dependabot-core
bin/spec (dry-run specs) never runs in CI
- Dominant language
- Ruby
- Stars
- 5.8k
- Forks
- 1.5k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 149
Description
### Code improvement description
The `ci (bin, dry_run, common)` job skips its tests and reports success. Its condition has not been able to match since #11981 added it.
- The matrix entry is `{ path: bin, name: dry_run, ecosystem: common }` ([ci.yml#L23](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L23)).
- Both steps are gated on `steps.changes.outputs[matrix.suite.path] == 'true'` ([L73](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L73), [L77](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L77)), i.e. `outputs['bin']`.
- The filter is named `dry_run`, not `bin` ([ci-filters.yml#L12](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/ci-filters.yml#L12)), so that output never exists. `bin` is the only matrix `path` without a matching filter.
Demo: lacostej/dependabot-core#1 adds only a spec that always fails (`expect(true).to be(false)`). The [dry_run job](https://github.com/lacostej/dependabot-core/actions/runs/34949660071/job/104317335349) is green in 21s. Its log shows `Filter dry_run = true` matching `bin/spec/ci_wiring_demo_spec.rb` and `Changes output set to ["dry_run","common"]`, then "Build ecosystem image" and "Run dry_run tests" skipped. Upstream PRs that changed `bin/dry-run.rb` show the same, e.g. #16085.
Renaming the filter alone would turn the skip into a failure:
- The step runs `cd /home/dependabot/bin && ./script/ci-test` ([L86](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L86)), but `bin/script/ci-test` does not exist.
- The `common` image copies only `common/` and `updater/` ([common/Dockerfile](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/common/Dockerfile#L8-L9)), not `bin/`.
- The specs run dry-run against `npm_and_yarn`, which the `common` image doesn't include. Run locally in the bundler dev image, 2 of 21 examples fail with `Unsupported package_manager npm_and_yarn`.
Found while working on #16298, which adds dry-run specs that this job will not run.
Contributor guide
Research direction
Start with the matrix and step conditions in .github/workflows/ci.yml, then compare the dry_run output in .github/ci-filters.yml. Read common/Dockerfile and inspect bin/spec plus the referenced bin/script/ci-test path; run the dry-run specs locally in the bundler development image. Done means CI executes these specs in an image with the required files and package manager support, exposing their failures instead of reporting a skipped green job.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions, ruby
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100