dependabot / dependabot/dependabot-core

bin/spec (dry-run specs) never runs in CI

Open
#16,299 0 comments 0 reactions 0 assignees View on GitHub
L: github:actions L: javascript L: ruby:bundler T: tech-debt ⚙️
Dominant language
Ruby
Stars
5.8k
Forks
1.5k
Avg merge
2d 18h
Merged PRs (30d)
149

Description

### Code improvement description

The `ci (bin, dry_run, common)` job skips its tests and reports success. Its condition has not been able to match since #11981 added it.

- The matrix entry is `{ path: bin, name: dry_run, ecosystem: common }` ([ci.yml#L23](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L23)).
- Both steps are gated on `steps.changes.outputs[matrix.suite.path] == 'true'` ([L73](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L73), [L77](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L77)), i.e. `outputs['bin']`.
- The filter is named `dry_run`, not `bin` ([ci-filters.yml#L12](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/ci-filters.yml#L12)), so that output never exists. `bin` is the only matrix `path` without a matching filter.

Demo: lacostej/dependabot-core#1 adds only a spec that always fails (`expect(true).to be(false)`). The [dry_run job](https://github.com/lacostej/dependabot-core/actions/runs/34949660071/job/104317335349) is green in 21s. Its log shows `Filter dry_run = true` matching `bin/spec/ci_wiring_demo_spec.rb` and `Changes output set to ["dry_run","common"]`, then "Build ecosystem image" and "Run dry_run tests" skipped. Upstream PRs that changed `bin/dry-run.rb` show the same, e.g. #16085.

Renaming the filter alone would turn the skip into a failure:

- The step runs `cd /home/dependabot/bin && ./script/ci-test` ([L86](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/.github/workflows/ci.yml#L86)), but `bin/script/ci-test` does not exist.
- The `common` image copies only `common/` and `updater/` ([common/Dockerfile](https://github.com/dependabot/dependabot-core/blob/2a997d2c7e6513d51b1ea70a6cbf9cf3381a1335/common/Dockerfile#L8-L9)), not `bin/`.
- The specs run dry-run against `npm_and_yarn`, which the `common` image doesn't include. Run locally in the bundler dev image, 2 of 21 examples fail with `Unsupported package_manager npm_and_yarn`.

Found while working on #16298, which adds dry-run specs that this job will not run.

Contributor guide

Open the contributing guide

Research direction

Start with the matrix and step conditions in .github/workflows/ci.yml, then compare the dry_run output in .github/ci-filters.yml. Read common/Dockerfile and inspect bin/spec plus the referenced bin/script/ci-test path; run the dry-run specs locally in the bundler development image. Done means CI executes these specs in an image with the required files and package manager support, exposing their failures instead of reporting a skipped green job.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions, ruby
Domain
ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.