Extending the reusable test-suite workflow in heavily-customized downstreams (e.g. setuptools)
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 150
- Forks
- 38
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
## Background
#199 factored the shared `test`/`collateral` setup into a reusable `test-suite.yml` (reusable workflows having been chosen over composite actions per the discussion in #162 / #163). For most consumers this is a clear win: `main.yml` shrinks to thin `uses:` calls, and the shared steps + env live in one place.
For heavily-customized downstreams it's a harder fit. setuptools is the motivating case. Its `test` job interleaves a lot of project-specific work *with* the shared steps:
- pre-building the sdist/wheel and exporting `PRE_BUILT_SETUPTOOLS_*`
- a cached download of setupcfg examples
- `SETUPTOOLS_USE_DISTUTILS` and a pypy `continue-on-error`
- publishing coverage
Plus two jobs that can't use the reusable workflow at all: `test_cygwin` (every step runs under a Cygwin bash shell) and `integration` (separate, tag-gated).
## The friction
1. **You can't inject steps into a reusable workflow.** A calling job may only set `uses`/`with`/`secrets`/`needs`/`if`/`strategy`/`permissions` — never `steps`. So setuptools can't "call the shared workflow and add its pre-build steps"; the custom steps have nowhere to live in the caller.
2. **So the only way to keep the pattern is to fork `test-suite.yml`** and add the project steps there, gated by an input (setuptools uses a `full-test` flag). But that customizes *the skeleton's own shared file* — the one the skeleton will keep iterating on — so every future skeleton update to `test-suite.yml` now conflicts. It inverts the usual "customize `main.yml`, inherit `test-suite.yml`" contract.
3. **Env can't be shared with decoupled jobs.** A called reusable workflow doesn't inherit the caller's top-level `env:`, and `test_cygwin`/`integration` can't call it, so the shared env block (`FORCE_COLOR`, `PIP_*`, `TOX_OVERRIDE`) ends up duplicated across `test-suite.yml` and each decoupled job.
setuptools' current resolution (in the pending skeleton merge) is Option-D-style: a forked `test-suite.yml` with a `full-test` input gating the setuptools-specific steps, and `test_cygwin`/`integration` decoupled with duplicated env. It works, but it accepts (2) and (3).
## Question
Could the skeleton offer extension points so a downstream can customize without forking `test-suite.yml`? A couple of directions:
- **Convention hooks** — `test-suite.yml` runs a downstream-provided script at a well-known path if it exists (e.g. a pre-tox `./.github/pre-test` and/or a post-tox hook). Degenerate/no-op by default; setuptools would move its pre-build / download-cache logic into such a script, keeping `test-suite.yml` upstream-pristine.
- **Composite actions for the genuinely shared concerns** (setup-python + tox; publish-coverage) that *both* `test-suite.yml` and decoupled jobs like `test_cygwin` could call — which would also address the env/coverage duplication in (3). (Acknowledging the composite-action tradeoffs already raised in #162 — collapsed/ugly log output in particular.)
Curious whether hooks, composites, or "downstreams just fork `test-suite.yml` and own it" is the intended story for cases like setuptools.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the reusable test-suite.yml and its main.yml callers, then compare the setuptools jobs described in the issue, including test_cygwin and integration. The work is done when the project agrees on an extension strategy—hooks, composite actions, or downstream-owned forks—and documents how customized consumers should use it without unresolved ambiguity.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, yaml
- Domain
- build-system, ci-cd
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100