rapidsai / rapidsai/shared-workflows
wheels CI: reduce CI time
@jameslamb is already working on this.
Since Apr 2, 2026.
- Dominant language
- Shell
- Stars
- 8
- Forks
- 33
- Avg merge
- 23h 51m
- Merged PRs (30d)
- 19
Description
Description
The wheels-publish workflow is responsible for a pretty narrow purpose... downloading wheel artifacts built in CI and publishing them to package repositories (like https://pypi.anaconda.org/rapidsai-wheels-nightly/simple/).
It really only needs a Python interpreter and a couple tools like twine, the anaconda CLI, and the gh CLI.
Today, it uses the rapidsai/ci-wheel container images (build from rapidsai/ci-imgs), which are around 4.5 GB compressed (https://hub.docker.com/r/rapidsai/ci-wheel/tags).
The time to run that workflow is dominated by container-image pulling. For example, look at a recent wheels-publish-cudf job (build link):
This issue tracks the work of designing and implementing changes to speed that up. Maybe some mix of:
- using a slimmer container image (or just
ubuntu-latest/ubuntu-slimrunners without a container) - removing telemetry
- using a shallow
gitcheckout
Benefits of this work
- reduced end-to-end time for RAPIDS nightly pipeline (https://github.com/rapidsai/workflows/actions/workflows/nightly-pipeline.yaml)
- reduced time to roll out a change across all of RAPIDS (related: https://github.com/rapidsai/build-planning/issues/238)
- reduced usage of CI resources
Acceptance Criteria
wheels-publishworkflows are faster than they were on February 17, 2026
Ideas
Just dumping some ideas here, this is not an implementation plan:
- use
ubuntu-latest/ubuntu-slimGitHub-hosted runners - use a smaller container image (like
python:latestorpython:3.13-slim) - create a new
rapidsai/ci-wheel-publishingor similar image- publishing a new image does bring new legal / compliance requirements though, this might be a lot of effort
- fiddle with the
rapidsai/ci-wheelimage- different compression?
- fewer layers?
- removing unused things?
- use shallow
gitcheckouts - remove telemetry (this workflow doesn't benefit much from it)
Approach
Related changes made since this started:
- remove docs files and build artifacts, consolidate RUN steps (https://github.com/rapidsai/ci-imgs/pull/368)
- clear caches, remove unnecessary
pyenvstuff (https://github.com/rapidsai/ci-imgs/pull/394 - more parallelization, skip unnecessary stuff like docs and tests in source builds of CPython and its dependencies (https://github.com/rapidsai/ci-imgs/pull/425)
- stop shipping
llvmin Ubuntucitestwheelimages (https://github.com/rapidsai/ci-imgs/pull/428) - remove source checkouts and telemetry from package-publishing jobs (https://github.com/rapidsai/shared-workflows/pull/585)
- use a different, faster-to-pull base image for wheel-publishing (#586)
- use
nvidia/cuda:*-base-*or similar forcitestwheel(https://github.com/rapidsai/ci-imgs/pull/408)
Notes
N/A
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.