CI: profile nightly image build (3-4x slower than 2023.09)
- Dominant language
- Jupyter Notebook
- Stars
- 12
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
The `nightly` variant of `build-test-image.yml` consistently takes substantially longer than `2023.09`. Observed on commit 593c05fd:
| Variant | Build time |
|---|---|
| `build 2023.09` | 6m56s |
| `build nightly` | ~24m (gated the whole `CI` workflow at 42m total) |
Since the nightly image is rebuilt on every change to `tests/environment.*.yml`, `docker/Dockerfile.nmisp-test`, or the workflow itself, this gating cost is paid often.
## Suggested investigation
1. Compare `tests/environment.2023.09.yml` and `tests/environment.nightly.yml` — likely the nightly env pulls heavier ML deps (TensorFlow / PyTorch nightly wheels are large).
2. Check whether `docker/build-push-action`'s GHA cache (`cache-from: type=gha`, `cache-to: type=gha,mode=max`) is actually hitting on these builds — if cache is invalidated by every Dockerfile change the cache provides little value.
3. Consider splitting the heavy ML install into its own Dockerfile stage so it caches independently from the rest of the env.
## Why
The nightly build dominates the total wall-clock of any CI change that touches the Dockerfile/env. Halving it would meaningfully tighten the iteration loop on container/CI changes.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.