kangwonlee / kangwonlee/nmisp

CI: tiered container split — design decision before implementation (slim-glibc vs Alpine-musl for the lean tier)

Open
#389 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
12
Forks
76
PR merge metrics
No merged PRs in 30d

Description

## Goal

Most notebooks in this repo don't need TensorFlow / PyTorch / torchvision. Currently the container CI uses one image with a 4.2 GB conda env regardless. Splitting into a lean tier (most notebooks) + an ML tier (the ~4 TF/PyTorch notebooks) should yield meaningful CI time + image-size savings.

This issue is for the **design decision**, not the implementation — there are real tradeoffs worth settling before the rewrite.

## Design space — two candidate base images for the lean tier

| Option | Base | Lean image (est.) | Wheel availability | Build cost |
|---|---|---|---|---|
| A | `python:3.X-slim-bookworm` (glibc, Debian) | ~150–250 MB | full PyPI manylinux — `pip install` of any test dep works | none, official images |
| B | `ghcr.io/kangwonlee/edu-scipy:v0.4.0` (Alpine, musl) | 147 MB | musllinux only — TF/PyTorch/torchvision lack musllinux wheels | none for lean tier; affects future flexibility |

The 4.2 GB conda env is the dominant cost in the current single-tier setup — **most of the size win comes from dropping conda for the non-ML notebooks, not from Alpine specifically**. Slim-glibc captures most of that win without musl complications.

## Constraints worth naming explicitly

1. **No official TF/PyTorch musllinux wheels** — verified 2026-04-21. Building from source is technically possible but heavy: TF needs Bazel (4–6 h, 10–30 GB peak disk), PyTorch needs CMake (1–3 h, 8–15 GB peak disk). GitHub Actions free runners cap at 6 h wall-clock — TF won't fit reliably. Building ourselves can be revisited when we get access to higher-caliber build hardware.
2. **Maintenance burden of custom wheels** — every upstream release = rebuild; every CVE in a transitive dep = rebuild. Becomes a project of its own.
3. **Student reproducibility** — students run notebooks on Colab / Windows / macOS, never on Alpine. Custom musllinux wheels solve a CI-only problem at classroom-scale ongoing cost.
4. **Existing prior work on Alpine** — `ghcr.io/kangwonlee/edu-scipy:v0.4.0` (Alpine, 147 MB) was built 2026-04-12 and is already published. Sunk cost; not by itself a reason to commit to the Alpine path.
5. **Routing mechanism** — already exists in the codebase: `TEST_IPYNB_IGNORE_FOLDER` + per-notebook matrix. Either option uses this unchanged.
6. **The ML tier stays glibc either way** — the ~4 ML notebooks need TF/PyTorch and aren't moving off conda+miniforge regardless of the lean-tier choice.

## Suggested investigation order

1. **Pick the lean base image** (slim-glibc vs Alpine-musl) — load-bearing call. Default-favor slim-glibc unless musl gives a measured benefit beyond the ~50–100 MB image-size delta. Reopen if a future ML-tier dep grows or if the lean tier itself needs something musl-native.
2. **Specify the tier split in the workflow** — which folders go to which tier; concretize the matrix. *Illustrative only* (folder names need verification before commit): `00` / `10` / `15` / `20` / `28` / `30` / `35` / `40` / `60` → lean; ML demos and any TF/PyTorch notebooks → ml.
3. **Build + publish** `nmisp-test-lean` (whichever base) alongside existing `nmisp-test-ml`.
4. **Stage the rollout** — lean-tier CI green on a feature branch before flipping main.

## Related

- #370 — profile nightly image build (3–4× slower than 2023.09). Tactical; this issue is the architectural followup. Together they bound the image-size pain.
- Local verification harness exists at `~/Dev/sync-assignments/docker-sim/docker-sim02/`.

## Why

Long-horizon architectural win. Lean-tier image-pull drops from 4.2 GB to a few hundred MB; the bulk of CI minutes follow. Iteration loop on non-ML changes tightens. The work is bounded — design call (this issue) plus one workflow rewrite — not open-ended.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.