Security(H-06): runtime and release artifacts use mutable tags without signatures, provenance, or SBOM

Summary

Runtime and release artifacts are referenced by mutable tags rather than digests, and the release pipeline pulls and executes tooling with no integrity check. There is no chart or image signing, provenance, SBOM, or vulnerability gate, so a moved tag or compromised installer can change deployed bytes without changing the Git tag.

Severity: High · CWE-494 (Download of Code Without Integrity Check)
Validation: Confirmed against HEAD f932d80 on 2026-07-14 (assessed at 69c84f7; unchanged since).

Evidence

Mutable image references:

• Hub / singleuser sha-46289ab (values.yaml:524-525, 614-615, and profile duplicates values.yaml:432,445,450,463).
• Nebi sha-be3187c (values.yaml:227,231), NFS 0.8-repack (values.yaml:199-200), bootstrap python:3.12-slim (values.yaml:326).
• BusyBox 1.31 (config/jupyterhub/01-spawner.py:864), Ubuntu 22.04 and Alpine 3 (templates/nfs-client-installer.yaml:29,50).
• A sha--prefixed docker tag is derived from a git SHA but is still a mutable quay.io tag, not an image digest. The corresponding digest is even known and sits in a comment (values.yaml:517-520) but the enforced tag: field still uses the mutable tag.

Dockerfile (images/Dockerfile):

• Dockerfile:1,3 FROM ubuntu:24.04; Dockerfile:17 curl -fsSL https://pixi.sh/install.sh | ... bash (no checksum).
• Nebi/Starship/font downloads without hash verification (Dockerfile:124-127,134-136,141-142,157-158).

Release workflow (.github/workflows/release.yaml):

• release.yaml:28 fetches get-helm-3 from the main branch and pipes it to bash.
• release.yaml:49 runs helm dependency update despite a committed Chart.lock (this re-resolves and rewrites the lock; helm dependency build would verify against it).
• release.yaml:53 helm package . with no signing/provenance; release.yaml:79 git push origin gh-pages --force; release.yaml:99-103 uses a central-repo PAT.
• A repo-wide grep of .github/workflows/ for cosign|sigstore|sbom|syft|trivy|grype|provenance|--sign|attest|pip-audit|safety finds no signing, provenance, SBOM, or vulnerability tooling.

Mitigating controls already present (kept for accuracy): image envs use pixi install --locked against a committed pixi.lock, nearly all GitHub Actions are SHA-pinned, and fork PRs are blocked from registry pushes (build-image.yaml:44-58). The weak spot is the base image, the installer curl-pipe, and downloaded binaries, not the conda/pypi graph.

Impact

A moved registry tag, compromised installer, replaced chart archive, or mutable base image can change deployed code without changing this Git tag. The highest-value targets are the components that hold Keycloak admin credentials, user refresh tokens, hub service tokens, persistent data, or privileged node access. A checksum produced through the same release channel proves consistency with that channel, not independent origin.

Remediation

• Pin every runtime and build image by digest.
• Pin every release tool to an exact version and verify its checksum.
• Use helm dependency build from the committed lock.
• Sign Helm charts and OCI images; publish source-linked provenance and SBOMs.
• Gate source and final images on vulnerability policy.
• Use protected release environments and least-privilege publishing identities; remove mutable @main references from secret-bearing automation.
• Enforce digest and signature policy at admission.

Acceptance criteria

• Reinstalling the release resolves to identical chart and image digests.
• The chart package is reconstructed from Chart.lock without renegotiation.
• Signatures and provenance verify independently of the download channel and name the exact source commit.
• SBOMs and scan results exist for every digest.
• Admission rejects mutable, unsigned, or unprovenanced images.

_{Source: data-science-pack 0.1.0 security assessment (pinned commit 69c84f72df259ec755ed40bfc83f20158c550d55), finding H-06.}

Start with values.yaml, config/jupyterhub/01-spawner.py, templates/nfs-client-installer.yaml, images/Dockerfile, and .github/workflows/release.yaml, then review the listed mutable references and the workflow's dependency and publishing steps. Done means the acceptance criteria are met: reproducible digests, locked chart dependencies, independently verifiable signatures and provenance, SBOMs and scan results, and admission rejection of mutable or unsigned images.