nebari-dev / nebari-dev/data-science-pack

Migrate release CI onto the shared nebari-dev/.github@v1 reusable workflows

Open
#180 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5
Forks
7
Avg merge
1d 20h
Merged PRs (30d)
13

Description

The org now has shared reusable release workflows in nebari-dev/.github, tagged @v1:

  • pack-build-image.yaml - build one image and push to ghcr.io + quay.io, tagged sha-<short> (+ latest on the default branch).
  • pack-release.yaml - on a Chart.yaml bump: pin the image tags to the release commit's sha (in the packaged chart only), helm package, create a GitHub Release with generated notes, and sync the chart to the central helm-repository.

nebari-dev/llm-serving-pack is the first consumer (see its .github/workflows/build-images.yaml + release.yaml, which call nebari-dev/.github/.github/workflows/pack-*.yaml@v1). This pack still uses its own copied build-image.yaml / build-images.yaml / bump-image-tags.yaml / release.yaml + scripts/bump_image_tags.py. Those are the source the shared workflows were extracted from, and they will drift over time. This issue tracks migrating data-science-pack onto the shared @v1 workflows so every pack shares one pipeline and inherits fixes centrally.

What migrating involves (and the open questions to scope first)

  • Model shift: bump-to-main -> pin-at-release. Today bump-image-tags.yaml auto-commits sha-<short> image tags into values.yaml on main using a BOT_PAT (a bypass actor on the branch ruleset). The @v1 pack-release pins tags at package time in the published artifact only, with no writes to main and no BOT_PAT. Migrating lets this pack retire bump-image-tags.yaml and the BOT_PAT dependency; main's values.yaml would float (e.g. latest) while the published chart carries the pinned shas.
  • Multi-arch gap (main blocker). This pack's build-image.yaml builds arm64 on native runners (per-arch build + digest merge). The @v1 pack-build-image is single-platform via a platforms input (amd64 today) with no native-per-arch digest-merge. The jupyterlab images build arm64, so either pack-build-image needs a multi-arch mode (single-step multi-platform, or a native-runner option) before this pack can fully migrate, or the pack accepts amd64-only. Worth scoping before anything else.
  • Bespoke pin targets. scripts/bump_image_tags.py rewrites profile-list image refs (jupyterhub.custom.profiles[*]... and per-profile choices), not just simple <component>.image.tag dotted paths. The @v1 pin step takes a tag-paths list of dotted keys, so the profile-list rewriting needs either an extension to the shared pin script or a pack-provided hook.
  • Latent quay-login bug worth checking. This pack logs into quay with ${{ vars.QUAY_USERNAME }}, but QUAY_USERNAME is an org secret, not an Actions variable (this bit llm-serving-pack on its first run with "Username required"). It is worth confirming this pack's quay pushes actually succeed today; the @v1 workflow reads secrets.QUAY_USERNAME (forwarded by the caller), which fixes it.

Fast-follow

software-pack-template should also reference the @v1 workflows so new packs inherit them instead of copying.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare .github/workflows/build-image.yaml, build-images.yaml, bump-image-tags.yaml, and release.yaml plus scripts/bump_image_tags.py with the @v1 callers in nebari-dev/llm-serving-pack. Scope the multi-arch build and profile-list pinning questions first, and confirm the current QUAY_USERNAME behavior; done means data-science-pack uses the shared workflows without the copied pipeline or BOT_PAT dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, helm, python
Domain
ci-cd, devops, release
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.