Publish Backend.AI service Docker images via GitHub Actions with Docker Hub OIDC authentication
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
## Objective
Automatically bake and publish the Backend.AI service Docker images from `docker/*.dockerfile` to the `lablup` Docker Hub organization on every release tag, authenticating with **Docker Hub OIDC connections** (no stored PAT/password secrets in GitHub).
Reference: [Docker OIDC connections for GitHub Actions](https://www.docker.com/blog/docker-oidc-connections-for-github-actions-available-for-docker-orgs/) / [Docker docs](https://docs.docker.com/enterprise/security/oidc-connections/create-manage/)
## Scope
|Dockerfile|Image|
|---|---|
|`docker/backend.ai-manager.Dockerfile`|`lablup/backend.ai-manager`|
|`docker/backend.ai-agent.dockerfile` (new, BA-7271)|`lablup/backend.ai-agent`|
|`docker/backend.ai-appproxy-coordinator.dockerfile`|`lablup/backend.ai-appproxy-coordinator`|
|`docker/backend.ai-appproxy-worker.dockerfile`|`lablup/backend.ai-appproxy-worker`|
|`docker/backend.ai-webserver.dockerfile`|`lablup/backend.ai-webserver`|
|`docker/backend.ai-storage-proxy.dockerfile`|`lablup/backend.ai-storage-proxy`|
|`docker/backend.ai-client.dockerfile`|`lablup/backend.ai-client`|
Out of scope:
- Infra images (`krunner-extractor`, `socket-relay`, `linuxkit-nsenter`) — self-contained in backend.ai-krunner, not published here.
- Refactoring agent `docker load`/`docker exec` subprocess calls to aiodocker (backlog follow-up noted in BA-7271).
- Not related to the isla-sorna pipeline; that dispatch stays as is.
## Design decisions (confirmed)
- **Auth**: Docker Hub OIDC connection (lablup org is a Team account with the feature available). `docker/login-action@v4` (≥ 4.5.0), `username: lablup`, no password, `DOCKERHUB_OIDC_CONNECTIONID` env from a GitHub Actions repo variable, `permissions: id-token: write`.
- **Ruleset subject claim**: `repo:lablup/backend.ai:environment:deploy-to-dockerhub` — push jobs run inside a protected GitHub environment (mirrors the existing `deploy-to-pypi` pattern used for PyPI trusted publishing).
- **Trigger**: tag push (`refs/tags/*`), chained after the existing `build-wheels` job in `ci.yml`; the image build reuses the `wheels` artifact (dockerfiles install wheels from `dist/` + root `requirements.txt`, build context = repo root, build args `PYTHON_VERSION` from `pants.toml`, `PKGVER` from `VERSION`). Manual reruns via `workflow_dispatch`.
- **Multi-arch**: linux/amd64 + linux/arm64 (`scripts/build-wheels.sh` already produces aarch64 wheels).
- **Tagging**: version tag from the git tag; `latest` only for final (non-rc) releases, reusing `scripts/determine-release-type.py`.
- **Agent image (BA-7271)**: DooD via the host Docker socket; Docker CLI obtained with `COPY --from=docker:-cli` (NOT as base image — Alpine/musl cannot install our manylinux wheels). Path parity, host network, and cgroupfs mount are deployment conventions documented in compose/docs, not code.
- Follow `.github/AGENTS.md` guardrails: logic in `scripts/`, no `${{ `}} interpolation into shell, third-party actions pinned to commit SHAs.
JIRA Issue: BA-7264
Contributor guide
Research direction
Read .github/AGENTS.md and the existing deploy-to-pypi pattern in ci.yml first. Inspect the listed docker/*.dockerfile files, the wheels artifact flow, scripts/build-wheels.sh, scripts/determine-release-type.py, pants.toml, and VERSION. Done means release tags publish all scoped amd64 and arm64 images through Docker Hub OIDC, with latest only for final releases and manual reruns supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions, python
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100