Add DOCKER install mode to backend.ai-installer for compose deployments of the published service images
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Motivation
The epic publishes the seven `lablup/backend.ai-*` service images, and `docker/README.md` now documents a verified compose deployment (config mounts, privilege matrix, path-parity contract). But standing such a deployment up is still entirely manual: the operator must hand-write the compose file and per-service TOMLs, initialize the database schema, load every fixture, populate etcd, pull the well-known kernel images, and trigger an image rescan — all things `backend.ai-installer` (`src/ai/backend/install/`) already automates for the DEVELOP and PACKAGE modes.
## Objective
A new **DOCKER** install mode: one installer run on a clean Docker host produces a fully bootstrapped, running deployment of the published images — feature-equivalent to what PACKAGE mode produces with wheels/scies.
## Details
1. **Mode plumbing**: add `DOCKER` to `InstallModes` (and the corresponding `InstallType`) in `src/ai/backend/install/types.py`, and a `DockerContext(Context)` alongside `DevContext` / `PackageContext` in `context.py`, selectable from the TUI (`app.py`) and CLI.
1. **Deployment generation**: emit a self-contained deployment directory —
- `docker-compose.yaml` composing the seven service images pinned to the installer's own version, plus the halfstack dependencies (PostgreSQL, Valkey, etcd) and the GraphQL gateway with the release's `supergraph.graphql`.
- Per-service config files at the mount targets documented in `docker/README.md` (`manager.toml`, `agent.toml`, `webserver.conf`, `storage-proxy.toml`, `proxy-coordinator.toml`, per-worker `proxy-worker.toml`s with unique authority/protocol/ports).
- The full privilege/parity contract for manager and agent from `docker/README.md`: `network_mode: host`, `privileged`, docker.sock, `/etc/machine-id`, agent `pid: host` + `cgroup: host`, GPU `deploy.resources.reservations.devices` on GPU nodes, and the parity mounts (`/var/lib/backend.ai`, `/tmp/backend.ai`, `/tmp/backend-ai-krunner`) matching the generated absolute `scratch-root` / `ipc-base-path` / `var-base-path`.
1. **Bootstrap steps** — reuse the existing `Context` machinery against the composed halfstack rather than re-implementing:
- Manager DB schema initialization + alembic migrations, and `install_appproxy_db()` for the appproxy schema.
- etcd configuration population (`etcd_put_json` config tree, redis/storage-proxy/volume settings).
- `load_fixtures()` plus `configure_appproxy_fixture()` / `configure_sftp_agent_fixture()`.
- `prepare_local_vfolder_host()` with the storage-proxy parity mount.
- `populate_images()` + `alias_image()`: pull the well-known kernel images in advance and register aliases, then trigger the manager image rescan so sessions are launchable immediately.
1. **Version alignment**: pin every service image to the same release as the installer binary — mixed versions on the shared event bus fail at runtime with message deserialization errors (see the warning in `docker/README.md`).
1. Existing modes (DEVELOP / PACKAGE / MAINTAIN / CONFIGURE) must be unaffected; DOCKER mode should reuse their step implementations wherever the step is not docker-specific.
## Success Criteria
- [ ] On a clean host with Docker + NVIDIA container toolkit, a single installer run in DOCKER mode yields a running deployment: all seven service containers healthy, schema migrated, fixtures loaded, etcd populated.
- [ ] Well-known kernel images are pulled and rescanned during install; an interactive session can be created end-to-end immediately after install without manual steps.
- [ ] The generated compose/config set conforms to the `docker/README.md` privilege and path-parity contract (including `/tmp/backend-ai-krunner`).
- [ ] Existing install modes behave identically to before.
JIRA Issue: BA-7321
Contributor guide
Research direction
Read src/ai/backend/install/types.py, context.py, and app.py, then review the deployment contract in docker/README.md. Trace how existing DEVELOP and PACKAGE contexts perform bootstrap steps before designing DockerContext and its generated compose/config directory. Done means a clean Docker host can run the published, version-aligned services with schemas, fixtures, etcd, kernel images, and rescan completed without changing existing modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose, postgresql, python
- Domain
- backend, devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100