lablup / lablup/backend.ai

Implement spec-driven multi-node deployment in the install package

Open
#8,838 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

Add a spec-driven deployment path to the ai.backend.install package that deploys Backend.AI to local, single-node remote (SSH), and multi-node (including HA) targets from one declarative spec. Single-node local is just a one-node spec running the same code path.

Approach (decided 2026-07):
- Do NOT reintroduce pyinfra. The earlier pyinfra scaffold was removed as unused (#11231) because it duplicated install logic that already lives in context.py; the BA-556x "unify TUI<->pyinfra config" stories are the drift tax of that duplication.
- Reuse the existing install steps in context.py. They already funnel through 4 primitives (run_exec / run_shell / copy_config / sed_in_place) and read all addresses from InstallInfo / HalfstackConfig / ServiceConfig (types.py).
- Externalize the spec: replace the hardcoded, localhost-only hydrate_install_info() (context.py:129 "# TODO: multi-node setup") with a loader that builds InstallInfo from a deployment spec.
- Add a RemoteContext: a Context subclass overriding the 4 primitives over SSH so every existing install step runs on a remote node unchanged. The local node keeps the current local Context.

Goals
- One declarative spec (nodes, roles, services, infra) drives local / single-node SSH / multi-node deployments.
- Multi-node HA wiring: Redis Sentinel, multi-Manager, webserver failover endpoint list.
- Steps split into individually executable, dependency-ordered units (halfstack -> manager -> migration -> agent/storage/webserver).
- Coexists with the existing Textual TUI installer, sharing its config templates and install logic.

Constraints
- Linux for remote targets; macOS local only.
- Deployment history/rollback out of scope (future SQLite extension).
- Keep install/ as a locked leaf component (deps: common/cli/plugin) - no heavy 3rd-party orchestration dependency; SSH transport via a thin library (asyncssh).

JIRA Issue: BA-4407

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.