lablup / lablup/backend.ai

LocalDockerDaemon container registry type for ease of development/single-node setup

Open
#411 1 comment 0 reactions 1 assignee Claimed by @jopemachine View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

Currently we need to explicitly configure the container registries and perform 'rescan' of them to get the image metadata and either pull the images to the agent nodes manually or let the session creation process pull automatically.

**Other container registries:**

```mermaid
flowchart TD
subgraph manager-node
Manager
end
CR[Container Registry]
subgraph agent-node
Agent
end
Manager -->|rescan| CR
Agent -->|pull images| CR
```

**LocalDockerDaemon container registry:**

```mermaid
flowchart TD
subgraph all-in-one-node
Manager
CR[local Docker daemon]
Agent
end
Manager -->|rescan| CR
Agent -.->|just use images| CR
```

In single-node / development setup, it might be better to make the manager to auto-detect any locally available Docker images and self-rescan them, so that we could just pull (or even build) the image to test it.

This may be also useful for building a CI/CD for our image builds because a single-node setup of Backend.AI cluster could be used to run an integration test of newly built container images, with less efforts to setup a separate registry and rescan it.

- [ ] Add a local config option (boolean flag) whether to use this feature or not: `manager.enable-local-docker-images` in `manager.toml`\* In production/multi-node setups, this should be disabled.

- [ ] Add a new container registry backend `LocalDockerDaemonRegistry` which uses the docker API to enumerate and fetch the images and their archs/labels from the local Docker daemon.
- [ ] Trigger rescanning with this registry when:\* The manager starts up.
- The manager receives an image pull event from the local docker daemon.\* This could be implemented as a cleanup context manager in `manager/server.py` to spawn and shutdown a separate async task to monitor Docker daemon events (like in agent).
- The Docker daemon event fetch loop could be refactored and migrated from the agent to the common.

JIRA Issue: BA-270

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.