lablup / lablup/backend.ai

Refactor container registry: split external-resource access (Client) from DB operations

Open
#12,437 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

**Goal:** Split the container-registry subsystem so that external-resource access (registry HTTP APIs, the AWS SDK, the local Docker daemon) lives in dedicated Client classes, separate from DB operations.

**Context:** BaseContainerRegistry currently bundles a generic Docker Registry v2 HTTP client, per-implementation catalog/auth logic, and the DB image-commit (commit_rescan_result) in one hierarchy, and the whole thing is driven from the db_source layer. All 9 implementations (DockerHub, Docker v2, Harbor v1/v2, GitHub, GitLab, AWS ECR, OpenShift, Local) reach external resources. External access clusters into 4 archetypes:

- Docker Registry v2 HTTP — DockerRegistry_v2, DockerHub, GitHub, GitLab, OpenShift (shared manifest scanning; differ only in catalog endpoint and auth).
- Harbor REST — HarborRegistry_v1/v2 (Harbor's own API; v2 also performs a write/untag via DELETE).
- AWS ECR — boto3 (AWS SDK) for repository listing and the auth token, HTTP for manifests.
- Local Docker daemon — LocalRegistry (talks to the local daemon, not a remote registry).

### Work items

**1. Dedicated Client implementations:** Extract external-resource access (HTTP/SDK) into Client classes. Generalizing a shared HTTP client across the Docker Registry v2 family — plus per-archetype clients for Harbor, AWS ECR, and the local Docker daemon — is encouraged.

**2. Separate DB from external access at the repository layer:** Today both DB operations and external-resource access are handled together in the db_source layer. Move external access into the Client(s) and have the repository layer orchestrate them, keeping the DB image upsert (commit_rescan_result) on the repository/DB path.

**3. Deprecate the legacy hierarchy:** Once the Client + repository split is in place, deprecate BaseContainerRegistry and its implementations.

JIRA Issue: BA-6634

Contributor guide

Open the contributing guide

Research direction

Read the db_source layer and map BaseContainerRegistry, its nine implementations, and commit_rescan_result before changing the design. Trace the Docker Registry v2, Harbor, AWS ECR, and Local external-access paths. Done means external access is in dedicated Client classes, DB image upserts remain on the repository/DB path, and the legacy hierarchy is deprecated.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python
Domain
api, backend, cloud, database
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.