lablup / lablup/backend.ai

Reconcile existing `images` table records when adding a new container registry to prevent duplicates

Open
#3,018 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

This is a follow-up to #1907 (its #1917 and related stack PRs).

After etcd-to-postgres container registry migration, all existing image records belong to either:

- a "local" registry when there is no corresponding container registry config, or
- a designated container registry instance migrated from the etcd config.
What if the admin adds a new container registry that has the already-migrated image records when rescanned?

For example:![https://github.com/user-attachments/assets/e776b223-c176-455d-b33b-5eef82928e1a](https://github.com/user-attachments/assets/e776b223-c176-455d-b33b-5eef82928e1a)`index.docker.io/nvcr.io/nvidia/nvhpc:24.3-runtime-cuda12.3-ubuntu22.04` is actually not an `index.docker.io` image but `nvcr.io`'s image. This result is because there is no matching `nvcr.io` container registry configuration and this record has been migrated to become a "local" image.Later, the admin adds the `nvcr.io` container registry configuration. Then, this entry should be _converted_ to use _that_ container registry instance.

There could be two ways to reconcile the potentially duplicate entries (existing vs. rescanned) in the database:

- (Option 1) When adding a new container registry record, enumerate all _local_ `images` records and update any record matching with the new registry's hostname and project.
- (Option 2) When rescanning after adding the container registry, check if there are any duplicate record matching with the new image item based on the reference string value.
- When such reconciliation happens, it should be explicitly logged in the console output and the log stream.
- Let's expose this reconciliation procedure as an admin CLI command so that we could rerun it when needed.

JIRA Issue: BA-66

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.