lablup / lablup/backend.ai

Improve handling of multiple concurrent docker image pulls in agents

Open
#898 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

Docker's image pull API works like a singletone queue inside the docker daemon, which means that:

- Regardless how many docker API clients request the pull of a single image, the pull operation occurs only once and all clients are notified with the same progress.
- If there are multiple pull requests for different images, they are queued and processed serially by the docker daemon.
- Once started, the image pull operation inside the Docker daemon is _not interruptible_ unless we force-terminate the entire docker service.
It seems that our agent (and maybe the manager) gets stuck when there are multiple concurrent image pulls. Let's investigate what's happening and improve the handling and progress reports.

Related issues:

- #227
- #381

JIRA Issue: BA-232

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.