Docker manifest push is slow can not operate in parallel
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 2.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
Description
The docker manifest push only deal with the metadata, but it is too slow.
In my server, this cost for about 30s to push the manifest which contain a X86 arch and an ARM arch.
and when I push three push manifest in parallel, it cost about 2min. not ~30s as expected.
Reproduce
docker manifest create --insecure harbor.xxx.local/x:v1 harbor.xxx.local/x_x86:v1 harbor.xxx.local/x_armv8:v1
docker manifest push --insecure harbor.xxx.local/x:v1
Expected behavior
expect the push can run in parallel
docker version
Client:
Cloud integration: v1.0.28
Version: 20.10.17
API version: 1.41
Go version: go1.17.11
Git commit: 100c701
Built: Mon Jun 6 23:04:45 2022
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Desktop 4.11.1 (84025)
Engine:
Version: 20.10.17
API version: 1.41 (minimum version 1.12)
Go version: go1.17.11
Git commit: a89b842
Built: Mon Jun 6 23:01:23 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.6
GitCommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
runc:
Version: 1.1.2
GitCommit: v1.1.2-0-ga916309
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.8.2)
compose: Docker Compose (Docker Inc., v2.7.0)
extension: Manages Docker extensions (Docker Inc., v0.2.8)
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
scan: Docker Scan (Docker Inc., v0.17.0)
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 20.10.17
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
runc version: v1.1.2-0-ga916309
init version: de40ad0
Security Options:
seccomp
Profile: default
cgroupns
Kernel Version: 5.10.104-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 5.805GiB
Name: docker-desktop
ID: 2VO2:ARRA:QFIB:UJQJ:ESEG:3YTI:WLMF:YW3C:Q2ZA:M7RN:LU2Q:D6LG
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
harbor.xxx.local
hubproxy.docker.internal:5000
registry.xxx.local
127.0.0.0/8
Registry Mirrors:
http://hub-mirror.c.163.com/
https://reg-mirror.qiniu.com/
Live Restore Enabled: false
Additional Info
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the documented docker manifest create and docker manifest push commands with one manifest and then three concurrent pushes. Trace the manifest-push entry point to identify where operations are serialized; done means independent pushes proceed concurrently without the reported roughly two-minute total delay.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100