COPY commands in multi-stage builds create bigger images than expected
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 72.1k
- Forks
- 19.2k
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 164
Description
Description
The result of building an image with multi-stage and several COPY commands is a bigger image than expected. Having several COPY commands, each affecting different files, should be essentially the same as a having a single COPY layer as explained here. This means that there should be no difference if you build with --squash since the layers don't overlap.
However this is not the behavior I found when copying several directories from one stage to the other. The image is bigger without squashing, even if the files don't overlap. Considering the main devs are interested in removing squash in favor of multi-stage I don't think squash should have any benefit in this particular scenario.
Steps to reproduce the issue:
- Build the following Dockerfile without
--squash:
FROM ubuntu:latest AS firststage
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y vim
FROM ubuntu:latest
COPY --from=firststage /usr/ /usr
COPY --from=firststage /etc/ /etc
- Build the Dockerfile again now adding
--squash
Describe the results you received:
Output of docker image ls:
REPOSITORY TAG IMAGE ID CREATED SIZE
germanrodriguez/tests weird-copy-squashed a090a8c6f428 8 minutes ago 136MB
germanrodriguez/tests weird-copy 051062522982 8 minutes ago 210MB
Describe the results you expected:
I expected both images to have the same size since the two COPY commands don't overlap with any files.
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client: Docker Engine - Community
Version: 20.10.14
API version: 1.41
Go version: go1.16.15
Git commit: a224086
Built: Thu Mar 24 01:48:02 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.14
API version: 1.41 (minimum version 1.12)
Go version: go1.16.15
Git commit: 87a90dc
Built: Thu Mar 24 01:45:53 2022
OS/Arch: linux/amd64
Experimental: true
containerd:
Version: 1.5.11
GitCommit: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc:
Version: 1.0.3
GitCommit: v1.0.3-0-gf46b6ba
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.8.1-docker)
scan: Docker Scan (Docker Inc., v0.17.0)
Server:
Containers: 13
Running: 0
Paused: 0
Stopped: 13
Images: 125
Server Version: 20.10.14
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: pptyhmpe6odz2k7puo0zatu17
Is Manager: true
ClusterID: jnlj97rvm7v5pfts6fce1lfzt
Managers: 1
Nodes: 1
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 10.240.10.15
Manager Addresses:
10.240.10.15:2377
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux nvidia
Default Runtime: runc
Init Binary: docker-init
containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc version: v1.0.3-0-gf46b6ba
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.13.0-40-generic
Operating System: Ubuntu 20.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 15.38GiB
Name: iiticublap258
ID: H6CT:3V23:UUXS:XUV3:IW25:XS3K:4WIH:YBSU:6S44:BR45:6DV5:GSBB
Docker Root Dir: /var/lib/docker
Debug Mode: false
Username: germanrodriguez
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.):
Native linux computer running Ubuntu 20.04
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the provided multi-stage Dockerfile with and without --squash, comparing the resulting image sizes. Then trace the Docker build handling for multiple COPY --from commands; done means non-overlapping copies produce the expected layer or image size, with regression coverage for this reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- build-system, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100