moby / moby/moby

COPY commands in multi-stage builds create bigger images than expected

Open
#43,614 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/builder
Dominant language
Go
Stars
72.1k
Forks
19.2k
Avg merge
1d 17h
Merged PRs (30d)
164

Description

Description

The result of building an image with multi-stage and several COPY commands is a bigger image than expected. Having several COPY commands, each affecting different files, should be essentially the same as a having a single COPY layer as explained here. This means that there should be no difference if you build with --squash since the layers don't overlap.

However this is not the behavior I found when copying several directories from one stage to the other. The image is bigger without squashing, even if the files don't overlap. Considering the main devs are interested in removing squash in favor of multi-stage I don't think squash should have any benefit in this particular scenario.

Steps to reproduce the issue:

  1. Build the following Dockerfile without --squash:
FROM ubuntu:latest AS firststage

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y vim 

FROM ubuntu:latest

COPY --from=firststage /usr/ /usr

COPY --from=firststage /etc/ /etc
  1. Build the Dockerfile again now adding --squash

Describe the results you received:
Output of docker image ls:

REPOSITORY                    TAG                               IMAGE ID       CREATED          SIZE
germanrodriguez/tests         weird-copy-squashed               a090a8c6f428   8 minutes ago    136MB
germanrodriguez/tests         weird-copy                        051062522982   8 minutes ago    210MB

Describe the results you expected:
I expected both images to have the same size since the two COPY commands don't overlap with any files.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client: Docker Engine - Community
 Version:           20.10.14
 API version:       1.41
 Go version:        go1.16.15
 Git commit:        a224086
 Built:             Thu Mar 24 01:48:02 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.14
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.15
  Git commit:       87a90dc
  Built:            Thu Mar 24 01:45:53 2022
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.5.11
  GitCommit:        3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc:
  Version:          1.0.3
  GitCommit:        v1.0.3-0-gf46b6ba
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.1-docker)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 13
  Running: 0
  Paused: 0
  Stopped: 13
 Images: 125
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: pptyhmpe6odz2k7puo0zatu17
  Is Manager: true
  ClusterID: jnlj97rvm7v5pfts6fce1lfzt
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8  
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 10.240.10.15
  Manager Addresses:
   10.240.10.15:2377
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux nvidia
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.0.3-0-gf46b6ba
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.13.0-40-generic
 Operating System: Ubuntu 20.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.38GiB
 Name: iiticublap258
 ID: H6CT:3V23:UUXS:XUV3:IW25:XS3K:4WIH:YBSU:6S44:BR45:6DV5:GSBB
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: germanrodriguez
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):
Native linux computer running Ubuntu 20.04

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the provided multi-stage Dockerfile with and without --squash, comparing the resulting image sizes. Then trace the Docker build handling for multiple COPY --from commands; done means non-overlapping copies produce the expected layer or image size, with regression coverage for this reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
build-system, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.