moby / moby/moby

docker build iidfile can't be used with process substitution / fd because it tries to remove it

Open
#46,571 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/0-triage
Dominant language
Go
Stars
72.1k
Forks
19.2k
Avg merge
1d 17h
Merged PRs (30d)
164

Description

Description

can't use --iidfile with process substitution or to write to stdout

$ docker build --iidfile=/dev/stdout . | xargs docker inspect
ERROR: removing image ID file: remove /dev/stdout: permission denied
$ docker build .  --iidfile=>(xargs docker inspect) 
ERROR: removing image ID file: remove /dev/fd/63: operation not permitted

can do

$ docker build . -q | xargs docker inspect --format '{{.Size}}'
116490258
Reproduce
echo 'FROM debian' > Dockerfile
docker build . --iidfile=>(docker image inspect)
$ docker build . --iidfile=>(docker image inspect)
"docker image inspect" requires at least 1 argument.
See 'docker image inspect --help'.

Usage:  docker image inspect [OPTIONS] IMAGE [IMAGE...]

Display detailed information on one or more images
ERROR: removing image ID file: remove /dev/fd/63: operation not permitted
Expected behavior

should work with fd

$ docker build . --iidfile=>(xargs docker inspect --format '{{.Size}}' )
sha256:db9749ac63b9c77f0325b0ca3358c50b28258b083f8c956f8d5ded66ba561818
116490258

should show build logs in stdout/stderr as well as print out size to std out

docker version
(base) vscode ➜ /workspaces/test-micromamba-features/test $ docker version
Client:
 Version:           23.0.6+azure-2
 API version:       1.41 (downgraded from 1.42)
 Go version:        go1.19.10
 Git commit:        ef23cbc4315ae76c744e02d687c09548ede461bd
 Built:             Thu May  4 10:51:27 UTC 2023
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          20.10.23
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.20.2
  Git commit:       v20.10.23
  Built:            Tue Jan  1 00:00:00 1980
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.0
  GitCommit:        v1.7.0
 runc:
  Version:          1.1.4
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:
docker info
(base) vscode ➜ /workspaces/test-micromamba-features/test $ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.11.2+azure-3
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.21.0-1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 42
  Running: 3
  Paused: 0
  Stopped: 39
 Images: 455
 Server Version: 20.10.23
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux nvidia runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: v1.7.0
 runc version: 
 init version: 
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 6.1.22
 Operating System: NixOS 23.05 (Stoat)
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 125.7GiB
 Name: reese
 ID: OEQH:PAN5:KRDU:G7IY:UZJP:FNH6:AZ2B:L3NG:NZ4R:QRVT:WFH4:JMJ2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: true
Additional Info

some possible fixes:

  1. don't delete if not regular file
  2. don't fail if can't delete
  3. always print iid to stdout (when pr was made build logs where sent to both stderr and stdout)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Docker CLI's docker build handling of the --iidfile option and reproduce the /dev/stdout and process-substitution commands from the issue. Trace how the image ID file is cleaned up after the build. Done means file-descriptor and stdout targets no longer cause a removal error while the image ID remains available to the consumer.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
build-system, cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.