docker / docker/cli

Docker build fails with confusing ZFS error message

Open
#5,775 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/0-triage
Dominant language
Go
Stars
6.1k
Forks
2.2k
Avg merge
1d 15h
Merged PRs (30d)
43

Description

Description

Update: This might have been triggered by being ZFS degraded, however it continued after resilver. Doing a prune -a fixed it.

Docker could do a better job of suggesting solutions in the error message rather than just passing through ZFS errors.


Running docker build with images on a ZFS file system that is degraded generate a confusing error:

ERROR: failed to solve: failed to prepare sha256:a790f937a6aea2600982de54a5fb995c681dd74f26968d6b74286e06839e4fb3 as i168knaqlfwod6v32797svyoh: exit status 2: "/usr/sbin/zfs snapshot data/d4cc76937f95999fda2d4dcbc4fcdca294e73d7f6202e62d62962ba9ce650287@913636948" => cannot open 'data/d4cc76937f95999fda2d4dcbc4fcdca294e73d7f6202e62d62962ba9ce650287': dataset does not exist
usage:
	snapshot [-r] [-o property=value] ... <filesystem|volume>@<snap> ...

For the property list, run: zfs set|get

For the delegated permission list, run: zfs allow|unallow
Reproduce

docker-build.out.txt
userApps.dockerfile.txt
zpool.stat.txt

Expected behavior

docker should be able to build a snapsnot even when ZFS file system is degraged.

If it can't for some reason, it should at least give a useful error message

thanks!

docker version
% docker version                                                                               
Client: Docker Engine - Community                                                                            
 Version:           27.4.1                                                                                   
 API version:       1.47                                                                                     
 Go version:        go1.22.10                                                                                
 Git commit:        b9d17ea                                                                                  
 Built:             Tue Dec 17 15:47:23 2024                                                                 
 OS/Arch:           linux/amd64                                                                              
 Context:           default                                                                                  
                                                                                                             
Server: Docker Engine - Community                                                                            
 Engine:                                                                                                     
  Version:          27.4.1                                                                                   
  API version:      1.47 (minimum version 1.24)                                                              
  Go version:       go1.22.10                                                                                
  Git commit:       c710b88                                                                                  
  Built:            Tue Dec 17 15:45:36 2024                                                                 
  OS/Arch:          linux/amd64                                                                              
  Experimental:     false                                                                                    
 containerd:                                                                                                 
  Version:          1.7.24                                                                                   
  GitCommit:        88bf19b2105c8b17560993bee28a01ddc2f97182                                                 
 runc:                                                                                                       
  Version:          1.2.2                                                                                    
  GitCommit:        v1.2.2-0-g7cb3632                                                                        
 docker-init:                                                                                                
  Version:          0.19.0                                                                                   
  GitCommit:        de40ad0
docker info
% docker info                                                                                  
Client: Docker Engine - Community                                                                            
 Version:    27.4.1                                                                                          
 Context:    default                                                                                         
 Debug Mode: false                                                                                           
 Plugins:                                                                                                    
  buildx: Docker Buildx (Docker Inc.)                                                                        
    Version:  v0.19.3                                                                                        
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx                                                  
  compose: Docker Compose (Docker Inc.)                                                                      
    Version:  v2.32.1                                                                                        
    Path:     /usr/libexec/docker/cli-plugins/docker-compose                                                 
                                                                                                             
Server:                                                                                                      
 Containers: 23                                                                                              
  Running: 0                                                                                                 
  Paused: 0                                                                                                  
  Stopped: 23                                                                                                
 Images: 99                                                                                                  
 Server Version: 27.4.1                                                                                      
 Storage Driver: zfs                                                                                         
  Zpool: data                                                                                                
  Zpool Health: DEGRADED                                                                                     
  Parent Dataset: data                                                                                       
  Space Used By Parent: 40241145674690                                                                       
  Space Available: 48629411644478                                                                            
  Parent Quota: no                                                                                           
  Compression: off                                                                                           
 Logging Driver: json-file                                                                                   
 Cgroup Driver: systemd                                                                                      
 Cgroup Version: 2                                                                                           
 Plugins:                                                                                                    
  Volume: local                                                                                              
  Network: bridge host ipvlan macvlan null overlay                                                           
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive                                                                                             
 Runtimes: io.containerd.runc.v2 runc                                                                        
 Default Runtime: runc                                                                                       
 Init Binary: docker-init                                                                                    
 containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182                                                
 runc version: v1.2.2-0-g7cb3632                                                                             
 init version: de40ad0                                                                                       
 Security Options:                                                                                           
  seccomp                                                                                                    
   Profile: builtin                                                                                          
  cgroupns                                                                                                   
 Kernel Version: 5.14.0-503.19.1.el9_5.x86_64                                                                
 Operating System: Rocky Linux 9.5 (Blue Onyx)                                                               
 OSType: linux                                                                                               
 Architecture: x86_64                                                                                        
 CPUs: 192                                                                                                   
 Total Memory: 3.937TiB                                                                                      
 Name: hgwdev                                                                                                
 ID: 102c1e67-7468-423e-8dd6-7cc2775e3a50                                                                    
 Docker Root Dir: /data/docker                                                                               
 Debug Mode: false                                                                                           
 Experimental: false                                                                                         
 Insecure Registries:                                                                                        
  127.0.0.0/8                                                                                                
 Live Restore Enabled: false
Additional Info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the docker build failure using the attached Dockerfile and build output on a degraded ZFS pool, then trace the Docker CLI build error path that surfaces the zfs snapshot failure. Check whether the expected result is successful snapshot creation or a more actionable error message; done means the behavior is covered for this case.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
cli, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.