'not found' for files with an absolute path over Windows max path length
Open
Nobody has claimed this yet.
area/builder/buildkit
platform/windows
version/20.10
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 2.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
Description
Steps to reproduce the issue:
- Clone https://github.com/NickDarvey/docker-cli-long-path-issue
- Run
docker build . --progress=plain --no-cache - Observe output 'a'
- Run
$env:DOCKER_BUILDKIT=0 - Run
docker build . --progress=plain --no-cache - Observe output 'b'
Describe the results you received:
Both builds complete successfully.
Describe the results you expected:
Build 'a' (with Buildkit enabled) fails with two files not found.
#7 ERROR: "/files/aaa[...]" not found: not found
#1 [internal] load build definition from Dockerfile
#1 sha256:14249b93010c4299011446a98437fd2845458441ab49355bf66709890f296b4a
#1 transferring dockerfile: 32B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:6288d866660b4ca6e77fa9d287980797b3740a85b5c9232d2b2d7e137c4f8a7e
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/node:14
#3 sha256:33cb2214dbe26f023b39a81571d3d32bc2f316813e650e050a1f3f54f4f80bfe
#3 DONE 0.0s
#8 [1/4] FROM docker.io/library/node:14
#8 sha256:ef27ac36e3258faee3dc1dfd79de989375da300c240c7dfab02d19ae49b69aa9
#8 CACHED
#4 [internal] load build context
#4 sha256:407c43aebb727cd875cd94bba3a91fee423feb4a580896efb94d48bd66c0737d
#4 transferring context: 2.58kB done
#4 DONE 0.0s
#5 [2/4] COPY files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaa.txt .
#5 sha256:14b62ba961eb7c5bd6ed02733a61e682e7f50b62b3c917910ca881c28ce37f4e
#5 CACHED
#6 [3/4] COPY files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/bbbb.txt .
#6 sha256:d34b437938142ddbac46871e16466dfe925584e086db5fb909f5c03bf071b320
#6 CACHED
#7 [4/4] COPY files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/ccccc.txt .
#7 sha256:e53fb275a58d5d201831cbc1b01419e77ae5bd183b588a299f4c5ae13da446bc
#7 ERROR: "/files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/ccccc.txt" not found: not found
------
> [4/4] COPY files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/ccccc.txt .:
------
failed to compute cache key: "/files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/ccccc.txt" not found: not found
Additional information you deem important (e.g. issue happens only occasionally):
This only occurs with absolute file path length is greater than 260 chars. i.e.
This is copied (with 260 chars):
C:/s/a/files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/bbbb.txt
This fails (with 261 chars):
C:/s/a/files/aaaaaaaaaaaaaaaaaaaaaaa/aaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaa/aaaaaaaaaaaaaaaa/aaaaaaaaaa/aaaaaaaaaaaa/aaaaaaaaaaaaaaaaaaaa/aaaaaaaaaaaaaaaa/ccccc.txt
File system activity for 'ccccc.txt' when $env:DOCKER_BUILDKIT=0 and docker build is invoked:
11:20:44.0605083 com.docker.cli.exe 30560 CreateFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
11:20:44.0605245 com.docker.cli.exe 30560 QueryNetworkOpenInformationFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS CreationTime: 2021-08-11 10:29:56, LastAccessTime: 2021-08-11 11:20:26, LastWriteTime: 2021-08-11 10:31:44, ChangeTime: 2021-08-11 10:31:44, AllocationSize: 1601-01-01 10:00:00, EndOfFile: 1601-01-01 10:00:00, FileAttributes: A
11:20:44.0605314 com.docker.cli.exe 30560 CloseFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS
11:20:44.0606483 com.docker.cli.exe 30560 CreateFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: R, ShareMode: Read, Write, AllocationSize: n/a, OpenResult: Opened
11:20:44.0606764 com.docker.cli.exe 30560 CloseFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS
11:20:44.9439440 com.docker.cli.exe 30560 CreateFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
11:20:44.9439621 com.docker.cli.exe 30560 QueryNetworkOpenInformationFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS CreationTime: 2021-08-11 10:29:56, LastAccessTime: 2021-08-11 11:20:26, LastWriteTime: 2021-08-11 10:31:44, ChangeTime: 2021-08-11 10:31:44, AllocationSize: 1601-01-01 10:00:00, EndOfFile: 1601-01-01 10:00:00, FileAttributes: A
11:20:44.9439685 com.docker.cli.exe 30560 CloseFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS
11:20:44.9441016 com.docker.cli.exe 30560 CreateFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
11:20:44.9441148 com.docker.cli.exe 30560 QueryNetworkOpenInformationFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS CreationTime: 2021-08-11 10:29:56, LastAccessTime: 2021-08-11 11:20:26, LastWriteTime: 2021-08-11 10:31:44, ChangeTime: 2021-08-11 10:31:44, AllocationSize: 1601-01-01 10:00:00, EndOfFile: 1601-01-01 10:00:00, FileAttributes: A
11:20:44.9441205 com.docker.cli.exe 30560 CloseFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS
11:20:44.9442170 com.docker.cli.exe 30560 CreateFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS Desired Access: Generic Read, Disposition: Open, Options: Sequential Access, Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Write, AllocationSize: n/a, OpenResult: Opened
11:20:44.9442480 com.docker.cli.exe 30560 ReadFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS Offset: 0, Length: 1, Priority: Normal
11:20:44.9442660 com.docker.cli.exe 30560 ReadFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt END OF FILE Offset: 1, Length: 32,767
11:20:44.9442761 com.docker.cli.exe 30560 CloseFile C:\s\a\files\aaaaaaaaaaaaaaaaaaaaaaa\aaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaa\aaaaaaaaaaaaaaaa\aaaaaaaaaa\aaaaaaaaaaaa\aaaaaaaaaaaaaaaaaaaa\aaaaaaaaaaaaaaaa\ccccc.txt SUCCESS
There is no file system activity for 'ccccc.txt' when $env:DOCKER_BUILDKIT=1 and docker build is invoked.
Output of docker version:
Client:
Cloud integration: 1.0.17
Version: 20.10.7
API version: 1.41
Go version: go1.16.4
Git commit: f0df350
Built: Wed Jun 2 12:00:56 2021
OS/Arch: windows/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: b0f5bc3
Built: Wed Jun 2 11:54:58 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.6
GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc:
Version: 1.0.0-rc95
GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
compose: Docker Compose (Docker Inc., v2.0.0-beta.6)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 9
Running: 1
Paused: 0
Stopped: 8
Images: 82
Server Version: 20.10.7
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.16.3-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 12.16GiB
Name: docker-desktop
ID: KIPN:OW2F:KK2O:MUQA:AR4O:YWK5:5WAM:3JDH:PENE:OOSD:2MUW:VP2Z
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
Additional environment details (AWS, VirtualBox, physical, etc.):
Long Paths on Windows is enabled in my environment and the machine was restarted after this configuration was changed. (HKLM\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled=1)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the linked docker-cli-long-path-issue repository and the two docker build . --progress=plain --no-cache runs, comparing BuildKit enabled with DOCKER_BUILDKIT=0. Trace the Windows path handling used by docker build and verify the 260-character path succeeds while the 261-character path no longer reports the existing file as missing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- build-system, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100