TARs with PaxHeader longer than 2 blocks is not handled
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 2.2k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 43
Description
Description
#1770 updated the code to handle 2 blocks instead of 1, since using git to tar would add additional headers longer than 1 block..
Right now, the code is here: https://github.com/docker/cli/blob/5bf109154fa97a8f6e7666b9d0e65e7578a7e935/cli/command/image/build/context.go#L107
However, tars generated using python may not work. Python 3.8 shutil.make_archive was used to generate the TAR below.
From the python docs
Changed in version 3.8: The modern pax (POSIX.1-2001) format is now used instead of the legacy GNU format for archives created with format="tar".
We have a service that accepts tarballs and sometimes have to untar them (we use similar code as docker/cli):
https://github.com/Azure/acr-builder/blob/fc51a5e39caece17df7f0db47c76b4cb849d6ee0/scan/context.go#L104
We recently ran into a problem reading the following tarball with extended paxheaders that went over more than 2 blocks:
././@PaxHeader0000000000000000000000000000003300000000000011451 xustar000000000000000027 mtime=1588632955.698403
./0000700000175100001660000000000000000000000011473 5ustar00vstsdocker00000000000000././@PaxHeader0000000000000000000000000000003300000000000011451 xustar000000000000000027 mtime=1588632955.698403
./msftkube-task.yaml0000644000175100001660000000264300000000000015156 0ustar00vstsdocker00000000000000# https://docs.microsoft.com/en-us/azure/container-registry/container-registry-tasks-reference-yaml
# https://docs.microsoft.com/en-us/azure/container-registry/container-registry-tasks-authentication-key-vault
version: v1.1.0
alias:
values:
TmpCertFile: ./cert.pfx
env:
- AZURE_CORE_COLLECT_TELEMETRY=false
- PYTHONUNBUFFERED=true
- AZURE_CLOUD_NAME={{.Values.cloudname}}
# $HOME and /workspace folders are shared between steps.
# /workspace is where Tasks download the remote context to and by default is current working folder for each step.
# even though this is implicitly set, we express that explicitly.
workingDirectory: /workspace
steps:
- cmd: 'az login --service-principal --username {{.Values.sp_client_id}} --password {{.Values.sp_client_secret}} --tenant {{.Values.sp_tenant_id}}'
- cmd: 'az account set --subscription {{.Values.subscription_id}}'
- cmd: 'az acr import --name $RegistryName --force --source viennadroptest.azurecr.io/parallelrunstep-agent:{{.Values.parallelrunstep_agent_ver}}
Steps to reproduce the issue:
- Can either use the tar above, or generate one using Python library.
Describe the results you received:
The code path:
https://github.com/docker/cli/blob/5bf109154fa97a8f6e7666b9d0e65e7578a7e935/cli/command/image/build/context.go#L179 will return false.
Describe the results you expected:
It should return true
Please let me know if you think using block size 2048 will solve the problem. If yes, I will send in a fix :)
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
(paste your output here)
Output of docker info:
(paste your output here)
Additional environment details (AWS, VirtualBox, physical, etc.):
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with cli/command/image/build/context.go, especially the archive detection logic around the referenced lines, and reproduce the issue with the supplied Python-generated TAR or Python shutil.make_archive. Done means an archive with a PaxHeader spanning more than two blocks is accepted and the code path returns true.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100