docker / docker/cli

TARs with PaxHeader longer than 2 blocks is not handled

Open
#2,525 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/builder version/19.03
Dominant language
Go
Stars
6.1k
Forks
2.2k
Avg merge
1d 15h
Merged PRs (30d)
43

Description

Description

#1770 updated the code to handle 2 blocks instead of 1, since using git to tar would add additional headers longer than 1 block..

Right now, the code is here: https://github.com/docker/cli/blob/5bf109154fa97a8f6e7666b9d0e65e7578a7e935/cli/command/image/build/context.go#L107

However, tars generated using python may not work. Python 3.8 shutil.make_archive was used to generate the TAR below.

From the python docs

Changed in version 3.8: The modern pax (POSIX.1-2001) format is now used instead of the legacy GNU format for archives created with format="tar".

We have a service that accepts tarballs and sometimes have to untar them (we use similar code as docker/cli):
https://github.com/Azure/acr-builder/blob/fc51a5e39caece17df7f0db47c76b4cb849d6ee0/scan/context.go#L104

We recently ran into a problem reading the following tarball with extended paxheaders that went over more than 2 blocks:

././@PaxHeader0000000000000000000000000000003300000000000011451 xustar000000000000000027 mtime=1588632955.698403
./0000700000175100001660000000000000000000000011473 5ustar00vstsdocker00000000000000././@PaxHeader0000000000000000000000000000003300000000000011451 xustar000000000000000027 mtime=1588632955.698403
./msftkube-task.yaml0000644000175100001660000000264300000000000015156 0ustar00vstsdocker00000000000000# https://docs.microsoft.com/en-us/azure/container-registry/container-registry-tasks-reference-yaml
# https://docs.microsoft.com/en-us/azure/container-registry/container-registry-tasks-authentication-key-vault
version: v1.1.0
alias:
  values:
    TmpCertFile: ./cert.pfx
env:
  - AZURE_CORE_COLLECT_TELEMETRY=false
  - PYTHONUNBUFFERED=true
  - AZURE_CLOUD_NAME={{.Values.cloudname}}
# $HOME and /workspace folders are shared between steps.
# /workspace is where Tasks download the remote context to and by default is current working folder for each step.
# even though this is implicitly set, we express that explicitly.
workingDirectory: /workspace
steps:
  - cmd: 'az login --service-principal --username {{.Values.sp_client_id}} --password {{.Values.sp_client_secret}} --tenant {{.Values.sp_tenant_id}}'
  - cmd: 'az account set --subscription {{.Values.subscription_id}}'
  - cmd: 'az acr import --name $RegistryName --force --source viennadroptest.azurecr.io/parallelrunstep-agent:{{.Values.parallelrunstep_agent_ver}}

Steps to reproduce the issue:

  1. Can either use the tar above, or generate one using Python library.

Describe the results you received:
The code path:
https://github.com/docker/cli/blob/5bf109154fa97a8f6e7666b9d0e65e7578a7e935/cli/command/image/build/context.go#L179 will return false.

Describe the results you expected:
It should return true

Please let me know if you think using block size 2048 will solve the problem. If yes, I will send in a fix :)

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

(paste your output here)

Output of docker info:

(paste your output here)

Additional environment details (AWS, VirtualBox, physical, etc.):

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cli/command/image/build/context.go, especially the archive detection logic around the referenced lines, and reproduce the issue with the supplied Python-generated TAR or Python shutil.make_archive. Done means an archive with a PaxHeader spanning more than two blocks is accepted and the code path returns true.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.