push with zstd compression throws warnings: encountered unknown type

Open
#5,154 0 comments 9 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
docker, go
Domain
devops

Research direction

Start in util/push/push.go around the childless media-type handling described in the issue, then compare it with containerd's core/images/image.go functions. Reproduce with the provided docker buildx command and verify that pushing an OCI image with zstd-compressed layers no longer logs the unknown-type warning while existing attestation handling remains intact.

Written by the indexing model from the issue text.

Description

Pushing images with zstd compression throws warning in logs:

encountered unknown type application/vnd.oci.image.layer.v1.tar+zstd; children may not be fetched

Command to reproduce is:

docker buildx build --output type=image,oci-mediatypes=true,compression=zstd,compression-level=3,force-compression=true,push=true --tag kkopachev/test:latest .

This is happening because zstd image layer is not listed of one of childless options in https://github.com/moby/buildkit/blob/e38c064155c1dd9046a53f73f1429636be6c810b/util/push/push.go#L252-L260

I am not sure what was the reason to copy this function from containerd but since it diverted a bit with addition of in-toto attestation check for example.

I am happy to open PR, but since this is my first, contribution guide suggest opening an issue to get direction.

I am thinking easiest solution would be to adopt containerd's functions to check for media type. Basically replace switch with something like this:

if images.IsManifestType(desc.MediaType) { ...
} else if images.IsIndexType(desc.MediaType) { ...
} else if images.IsLayerType(desc.MediaType) || images.IsKnownConfig(desc.MediaType) ||
			desc.MediaType == intoto.PayloadType {
...
}

Or another approach I see could be to wrap containerd's Children function with pre and post-logic, so buildkit does not keep maintaining it's fork of such function. Something like this with a bit more checks:

if (desc.MediaType == intoto.PayloadType) {
  return nil, nil
}
return images.Children(.....)
Dominant language
Go
Stars
10.3k
Forks
1.5k
Avg merge
1d 23h
Merged PRs (30d)
48

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from moby/buildkit

All issues in moby/buildkit

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.