containerd / containerd/nerdctl

Converting image present in snapshotter fails

Open
#3,764 33 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
10.4k
Forks
826
Avg merge
1d 23h
Merged PRs (30d)
44

Description

### Description

Most likely related to https://github.com/containerd/nerdctl/pull/3435.

When using the GKE [image streaming snapshotter](https://cloud.google.com/kubernetes-engine/docs/how-to/image-streaming), converting an image after we commit it locally from a running container (e.g. with docker), the converter tries to pull the image from remote which then fails (because we haven't pushed it).

The image is present in the snapshotter (as we can run it) but apparently not in the content store.

### Steps to reproduce the issue

```
> docker run --name foo alpine
> docker commit foo gcr.io/$repo/cbruckmayer:foo

# I can use the image with nerdctl
> nerdctl --address /var/run/docker/containerd/containerd.sock --namespace moby --snapshotter gcfs run -it --network none gcr.io/$repo/cbruckmayer:foo

# However, when trying to convert the image, it will try to pull it from remote which fails with not found

> nerdctl --address /var/run/docker/containerd/containerd.sock --namespace moby --snapshotter gcfs image convert --zstd --oci gcr.io/$repo/cbruckmayer:foo gcr.io/$repo/cbruckmayer:foo-zstd
gcr.io/$repo/cbruckmayer:foo: resolving |--------------------------------------|
elapsed: 0.1 s total: 0.0 B (0.0 B/s)
INFO[0000] fetch failed after status: 404 Not Found host=gcr.io
FATA[0000] failed to resolve reference "gcr.io/$repo/cbruckmayer:foo": gcr.io/$repo/cbruckmayer:foo: not found
root@docker-daemon-rzhtc:/app# nerdctl --debug-full --address /var/run/docker/containerd/containerd.sock --namespace moby --snapshotter gcfs image convert --zstd --oci gcr.io/$repo/cbruckmayer:foo gcr.io/$repo/cbruckmayer:foo-zstd
DEBU[0000] fetching image="gcr.io/$repo/cbruckmayer:foo"
DEBU[0000] resolving host=gcr.io
DEBU[0000] do request host=gcr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=containerd/2.0.0+unknown request.method=HEAD url="https://gcr.io/v2/$repo/cbruckmayer/manifests/foo"
DEBU[0000] fetch response received host=gcr.io response.header.accept-ranges=none response.header.content-type=application/json response.header.date="Fri, 13 Dec 2024 13:10:25 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.server="Docker Registry" response.header.vary=Accept-Encoding response.header.www-authenticate="Bearer realm=\"https://gcr.io/v2/token\",service=\"gcr.io\",scope=\"repository:$repo/cbruckmayer:pull\"" response.header.x-frame-options=SAMEORIGIN response.header.x-xss-protection=0 response.status="401 Unauthorized" url="https://gcr.io/v2/$repo/cbruckmayer/manifests/foo"
DEBU[0000] Unauthorized header="Bearer realm=\"https://gcr.io/v2/token\",service=\"gcr.io\",scope=\"repository:$repo/cbruckmayer:pull\"" host=gcr.io
DEBU[0000] do request host=gcr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=containerd/2.0.0+unknown request.method=HEAD url="https://gcr.io/v2/$repo/cbruckmayer/manifests/foo"
DEBU[0000] fetch response received host=gcr.io response.header.accept-ranges=none response.header.content-type=application/json response.header.date="Fri, 13 Dec 2024 13:10:25 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.server="Docker Registry" response.header.vary=Accept-Encoding response.header.x-frame-options=SAMEORIGIN response.header.x-xss-protection=0 response.status="404 Not Found" url="https://gcr.io/v2/$repo/cbruckmayer/manifests/foo"
INFO[0000] fetch failed after status: 404 Not Found host=gcr.io
FATA[0000] failed to resolve reference "gcr.io/$repo/cbruckmayer:foo": gcr.io/shopify-docker-images/cbruckmayer:foo: not found
```

### Describe the results you received and expected

I expect to be able to convert (tagging, committing, saving) the image.

### What version of nerdctl are you using?

```
> nerdctl --address /var/run/docker/containerd/containerd.sock --namespace moby version
Client:
Version: v2.0.2
OS/Arch: linux/amd64
Git commit: 1220ce7ec2701d485a9b1beeea63dae3da134fb5
buildctl:
Version:

Server:
containerd:
Version: 1.7.24
GitCommit: 88bf19b2105c8b17560993bee28a01ddc2f97182
runc:
Version: 1.2.2
GitCommit: v1.2.2-0-g7cb3632
```

### Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

### Host information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.