openshift / openshift/origin

some `oc image mirror` issues

Open
#19,769 16 comments 1 reaction 2 assignees View on GitHub

@bparees is already working on this.

Since May 18, 2018.

lifecycle/frozen
Dominant language
Go
Stars
8.7k
Forks
4.8k
Avg merge
4d 10h
Merged PRs (30d)
53

Description

  1. Not canonical repository name

According to the documentation:

# Copy image to S3 (pull from <bucket>.s3.amazonaws.com/image:latest)
oc image mirror myregistry.com/myimage:latest s3://s3.amazonaws.com/<region>/<bucket>/image:latest

Actual result:

$ oc image mirror docker.io/library/busybox:latest s3://s3.amazonaws.com/us-east-1/aos-agladkov-registry/myproject/toybox

error: unable to connect to s3.amazonaws.com/us-east-1/aos-agladkov-registry/myproject/toybox: repository name must be canonical
error: an error occurred during planning

It happens because this command uses ParseDockerImageReference to parse image reference, but this function requires that reference be canonical.

https://github.com/openshift/origin/blob/master/pkg/oc/cli/cmd/image/mirror/mappings.go#L29
https://github.com/openshift/origin/blob/master/pkg/oc/cli/cmd/image/mirror/mappings.go#L46

One way to make this link canonical is to specify a domain with a dot or add localhost:

$ oc image mirror docker.io/library/busybox:latest s3://s3.amazonaws.com/us-east-1/aos-agladkov-registry/example.com/myproject/toybox
$ oc image mirror docker.io/library/busybox:latest s3://s3.amazonaws.com/us-east-1/aos-agladkov-registry/localhost/myproject/toybox

But even if it turns out so to put something like that, the URL to the image will be:

aos-agladkov-registry.s3.amazonaws.com/example.com/myproject/toybox
aos-agladkov-registry.s3.amazonaws.com/localhost/myproject/toybox
  1. Unable to push manifest
$ oc image mirror docker.io/library/busybox:latest s3://s3.amazonaws.com/us-east-1/aos-agladkov-registry/example.com/myproject/toybox
...
phase 0:
  s3.amazonaws.com example.com/myproject/toybox blobs=16 mounts=0 manifests=1 shared=0

info: Planning completed in 2.93s
error: unable to upload blob sha256:4639278eb7ec424805d31508723f2868f2b3d26475c76f09f4aa87be6d55572c to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to upload blob sha256:b6467317087572599bdc9ff215788f7bf01aa201b0b377971901f1b4ca5c025d to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to upload blob sha256:03f1257c1e23796261321cc0b8b518b35e65a965f16547f087c9b7f89afc13b4 to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to upload blob sha256:1f4144adca0f4a32f35c8aeaa6e33a2470dbf226cd06cec72eae8754e071dd15 to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to upload blob sha256:45354500ca49c359a901800c06c9664e336d4652e6a7ed39b91e594f7c5d5f09 to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to upload blob sha256:f70adabe43c0cccffbae8785406d490e26855b8748fc982d14bc2b20c778b929 to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to push docker.io/library/busybox: failed to upload blob sha256:b8a401baa6b218ffa521c651eba0c876a7a211d840734b33c009c4d8af89cda1: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error: unable to upload blob sha256:52894eca45f5ddc393bb8211350d916060ddf791d480160f538465dfc76048b3 to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to upload blob sha256:3f46e1095de266d1af2b069542c947ebe417063bedd12d781dcd39c9f6759ed3 to s3.amazonaws.com/example.com/myproject/toybox: S3 target blob store requires blobs to have mount stats that include a digest
error: unable to push docker.io/library/busybox: failed to upload blob sha256:8ac48589692a53a9b8c2d1ceaa6b402665aa7fe667ba51ccc03002300856d8c7: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error: unable to push docker.io/library/busybox: failed to upload blob sha256:b62c88e2a3d11d887c7ca2bd57be02a7998d4255f349c02a39ee6cca408891e4: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error: unable to push docker.io/library/busybox: failed to upload blob sha256:1063bcd1bb2a382f78d888cf6c6f891c35123352e1c2950079da2f362086f5bb: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error: unable to push docker.io/library/busybox: failed to upload blob sha256:f688e3dede8cf452d4c06a3f91eab2cba12a2ffcbbe3467469aa89bbb3a93afd: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error: unable to push docker.io/library/busybox: failed to upload blob sha256:420f307579300330442e3cadbd50b43bfd8870ff79dc90be8492826ec20e412a: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error: unable to push docker.io/library/busybox: failed to upload blob sha256:3d9f9d95c0302dfaa04f09d22cb49ed6de296a190cf0ebbb582e9ff665c5ed3d: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
error: unable to push docker.io/library/busybox: failed to upload blob sha256:ba3a78826904c625e65a2eed1f247bbab59898f043490e7113e88907bf7c6b3b: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
info: Mirroring completed in 520ms (0B/s)
error: one or more errors occurred while uploading images

I did not find a way to fix it.

  1. S3 credentials

I found the only way to specify the login and password for S3: The S3 credentials must be encoded and written to the ~/.docker/config.json manually.

{                                                                                
    "auths": {                                                                    
       "s3.amazonaws.com": {                                                      
           "auth": "dXNlcm5hbWU6cGFzc3dvcmQ="                                      
       }                                                                          
    }                                                                             
}      

Is this really the only way?

  1. Docker defaults

When you are using a reference to dockerhub, there are a number of defaults that allow not to specify docker.io, latest as tag and library component in case the image is official. This works in the docker client, it works in other oc sub-commands, but does not work here. The error message does not reflect the essence of the problem:

$ oc image mirror docker.io/busybox:latest docker.io/foo

error: unable to retrieve source image docker.io/library/busybox by tag: errors:
denied: requested access to the resource is denied
unauthorized: authentication required

error: an error occurred during planning

Moreover, the message shows a normalized repository name, which is even more confusing.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.