containerd / containerd/nerdctl
`nerdctl push` interop issue with Harbor registry
- Dominant language
- Go
- Stars
- 10.4k
- Forks
- 826
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 44
Description
### Description
Hi, I'm trying `nerdctl push` on a private Harbor registry with Bearer authentication and a custom token service, but the command fails with `401 Unauthorized`. After further investigation, it seems to be an interoperability issue between nerdctl and Harbor registries that have disabled basic authentication.
When nerdctl requests `GET /v2//blobs/`, the registry returns a `WWW-Authenticate` header that asks the client to request for a Bearer token with the scope of `repository::pull` from the specified `realm`. nerdctl would faithfully do so and retry the request with the right credentials, so far so good.
But when nerdctl reuses the same token for `POST /v2//blobs/uploads/` requests, the scope required is `repository::pull,push`, so this request is denied and the registry returns `401 Unauthorized`. The problem is that when a request contains a `Authorization` header but failed to authenticate, Harbor uses `Basic realm="harbor"` as the authentication challenge[[ref]](https://github.com/goharbor/harbor/blob/6113469a567623054a58d0f5506b90a55adca7ff/src/server/middleware/v2auth/auth.go#L95-L98), instead of the token service configured.
The registry I'm accessing exclusively relies on Bearer tokens for authentication and has been configured to deny all requests with basic auth. Being mislead by the new `WWW-Authenticate` header, all subsequent requests made by nerdctl would fail.
To solve this interop issue, nerdctl could do one of the following:
1. Make a "preflight" request to the `GET /v2/` API Version Check endpoint to retrieve a token without scope constraint and cache it for future requests.
2. Retry the request without an `Authorization` header. In this case, Harbor would return the correct `WWW-Authenticate` header that points to the token service.
On Harbor's side, there is https://github.com/goharbor/harbor/issues/17930, but no decision has been made yet.
### Steps to reproduce the issue
1. `nerdctl login ...`
2. `nerdctl push `
### Describe the results you received and expected
Received: `unexpected status from POST request to .../blobs/uploads/: 401 Unauthorized`
Expected: Command succeed without errors.
### What version of nerdctl are you using?
```
Client:
Version: v1.0.0
OS/Arch: linux/amd64
Git commit: c00780a1f5b905b09812722459c54936c9e070e6
buildctl:
Version: v0.10.5
GitCommit: bc26045116045516ff2427201abd299043eaf8f7
Server:
containerd:
Version: v1.6.8
GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc:
Version: 1.1.4
GitCommit: 5fd4c4d144137e991c4acebb2146ab1483a97925
```
### Are you using a variant of nerdctl? (e.g., Rancher Desktop)
Rancher Desktop for macOS
### Host information
```
Client:
Namespace: default
Debug Mode: false
Server:
Server Version: v1.6.8
Storage Driver: overlayfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Log: fluentd journald json-file syslog
Storage: native overlayfs
Security Options:
seccomp
Profile: default
Kernel Version: 5.15.78-0-virt
Operating System: Alpine Linux v3.16
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 5.809GiB
Name: lima-rancher-desktop
ID: 6238c1b1-ca3b-4e66-938c-d52f6c1895f5
WARNING: IPv4 forwarding is disabled
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.