dependabot / dependabot/dependabot-core
Docker registry base URL is not detected correctly when the URL includes a `@path`
- Dominant language
- Ruby
- Stars
- 5.8k
- Forks
- 1.5k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 149
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Package ecosystem
Docker
### What you expected to see, versus what you actually saw
This issue is very similar to #6804 but for docker
**Expected:**
The registry base URL is detected correctly and it includes the right base URL. In this example
https://registry.myCompany.com/dockerproxy/
**Actual**
It is assumed that the base URL for the docker API endpoint is `https://hostname/v2`
For this example, it is resolved to https://registry.myCompany.com/ incorrectly
### Native package manager behavior
Running docker pull from `https://registry.myCompany.com/dockerproxy` works as expected
### Smallest manifest that reproduces the issue
```
FROM registry.myCompany.com/dockerproxy/azul/zulu-openjdk@sha256:59fe77a0432aa1393d3b8c8d0cf39b551c2516e899508fa809346b1fe9966d13 as zulu-amd64
```
### Additional context
After investigating, this issue can be divided in two parts:
1. How we parse the `registry_hostname` as we assume that the hostname is the only information needed (and what we pass to the client)
Reproducer : https://github.com/yeikel/dependabot-core/commit/4c4cfbae5e584a114516f370ba036d112994318e
I am not sure how we can do without **brute forcing** and checking every URI and see if it resolves to a docker API instance.
For example, in this example, all these urls could be the base path in theory as `dockerproxy` could be the name of an organization.
Bruce force attempts :
```shell
HEAD registry.myCompany.com/dockerproxy/v2/
HEAD registry.myCompany.com/v2/
HEAD registry.myCompany.com/azul/v2
```
As the header `docker-distribution-api-version` should be part of the response [according to the SPEC](https://docs.docker.com/registry/spec/api/)
> When a 200 OK or 401 Unauthorized response is returned, the “Docker-Distribution-API-Version” header should be set to “registry/2.0”. Clients may require this header value to determine if the endpoint serves this API. When this header is omitted, clients may fallback to an older API version.
For example :
> curl --location --head https://cloud.canister.io:5000/v2/
> HTTP/1.1 401 Unauthorized
> content-type: application/json; charset=utf-8
> docker-distribution-api-version: registry/2.0
> www-authenticate: Bearer realm="https://cloudapi.canister.io/authorization",service="cloud.canister.io"
> x-content-type-options: nosniff
> date: Sun, 23 Apr 2023 05:29:18 GMT
> content-length: 87
The only challenge however, is that it does not seem that all implementations are following the spec 1:1. For example, in my internal server, the url resolves to `200 OK` but that header is not present at all
Alternatively, querying the [_catalog endpoint ](https://docs.docker.com/registry/spec/api/#listing-repositories) seems to be more reliable across implementations
```shell
HEAD registry.myCompany.com/dockerproxy/v2/_catalog
HEAD registry.myCompany.com/v2/_catalog
HEAD registry.myCompany.com/azul/v2/_catalog
```
As that resolves to `200 OK` if found. I think it might be safe to assume that `v2/_catalog` is a docker endpoint.
2. Underlying bug in our docker client where it does not parse the `@path` correctly even if we identify it and we pass it
For that I reported https://github.com/deitch/docker_registry2/issues/82
edit: This is no longer an issue after https://github.com/deitch/docker_registry2/pull/84 was merged
Contributor guide
Research direction
Start by reproducing the smallest manifest case and reviewing the referenced reproducer commit, then compare candidate registry URLs with the Docker Registry API behavior described in the issue. Done means Docker registry base URLs containing an @path are detected correctly across the stated response variants and passed through the client without losing the path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- devops
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100