enroot import Vs harbor (https) +s3 backend (http)

Open
#222 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
docker, shell
Domain
cli, networking

Research direction

Start with the enroot.conf settings and the curl_proto/curl_opts logic shown in the issue. Trace how registry redirects are handled during import, then define and test the proposed HTTPS-preference behavior while preserving the existing default and HTTP allowance semantics.

Written by the indexing model from the issue text.

Description

Hi,
we've found an issue with installation where

  1. Harbor (docker registry) works only via HTTPS (and return 308 redirect to https for all plain reqests)
  2. Harbor has s3 storage backend connected via http protocol

In this case registry returns
Location: http://ceph.storage.local/harbor05/docker/registry/v2/blobs/sha256/d1/d1eca21af2432ccd13f82f7d5d346a9158523fb32a5462042c7ceba25c6ffdda/data?some_X-Amz_args
for HTTPS request to the registry
GET v2/dockerhub-proxy/oguzpastirmaci/gpu-burn/blobs/sha256:d1eca21af2432ccd13f82f7d5d346a9158523fb32a5462042c7ceba25c6ffdda

As I understand, this behavior controls by storage_service: redirect parameter, but I can't change this (not my infrastructure)

enroot.conf config file has ENROOT_ALLOW_HTTP option, but it sets HTTP as a prefer protocol, which is incorrect in our case

if [ -n "${ENROOT_ALLOW_HTTP-}" ]; then
    readonly curl_proto="http"
    readonly curl_opts=("--proto" "=http,https" "--retry" "${ENROOT_TRANSFER_RETRIES}" "--connect-timeout" "${ENROOT_CONNECT_TIMEOUT}" "--max-time" "${ENROOT_TRANSFER_TIMEOUT}" "-SsL")
else
    readonly curl_proto="https"
    readonly curl_opts=("--proto" "=https" "--retry" "${ENROOT_TRANSFER_RETRIES}" "--connect-timeout" "${ENROOT_CONNECT_TIMEOUT}" "--max-time" "${ENROOT_TRANSFER_TIMEOUT}" "-SsL")
fi

To fix my concrete issue I've just changed above script to

if [ -n "${ENROOT_ALLOW_HTTP-}" ]; then
    readonly curl_proto="https"
    readonly curl_opts=("--proto" "=http,https" "--retry" "${ENROOT_TRANSFER_RETRIES}" "--connect-timeout" "${ENROOT_CONNECT_TIMEOUT}" "--max-time" "${ENROOT_TRANSFER_TIMEOUT}" "-SsL")
else
    readonly curl_proto="https"
    readonly curl_opts=("--proto" "=https" "--retry" "${ENROOT_TRANSFER_RETRIES}" "--connect-timeout" "${ENROOT_CONNECT_TIMEOUT}" "--max-time" "${ENROOT_TRANSFER_TIMEOUT}" "-SsL")
fi

But, I believe that it will be better to add extra option like ENROOT_PREFER_HTTPS which will describe my situation/setup

Dominant language
Shell
Stars
1.1k
Forks
139
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/enroot

All issues in NVIDIA/enroot

Similar issues

More Shell/Bash issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.