docker / docker/buildx

inside image: "Address not available" error for ipv6

Open
#609 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
4.5k
Forks
682
Avg merge
2d 14h
Merged PRs (30d)
29

Description

I use buildx to build an alpine based multiplatform docker image for youtube-dl in the gitlab-ci.
If I run the image on a Raspberry 2 youtube-dl print the following error:

# youtube-dl -v --no-check-certificate https://www.youtube.com/watch?v=<vidoe_id>
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--no-check-certificate', 'https://www.youtube.com/watch?v=rjv91XBT1xI']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.04.01
[debug] Python version 3.8.8 (CPython) - Linux-5.10.29-0-rpi2-armv7l-with
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1
[debug] Proxy map: {}
[youtube] <vidoe_id>: Downloading webpage
WARNING: Unable to download webpage: <urlopen error [Errno 99] Address not available>
[youtube] <vidoe_id>: Downloading API JSON
ERROR: Unable to download API page: <urlopen error [Errno 99] Address not available> (caused by URLError(OSError(99, 'Address not available')))
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2279, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/site-packages/youtube_dl/utils.py", line 2734, in https_open
    return self.do_open(functools.partial(
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)

After some test with wget inside the container, I notify this always happen if a program does use ipv6, but ipv4 seems to be fine:

# wget 'https://www.youtube.com'
Connecting to www.youtube.com ([2607:f8b0:4001:c0f::5b]:443)
wget: can't connect to remote host: Address not available

# wget 'https://loopsofzen.uk/' 
Connecting to loopsofzen.uk ([2001:8b0:0:30::666:102]:443)
wget: can't connect to remote host: Address not available

# wget 'https://docs.docker.com'
Connecting to docs.docker.com (13.224.193.38:443)
saving to 'index.html'
index.html           100% |**************************************************************************************************************| 32834  0:00:00 ETA
'index.html' saved

# wget 'https://www.google.com' && rm index.html 
Connecting to www.google.com (142.250.181.228:443)
saving to 'index.html'
index.html           100% |**************************************************************************************************************| 12935  0:00:00 ETA
'index.html' saved

If I build the image on my Raspberry without buildx, everything works fine.

Dockerfile:

FROM alpine
MAINTAINER kev <noreply@easypi.pro>

RUN set -xe \
    && apk add --no-cache ca-certificates \
                          ffmpeg \
                          openssl \
                          aria2 \
                          youtube-dl

# Try to run it so we know it works
RUN youtube-dl --version

WORKDIR /data

COPY entrypoint.sh /

RUN chmod +x /entrypoint.sh \
 && echo "* * * * * echo please mount cron tasks to /tasks.cron" >> /tasks.cron

RUN addgroup -g 1000 -S dockeruser \
 && adduser -h /data -u 1000 -S dockeruser -G dockeruser

ENTRYPOINT  ["/entrypoint.sh"]

.gitlab-ci.yml:

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_HOST: tcp://docker:2375/

docker-build:
  # Use the docker image with buildx for multiplatform build.
  image: lukas1818/docker-with-buildx:latest
  stage: build
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  # Default branch leaves tag empty (= latest tag)
  # All other branches are tagged with the escaped branch name (commit ref slug)
  script:
    - |
      if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
        tag=""
        echo "Running on default branch '$CI_DEFAULT_BRANCH': tag = 'latest'"
      else
        tag=":$CI_COMMIT_REF_SLUG"
        echo "Running on branch '$CI_COMMIT_BRANCH': tag = $tag"
      fi
    - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes; docker buildx create --use
    - docker buildx build --push --platform linux/arm/v7,linux/arm64/v8,linux/amd64 --tag "$CI_REGISTRY_IMAGE${tag}" .
  # Run this job in a branch where a Dockerfile exists
  retry: 2
  rules:
    - if: $CI_COMMIT_BRANCH
      exists:
        - Dockerfile

A debian based stubby image have also connection issues on my Raspberry if it was build with buildx.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the IPv6 failure with the Dockerfile and .gitlab-ci.yml buildx command on the Raspberry Pi targets, comparing IPv4 and IPv6 connections inside the resulting images. Trace the buildx and QEMU-based multiplatform path before changing anything. Done means images built with buildx can establish IPv6 connections as well as images built natively.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
build-system, infrastructure, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.