actions / actions/actions-runner-controller
containers create by Github workflow have wrong dockerMTU
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.5k
- Forks
- 1.5k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 27
Description
Describe the bug
No network possible from Docker containers created by Github workflows (dind - Docker in Docker)
To Reproduce
Steps to reproduce the behavior:
- create workflow with docker image, e.g.
name: Setup Go
on:
workflow_dispatch:
jobs:
setupgo:
runs-on: [kubernetes]
container:
image: 'ubuntu:latest'
steps:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.17
- name: sleep
shell: bash
run: sleep 300
- exec into the dind sidecar container and confirm that mtu parameter is propagated properly from runner deployment spec:
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: my-runner
spec:
template:
spec:
organization: abc
ephemeral: true
dockerMTU: 1440
$ kubectl exec -it my-runner-xyz -c docker -- /bin/sh
$ ps auxww
1 root 0:00 docker-init -- dockerd --host=unix:///var/run/docker.sock --host=tcp://0.0.0.0:2376 --tlsverify --tlscacert /certs/server/ca.pem --tlscert /certs/server/cert.pem --tlskey /certs/server/key.pem --mtu 1440
53 root 0:03 dockerd --host=unix:///var/run/docker.sock --host=tcp://0.0.0.0:2376 --tlsverify --tlscacert /certs/server/ca.pem --tlscert /certs/server/cert.pem --tlskey /certs/server/key.pem --mtu 1440
61 root 0:04 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
...
$ ip a
...
4: eth0@if700: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1440 qdisc noqueue state UP
link/ether d2:ee:d5:2f:04:ae brd ff:ff:ff:ff:ff:ff
inet 100.96.2.200/32 brd 100.96.2.200 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::d0ee:d5ff:fe2f:4ae/64 scope link
valid_lft forever preferred_lft forever
- run the workflow and exec into the workflow docker container
$ kubectl exec -it my-runner-xyz -c docker -- /bin/sh
$ docker ps
$ docker exec -it <containerID> bash
$ apt update
$ apt install curl -y
$ curl -vOL https://github.com/actions/go-versions/releases/download/1.17.6-1668090892/go-1.17.6-linux-x64.tar.gz
0 0 0 0 0 0 0 0 --:--:-- 0:00:54 --:--:-- 0
-> nothing is happening...
$ apt install iproute2 -y
$ ip a
...
eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
-> wrong MTU
Expected behavior
I would expect that downloading files should work from docker containers.
If I manually start a docker container from the dind sidecar container -> no network problems:
$ kubectl exec -it my-runner-xyz -c docker -- /bin/sh
$ docker run -it --rm ubuntu bash
$ apt update
$ apt install curl -y
$ curl -vOL https://github.com/actions/go-versions/releases/download/1.17.6-1668090892/go-1.17.6-linux-x64.tar.gz
100 128M 100 128M 0 0 22.9M 0 0:00:05 0:00:05 --:--:-- 23.7M
--> download completed
$ apt install iproute2 -y
$ ip a
...
eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1440
-> correct MTU
Environment:
- Helm Chart Version 0.14
- Controller Version: 0.20.2
- Deployment Method: Helm + Kustomize + fluxV2
- Kubernetes: ip-in-ip Calico on AWS
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the dockerMTU value from the RunnerDeployment spec into the Docker-in-Docker sidecar and then into containers created by the workflow. Reproduce the mismatch with the provided Kubernetes and workflow configuration, comparing the sidecar's 1440 MTU with the workflow container's 1500 MTU; done means workflow-created containers inherit the configured MTU and downloads succeed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, kubernetes
- Domain
- devops, infrastructure, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100