actions / actions/actions-runner-controller

Docker in docker no longer works in summerwind/actions-runner-dind-rootless, docker socket not properly shared

Open
#2,548 17 comments 9 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs triage
Dominant language
Go
Stars
6.5k
Forks
1.5k
Avg merge
2d 2h
Merged PRs (30d)
27

Description

Checks
Controller Version

0.27.3

Helm Chart Version

0.23.2

CertManager Version

N/A

Deployment Method

Helm

cert-manager installation

N/A

Checks
  • This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
  • I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
  • My actions-runner-controller version (v0.x.y) does support the feature
  • I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
  • I've migrated to the workflow job webhook event (if you using webhook driven scaling)
Resource Definitions
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: runner-<hidden>-test
  namespace: actions-runners-dev
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: "<hidden>-controller-dev-runner"
    spec:
      ephemeral: true
      dockerdWithinRunnerContainer: true
      repository: <hidden>/<hidden>
      image: summerwind/actions-runner-dind-rootless:ubuntu-22.04
      labels:
        - <mylabel>
To Reproduce
1. Run workflow based on `actions-runner-dind-rootless` which uses docker inside container, as an example:

jobs:
  test-runner-in-container:
    runs-on: [ self-hosted, <hidden> ]
    container:
      image: docker:23.0
    steps:
      - name: check out repository code
        uses: actions/checkout@v3
      - name: test docker inside container
        run: |
          docker ps
  1. docker ps will fail because /var/run/docker.sock is not a docker socket.
Describe the bug

Docker containers started in the runner don't have a working docker:

runner@runner-myorg-test-q56m5-sq49m:~$ docker ps
CONTAINER ID   IMAGE         COMMAND               CREATED          STATUS          PORTS           NAMES
ea6daf34cb62   docker:23.0   "tail -f /dev/null"   12 minutes ago   Up 12 minutes   2375-2376/tcp   945bfd4a364f443989b9c74270e20b29_docker230_e5a62a

runner@runner-myorg-test-q56m5-sq49m:~$ docker exec -it ea6daf34cb62 sh

/__w/<hidden>/ # docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

It is because docker.sock is not shared properly between the runner and containers. On the runner DOCKER_HOST is:

runner@runner-myorg-test-q56m5-sq49m:~$ echo $DOCKER_HOST
unix:///run/user/1000/docker.sock

But runner is trying to share /var/run/docker.sock with the container:

runner@runner-myorg-test-q56m5-sq49m:~$ docker inspect ea6daf34cb62
            {
                "Type": "bind",
                "Source": "/var/run/docker.sock",
                "Destination": "/var/run/docker.sock",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },

/var/run/docker.sock does not exist on actions-runner-dind-rootless runners. /run/user/1000/docker.sock should be a Source of the above bind.

Describe the expected behavior

I would like docker to work inside containers started by the runner when using actions-runner-dind-rootless.

Whole Controller Logs
N/A
Whole Runner Pod Logs
N/A
Additional Context

N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the RunnerDeployment manifest and the reproduction workflow in the issue, then inspect how dockerdWithinRunnerContainer and the runner's Docker socket are passed into job containers. Reproduce docker ps with actions-runner-dind-rootless:ubuntu-22.04 and verify the bind source uses /run/user/1000/docker.sock rather than /var/run/docker.sock.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions, helm, kubernetes
Domain
ci-cd, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.