actions / actions/actions-runner-controller

Runners created with actions-runner-controller in we have a lot of pods with errors: "Cannot connect to the Docker daemon at unix:///run/docker.sock. Is the docker daemon running?"

Open
#3,257 19 comments 10 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug community needs triage
Dominant language
Go
Stars
6.5k
Forks
1.5k
Avg merge
2d 2h
Merged PRs (30d)
27

Description

Checks
Controller Version

latest

Helm Chart Version

0.27.6

CertManager Version

1.13.1

Deployment Method

Helm

cert-manager installation

Installed ok by Chart.yaml

Checks
  • This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
  • I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
  • My actions-runner-controller version (v0.x.y) does support the feature
  • I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
  • I've migrated to the workflow job webhook event (if you using webhook driven scaling)
Resource Definitions
apiVersion: actions.summerwind.dev/v1alpha1 
kind: RunnerDeployment
metadata:
  {{- if hasSuffix "-dev" .Release.Namespace  }}
  name: {{ .Values.runnerDeploymentDev.name }}
  namespace: {{ .Release.Namespace }}
  {{- end  }}
  {{- if hasSuffix "-prd" .Release.Namespace }}
  name: {{ .Values.runnerDeploymentPrd.name }}
  namespace: {{ .Release.Namespace }}
  {{- end }}
spec:
  {{- if hasSuffix "-dev" .Release.Namespace  }}
  replicas: {{ .Values.runnerDeploymentDev.replicas }}
  {{- end }}
  {{- if hasSuffix "-prd" .Release.Namespace }}
  replicas: {{ .Values.runnerDeploymentPrd.replicas }}
  {{- end }}
  template:
    spec:
      {{- if hasSuffix "-dev" .Release.Namespace  }}
      image: {{ .Values.runnerDeploymentDev.image }} ## Alterar para repositório de DEV
      {{- end }}
      {{- if hasSuffix "-prd" .Release.Namespace }}
      image: {{ .Values.runnerDeploymentPrd.image }} ## Alterar para repositório de Prd
      {{- end }}
      organization: company-a
      {{- if hasSuffix "-dev" .Release.Namespace  }}
      labels:
        {{- range .Values.runnerDeploymentDev.labels }}
        {{ "-" }} {{ . }}
        {{- end }}
      {{- end }}
      {{- if hasSuffix "-prd" .Release.Namespace }}
      labels:
        {{- range .Values.runnerDeploymentPrd.labels }}
        {{ "-" }} {{ . }}
        {{- end }}
      {{- end }}
      env:
        - name: teste
          {{- if hasSuffix "-dev" .Release.Namespace  }}
          value: a
          {{- end }}
          {{- if hasSuffix "-prd" .Release.Namespace  }}
          value: b
          {{- end }}
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  {{- if hasSuffix "-dev" .Release.Namespace  }}
  name: {{ .Values.HpaDev.name }}
  namespace: {{ .Release.Namespace }}
  {{- end }}
  {{- if hasSuffix "-prd" .Release.Namespace }} 
  name: {{ .Values.HpaPrd.name }}
  namespace: {{ .Release.Namespace }}
  {{- end }}
spec:
  scaleTargetRef:
    kind: RunnerDeployment
    {{- if hasSuffix "-dev" .Release.Namespace  }}
    name: {{ .Values.HpaDev.nameRunner }}
    {{- end }}
    {{- if hasSuffix "-prd" .Release.Namespace }} 
    name: {{ .Values.HpaPrd.nameRunner }}
    {{- end }}
  {{- if hasSuffix "-dev" .Release.Namespace  }}
  minReplicas: {{ .Values.HpaDev.minReplicas }}
  maxReplicas: {{ .Values.HpaDev.maxReplicas }}
  scaleDownDelaySecondsAfterScaleOut: {{ .Values.HpaDev.scaleDownDelaySecondsAfterScaleOut }}
  metrics:
  - type: {{ .Values.HpaDev.type }}
    scaleUpThreshold: '{{ .Values.HpaDev.scaleUpThreshold }}'   # The percentage of busy runners at which the number of desired runners are re-evaluated to scale up
    scaleDownThreshold: '{{ .Values.HpaDev.scaleDownThreshold }}'  # The percentage of busy runners at which the number of desired runners are re-evaluated to scale down
    scaleUpAdjustment: {{ .Values.HpaDev.scaleUpAdjustment }}        # The scale up runner count added to desired count
    scaleDownAdjustment: {{ .Values.HpaDev.scaleDownAdjustment }}     # The scale down runner count subtracted from the desired count
    # Podemos usar os parametros de Factor ou Adjustment acima, mas não os dois juntos.
    # scaleUpFactor: {{ .Values.HpaDev.scaleUpFactor }}        # The scale up runner count added to desired count
    # scaleDownFactor: {{ .Values.HpaDev.scaleDownFactor }}     # The scale down runner count subtracted from the desired count
    
  {{- end }}
  {{- if hasSuffix "-prd" .Release.Namespace }}
  minReplicas: {{ .Values.HpaPrd.minReplicas }}
  maxReplicas: {{ .Values.HpaPrd.maxReplicas }}
  scaleDownDelaySecondsAfterScaleOut: {{ .Values.HpaPrd.scaleDownDelaySecondsAfterScaleOut }}
  metrics:
  - type: {{ .Values.HpaPrd.type }}
    scaleUpThreshold: '{{ .Values.HpaPrd.scaleUpThreshold }}'   # The percentage of busy runners at which the number of desired runners are re-evaluated to scale up
    scaleDownThreshold: '{{ .Values.HpaPrd.scaleDownThreshold }}'  # The percentage of busy runners at which the number of desired runners are re-evaluated to scale down
    scaleUpAdjustment: {{ .Values.HpaPrd.scaleUpAdjustment }}       # The scale up runner count added to desired count
    scaleDownAdjustment: {{ .Values.HpaPrd.scaleDownAdjustment }}     # The scale down runner count subtracted from the desired count
    # Podemos usar os parametros de Factor ou Adjustment acima, mas não os dois juntos.
    # scaleUpFactor: {{ .Values.HpaPrd.scaleUpFactor }}       # The scale up runner count added to desired count
    # scaleDownFactor: {{ .Values.HpaPrd.scaleDownFactor }}     # The scale down runner count subtracted from the desired count
  {{- end }}
To Reproduce
1. Create a runner
2. Watch runner container logs
Describe the bug

I'm using a GKE version: 1.26.10-gke.1101000. In my Dockerfile, I'm using: FROM summerwind/actions-runner:latest.

In values.yaml, I'm using:

image:
  repository: "summerwind/actions-runner-controller"
  actionsRunnerRepositoryAndTag: "summerwind/actions-runner:latest"
  dindSidecarRepositoryAndTag: "docker:dind"
  pullPolicy: IfNotPresent
  # The default image-pull secrets name for self-hosted runner container.
  # It's added to spec.ImagePullSecrets of self-hosted runner pods. 
  actionsRunnerImagePullSecrets: []

But when deploy is done, in GKE and get a lot of pods, with error: "Cannot connect to the Docker daemon at unix:///run/docker.sock. Is the docker daemon running?"

The pods are restarting with error in container "docker" with this message: "Cannot connect to the Docker daemon at unix:///run/docker.sock. Is the docker daemon running?". It died and start new with the same problem.

I've already follow this issue: 2490, but doesn't work.

Could help me please?

Describe the expected behavior

Doesn't get this situation with error, and running normally.

Whole Controller Logs
Nothing logs in controller with errors.
Whole Runner Pod Logs
In pods I got the same error: "Cannot connect to the Docker daemon at unix:///run/docker.sock. Is the docker daemon running?"

I've tried to change $DOCKER_HOST to DOCKER_HOST="tcp://localhost:2375", but when I open the running that I can, a do echo $DOCKER_HOST and my response is: unix:///run/docker.sock. I don't think this can be the error.
Additional Context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the RunnerDeployment resource definition, values.yaml, and the Dockerfile using summerwind/actions-runner:latest. Compare the dind sidecar setup and DOCKER_HOST behavior with TROUBLESHOOTING.md and issue 2490, then reproduce the runner pod logs. Done means the runner pods start without Docker daemon errors and remain healthy.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, gcp, github-actions, go, helm, kubernetes
Domain
cloud, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.