jenkinsci / jenkinsci/docker-workflow-plugin

[JENKINS-68897] Docker agent startup failure when running on rootless docker

Open

#693 opened on Jul 1, 2022 0 comments 0 reactions 0 assignees View on GitHub

component:docker-workflow-plugin imported-jira-issue priority:minor resolution:unresolved

Dominant language: Java
Stars: 527
Forks: 422
Avg merge: 10m
Merged PRs (30d): 1

Description

Test pipeline:

pipeline {

    agent none

    stages {

stage('run') {

    agent {

docker {

    image 'python-3.10.5'

    label 'docker-rootless'

}

    }

    steps {

sh('echo test')

    }

}

    }

}

How to reproduce

If I run the above pipeline on the Ubuntu node with a regular docker-ce install, then the job is successful.

If I change the Ubuntu agent to run rootless docker, following the guide at https://docs.docker.com/engine/security/rootless/, then the build fails with this error:

```
Started by user admin
[Pipeline] Start of Pipeline
[Pipeline] stage
[Pipeline] { (run)
[Pipeline] node
Running on ubuntu-rootless-docker in /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker
[Pipeline] {
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . python:3.10.5-slim
.
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] withDockerContainer
ubuntu-rootless-docker does not seem to be running inside a container
$ docker run -t -d -u 128:134 -w /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker -v /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker:/var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker:rw,z -v /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker@tmp:/var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** python:3.10.5-slim cat
$ docker top 2aad1b12fa63578e2fb54afb232c7601c77afbbe76664dc40a347bc3fbf35cd1 -eo pid,comm
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
java.io.IOException: Failed to run top '2aad1b12fa63578e2fb54afb232c7601c77afbbe76664dc40a347bc3fbf35cd1'. Error: Error response from daemon: runc did not terminate successfully: exit status 1: unable to get all container pids: lstat : no such file or directory
: unknown
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.listProcess(DockerClient.java:152)
at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:201)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:322)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:47)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:140)
at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66)
at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:125)
at org.jenkinsci.plugins.docker.workflow.declarative.DockerPipelineScript.runImage(DockerPipelineScript.groovy:54)
at org.jenkinsci.plugins.docker.workflow.declarative.AbstractDockerPipelineScript.configureRegistry(AbstractDockerPipelineScript.groovy:63)
at org.jenkinsci.plugins.docker.workflow.declarative.AbstractDockerPipelineScript.run(AbstractDockerPipelineScript.groovy:50)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.CheckoutScript.checkoutAndRun(CheckoutScript.groovy:64)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.CheckoutScript.doCheckout(CheckoutScript.groovy:40)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.LabelScript.run(LabelScript.groovy:43)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at jdk.internal.reflect.GeneratedMethodAccessor260.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:95)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Finished: FAILURE

```

The only other change made to the rootless docker node is to set

DOCKER_HOST=unix:///run/user/128/docker.sock  # jenkins-agent uid=128

in the Node Properties -> Environment Variables configuration section.

---
Originally reported by wwuck, imported from: Docker agent startup failure when running on rootless docker

status: Open
priority: Minor
component(s): docker-workflow-plugin
resolution: Unresolved
votes: 0
watchers: 1
imported: 2025-12-07

Raw content of original issue

Test pipeline:

pipeline {

    agent none

    stages {

        stage('run') {

            agent {

                docker {

                    image 'python-3.10.5'

                    label 'docker-rootless'

                }

            }

            steps {

                sh('echo test')

            }

        }

    }

}

How to reproduce

If I run the above pipeline on the Ubuntu node with a regular docker-ce install, then the job is successful.

If I change the Ubuntu agent to run rootless docker, following the guide at https://docs.docker.com/engine/security/rootless/, then the build fails with this error:

Started by user admin

[Pipeline] Start of Pipeline

[Pipeline] stage

[Pipeline] { (run)

[Pipeline] node

Running on ubuntu-rootless-docker in /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker

[Pipeline] {

[Pipeline] isUnix

[Pipeline] withEnv

[Pipeline] {

[Pipeline] sh

+ docker inspect -f . python:3.10.5-slim

.

[Pipeline] }

[Pipeline] // withEnv

[Pipeline] withDockerContainer

ubuntu-rootless-docker does not seem to be running inside a container

$ docker run -t -d -u 128:134 -w /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker -v /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker:/var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker:rw,z -v /var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker@tmp:/var/lib/jenkins-agent/agent/workspace/test-docker-rootless-docker@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** python:3.10.5-slim cat

$ docker top 2aad1b12fa63578e2fb54afb232c7601c77afbbe76664dc40a347bc3fbf35cd1 -eo pid,comm

[Pipeline] // withDockerContainer

[Pipeline] }

[Pipeline] // node

[Pipeline] }

[Pipeline] // stage

[Pipeline] End of Pipeline

java.io.IOException: Failed to run top '2aad1b12fa63578e2fb54afb232c7601c77afbbe76664dc40a347bc3fbf35cd1'. Error: Error response from daemon: runc did not terminate successfully: exit status 1: unable to get all container pids: lstat : no such file or directory

: unknown

	at org.jenkinsci.plugins.docker.workflow.client.DockerClient.listProcess(DockerClient.java:152)

	at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:201)

	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:322)

	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)

	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)

	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:47)

	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)

	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)

	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)

	at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:140)

	at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66)

	at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:125)

	at org.jenkinsci.plugins.docker.workflow.declarative.DockerPipelineScript.runImage(DockerPipelineScript.groovy:54)

	at org.jenkinsci.plugins.docker.workflow.declarative.AbstractDockerPipelineScript.configureRegistry(AbstractDockerPipelineScript.groovy:63)

	at org.jenkinsci.plugins.docker.workflow.declarative.AbstractDockerPipelineScript.run(AbstractDockerPipelineScript.groovy:50)

	at org.jenkinsci.plugins.pipeline.modeldefinition.agent.CheckoutScript.checkoutAndRun(CheckoutScript.groovy:64)

	at org.jenkinsci.plugins.pipeline.modeldefinition.agent.CheckoutScript.doCheckout(CheckoutScript.groovy:40)

	at org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.LabelScript.run(LabelScript.groovy:43)

	at ___cps.transform___(Native Method)

	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)

	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)

	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)

	at jdk.internal.reflect.GeneratedMethodAccessor260.invoke(Unknown Source)

	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.base/java.lang.reflect.Method.invoke(Method.java:566)

	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)

	at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)

	at com.cloudbees.groovy.cps.Next.step(Next.java:83)

	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)

	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)

	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)

	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)

	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)

	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)

	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)

	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)

	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420)

	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:95)

	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330)

	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294)

	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)

	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)

	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)

	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)

	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

	at java.base/java.lang.Thread.run(Thread.java:829)

Finished: FAILURE

The only other change made to the rootless docker node is to set

DOCKER_HOST=unix:///run/user/128/docker.sock  # jenkins-agent uid=128

in the Node Properties -> Environment Variables configuration section.

environment

```
Jenkins controller:

docker run -p 8080:8080 --restart=on-failure -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts-jdk11

Node:

Ubuntu 20.04.4 VM running docker-ce 5:20.10.17~3-0~ubuntu-focal, connected to controller via SSH

Jenkins: 2.346.1

OS: Linux - 5.18.0-0.bpo.1-amd64

---

ace-editor:1.1

ant:475.vf34069fef73c

antisamy-markup-formatter:2.7

apache-httpcomponents-client-4-api:4.5.13-1.0

authentication-tokens:1.4

bootstrap5-api:5.1.3-7

bouncycastle-api:2.26

branch-api:2.1046.v0ca_37783ecc5

build-timeout:1.21

caffeine-api:2.9.3-65.v6a_47d0f4d1fe

checks-api:1.7.4

cloudbees-folder:6.729.v2b_9d1a_74d673

command-launcher:84.v4a_97f2027398

commons-lang3-api:3.12.0.0

commons-text-api:1.9-9.v39a_53e2e0343

configuration-as-code:1464.vd8507b_82e41a_

credentials:1129.vef26f5df883c

credentials-binding:523.vd859a_4b_122e6

display-url-api:2.3.6

docker-commons:1.19

docker-workflow:1.29

durable-task:496.va67c6f9eefa7

echarts-api:5.3.3-1

email-ext:2.89

font-awesome-api:6.1.1-1

git:4.11.3

git-client:3.11.0

github:1.34.4

github-api:1.303-400.v35c2d8258028

github-branch-source:1656.v77eddb_b_e95df

gradle:1.39.2

handlebars:3.0.8

jackson2-api:2.13.3-285.vc03c0256d517

javax-activation-api:1.2.0-3

javax-mail-api:1.6.2-6

jaxb:2.3.6-1

jdk-tool:1.5

jjwt-api:0.11.5-77.v646c772fddb_0

jquery3-api:3.6.0-4

jsch:0.1.55.2

junit:1119.1121.vc43d0fc45561

ldap:2.10

mailer:414.vcc4c33714601

matrix-auth:3.1.3

matrix-project:772.v494f19991984

mina-sshd-api-common:2.8.0-21.v493b_6b_db_22c6

mina-sshd-api-core:2.8.0-21.v493b_6b_db_22c6

momentjs:1.1.1

okhttp-api:4.9.3-105.vb96869f8ac3a

pam-auth:1.8

pipeline-build-step:2.18

pipeline-github-lib:38.v445716ea_edda_

pipeline-graph-analysis:195.v5812d95a_a_2f9

pipeline-groovy-lib:593.va_a_fc25d520e9

pipeline-input-step:449.v77f0e8b_845c4

pipeline-milestone-step:101.vd572fef9d926

pipeline-model-api:2.2097.v33db_b_de764b_e

pipeline-model-definition:2.2097.v33db_b_de764b_e

pipeline-model-extensions:2.2097.v33db_b_de764b_e

pipeline-rest-api:2.24

pipeline-stage-step:293.v200037eefcd5

pipeline-stage-tags-metadata:2.2097.v33db_b_de764b_e

pipeline-stage-view:2.24

plain-credentials:1.8

plugin-util-api:2.17.0

popper2-api:2.11.5-2

resource-disposer:0.19

scm-api:608.vfa_f971c5a_a_e9

script-security:1175.v4b_d517d6db_f0

snakeyaml-api:1.30.2-76.vc104f7ce9870

ssh-credentials:291.v8211e4f8efb_c

ssh-slaves:1.821.vd834f8a_c390e

sshd:3.242.va_db_9da_b_26a_c3

structs:318.va_f3ccb_729b_71

timestamper:1.18

token-macro:293.v283932a_0a_b_49

trilead-api:1.57.v6e90e07157e1

variant:1.4

workflow-aggregator:590.v6a_d052e5a_a_b_5

workflow-api:1165.v02c3db_a_6b_e36

workflow-basic-steps:948.v2c72a_091b_b_68

workflow-cps:2725.v7b_c717eb_12ce

workflow-durable-task-step:1155.v79567b_e0a_2de

workflow-job:1189.va_d37a_e9e4eda_

workflow-multibranch:716.vc692a_e52371b_

workflow-scm-step:400.v6b_89a_1317c9a_

workflow-step-api:625.vd896b_f445a_f8

workflow-support:820.vd1a_6cc65ef33

ws-cleanup:0.42
```

1 attachment

- [support_2022-07-01_12.49.35.zip](https://issues.jenkins.io/secure/attachment/58405/support_2022-07-01_12.49.35.zip)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

[JENKINS-68897] Docker agent startup failure when running on rootless docker

Description

Test pipeline:

How to reproduce

Test pipeline:

How to reproduce

Contributor guide

Assessment

Get new issues in your inbox