jenkinsci / jenkinsci/workflow-durable-task-step-plugin

[JENKINS-72087] Hangs of executors

Open
#598 6 comments 0 reactions 0 assignees View on GitHub
component:workflow-durable-task-step-plugin imported-jira-issue priority:major resolution:unresolved
Dominant language
Java
Stars
46
Forks
110
PR merge metrics
No merged PRs in 30d

Description

From time to time - about 2 or 3 free weeks - we're facing strange issue.

In our Jenkins instance with about 100 agents all executors starting to hangs.

In GUI it shows that executor is busy with build that is already finished.

If we try to stop this build it asks about "null" build.

From script console, something like this will return "null", while normally it return build name.

computer.getAllExecutors().each { e ->

println(e.getCurrentExecutable().getParent().run())
}

There's no any usable information in logs.

Only restart jenkins instance helps.

 

Most of our agents are permanent, launched via SSH, but we are also using Docker Swarm (via docker-swarm:1.11 plugin).

Last version problem accour was 2.414.2 - after 2 weeks of usage

First time we observe it around 2.440, but it's probably not version related.

We're using only LTS releases.

We start Jenkins in docker container from official LTS image (currently it's jenkins/jenkins:2.414.2), but with modified entrypoint like

# Additional Jenkins options

# -XX:+AlwaysPreTouch - pre-zeroes memory mapped pages on JVM startup
# -XX:+UseStringDeduplication - looks for the strings with the same contents and deduplicates them
# -XX:+ParallelRefProcEnabled - enables parallelize reference processing, reducing young and old GC times.
# -XX:+DisableExplicitGC - disables the system.gc() method called often used by third party plugins to explicitly invoke the garbage collector.
# -Xms - Allocate min memory pool for operations, garbage collection, etc.
# -Xmx - Allocate max memory pool for operations, garbage collection, etc.
# -Djenkins.install.runSetupWizard=false - Skip initial setup wizard. Do not
#       ask for plugin selection, do not create admin user, do not setup proxy.
# -Dhudson.slaves.WorkspaceList=_ - use underscore instead @ to create concurrent workspaces
export JAVA_OPTS="-Xms4096m -Xmx16384m -XX:+AlwaysPreTouch -XX:+UseStringDeduplication -XX:+ParallelRefProcEnabled -XX:+DisableExplicitGC -Djenkins.install.runSetupWizard=false -Dhudson.slaves.WorkspaceList=_"
export JENKINS_OPTS="--logfile=/var/log/jenkins/jenkins.log --httpPort=-1 --httpsPort=8080 --http2Port=8443 --httpsKeyStore=${JENKINS_HOME}/https/jenkins_ssl.jks --httpsKeyStorePassword=${SSL_CRT_PWD}"# Call the Jenkins entrypoint
exec /usr/bin/tini -- /usr/local/bin/jenkins.sh

---
Originally reported by xjjx, imported from: Hangs of executors


  • status: Open
  • priority: Major
  • component(s): workflow-durable-task-step-plugin
  • resolution: Unresolved
  • votes: 1
  • watchers: 4
  • imported: 20251212-090250

Raw content of original issue

From time to time - about 2 or 3 free weeks - we're facing strange issue.
In our Jenkins instance with about 100 agents all executors starting to hangs.
In GUI it shows that executor is busy with build that is already finished.
If we try to stop this build it asks about "null" build.

From script console, something like this will return "null", while normally it return build name.



computer.getAllExecutors().each { e ->

println(e.getCurrentExecutable().getParent().run())
}


There's no any usable information in logs.
Only restart jenkins instance helps.

 

Most of our agents are permanent, launched via SSH, but we are also using Docker Swarm (via docker-swarm:1.11 plugin).

Last version problem accour was 2.414.2 - after 2 weeks of usage

First time we observe it around 2.440, but it's probably not version related.

We're using only LTS releases.

We start Jenkins in docker container from official LTS image (currently it's jenkins/jenkins:2.414.2), but with modified entrypoint like



# Additional Jenkins options

# -XX:+AlwaysPreTouch - pre-zeroes memory mapped pages on JVM startup
# -XX:+UseStringDeduplication - looks for the strings with the same contents and deduplicates them
# -XX:+ParallelRefProcEnabled - enables parallelize reference processing, reducing young and old GC times.
# -XX:+DisableExplicitGC - disables the system.gc() method called often used by third party plugins to explicitly invoke the garbage collector.
# -Xms - Allocate min memory pool for operations, garbage collection, etc.
# -Xmx - Allocate max memory pool for operations, garbage collection, etc.
# -Djenkins.install.runSetupWizard=false - Skip initial setup wizard. Do not
#       ask for plugin selection, do not create admin user, do not setup proxy.
# -Dhudson.slaves.WorkspaceList=_ - use underscore instead @ to create concurrent workspaces
export JAVA_OPTS="-Xms4096m -Xmx16384m -XX:+AlwaysPreTouch -XX:+UseStringDeduplication -XX:+ParallelRefProcEnabled -XX:+DisableExplicitGC -Djenkins.install.runSetupWizard=false -Dhudson.slaves.WorkspaceList=_"
export JENKINS_OPTS="--logfile=/var/log/jenkins/jenkins.log --httpPort=-1 --httpsPort=8080 --http2Port=8443 --httpsKeyStore=${JENKINS_HOME}/https/jenkins_ssl.jks --httpsKeyStorePassword=${SSL_CRT_PWD}"# Call the Jenkins entrypoint
exec /usr/bin/tini -- /usr/local/bin/jenkins.sh

  • environment: plugins files is in attachment

3 attachments

- [jenkins.fail.1.PNG](https://issues.jenkins.io/secure/attachment/61215/jenkins.fail.1.PNG)
> ![jenkins.fail.1.PNG](https://issues.jenkins.io/secure/attachment/61215/jenkins.fail.1.PNG)
- [jenkins.fail.2.PNG](https://issues.jenkins.io/secure/attachment/61214/jenkins.fail.2.PNG)
> ![jenkins.fail.2.PNG](https://issues.jenkins.io/secure/attachment/61214/jenkins.fail.2.PNG)
- [plugins.txt](https://issues.jenkins.io/secure/attachment/61266/plugins.txt)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.