jenkinsci / jenkinsci/workflow-cps-plugin
[JENKINS-69380] Some pipeline break with ClassCastException after jenkins restart
- Dominant language
- Java
- Stars
- 186
- Forks
- 213
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 8
Description
Before the Jenkins restart about 300 pipeline where running.
4 of them did not survive the restart. In the Jenkins logs, I found the following Exception that relates to one of the affected pipelines:
```
2022-08-17 05:13:48.280+0000 [id=5356] WARNING o.j.p.w.cps.CpsFlowExecution#cleanUpHeap: failed to clean up memory from Owner[somePipeline/195:somePipeline #195]
java.lang.ClassCastException: class java.io.ObjectStreamClass$Caches$1 cannot be cast to class java.util.concurrent.ConcurrentMap (java.io.ObjectStreamClass$Caches$1 and java.util.concurrent.ConcurrentMap are in module java.base of loader 'bootstrap')
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpObjectStreamClassCaches(CpsFlowExecution.java:1426)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpLoader(CpsFlowExecution.java:1321)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpHeap(CpsFlowExecution.java:1287)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:444)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
```
---
Originally reported by shoelzle, imported from: Some pipeline break with ClassCastException after jenkins restart
Raw content of original issue
Before the Jenkins restart about 300 pipeline where running.
4 of them did not survive the restart. In the Jenkins logs, I found the following Exception that relates to one of the affected pipelines:2022-08-17 05:13:48.280+0000 [id=5356] WARNING o.j.p.w.cps.CpsFlowExecution#cleanUpHeap: failed to clean up memory from Owner[somePipeline/195:somePipeline #195]
java.lang.ClassCastException: class java.io.ObjectStreamClass$Caches$1 cannot be cast to class java.util.concurrent.ConcurrentMap (java.io.ObjectStreamClass$Caches$1 and java.util.concurrent.ConcurrentMap are in module java.base of loader 'bootstrap')
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpObjectStreamClassCaches(CpsFlowExecution.java:1426)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpLoader(CpsFlowExecution.java:1321)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.cleanUpHeap(CpsFlowExecution.java:1287)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:444)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
environment
```
Jenkins: 2.318
plugins:
ace-editor: "1.1"
antisamy-markup-formatter: "2.4"
apache-httpcomponents-client-4-api: "4.5.13-1.0"
authentication-tokens: "1.4"
authorize-project: "1.4.0"
badge: "1.9"
blueocean: "1.25.1"
blueocean-autofavorite: "1.2.4"
blueocean-bitbucket-pipeline: "1.25.1"
blueocean-commons: "1.25.1"
blueocean-config: "1.25.1"
blueocean-core-js: "1.25.1"
blueocean-dashboard: "1.25.1"
blueocean-display-url: "2.4.1"
blueocean-events: "1.25.1"
blueocean-git-pipeline: "1.25.1"
blueocean-github-pipeline: "1.25.1"
blueocean-i18n: "1.25.1"
blueocean-jwt: "1.25.1"
blueocean-personalization: "1.25.1"
blueocean-pipeline-api-impl: "1.25.1"
blueocean-pipeline-editor: "1.25.1"
blueocean-pipeline-scm-api: "1.25.1"
blueocean-rest: "1.25.1"
blueocean-rest-impl: "1.25.1"
blueocean-web: "1.25.1"
bootstrap4-api: "4.6.0-3"
bootstrap5-api: "5.1.1-1"
bouncycastle-api: "2.20"
branch-api: "2.7.0"
build-with-parameters: "1.6"
caffeine-api: "2.9.2-29.v717aac953ff3"
checks-api: "1.7.2"
cloudbees-bitbucket-branch-source: "2.9.11"
cloudbees-folder: "6.16"
command-launcher: "1.2"
conditional-buildstep: "1.4.1"
configuration-as-code: "1.54"
configuration-as-code-groovy: "1.1"
credentials: "2.6.2"
credentials-binding: "1.27"
display-url-api: "2.3.5"
durable-task: "1.39"
echarts-api: "5.2.1-2"
favorite: "2.3.3"
font-awesome-api: "5.15.4-1"
git: "4.9.0"
git-client: "3.10.0"
git-server: "1.10"
github: "1.34.1"
github-api: "1.133"
github-branch-source: "2.11.3"
handlebars: "3.0.8"
handy-uri-templates-2-api: "2.1.8-1.0"
htmlpublisher: "1.27"
http_request: "1.11"
jackson2-api: "2.13.0-230.v59243c64b0a5"
javadoc: "1.6"
jaxb: "2.3.0"
jdk-tool: "1.0"
jenkins-design-language: "1.25.1"
jjwt-api: "0.11.2-9.c8b45b8bb173"
job-dsl: "1.77"
jobConfigHistory: "2.28.1"
jquery3-api: "3.6.0-2"
jsch: "0.1.55.2"
junit: "1.53"
lockable-resources: "2.12"
mailer: "1.34"
matrix-auth: "2.6.8"
matrix-project: "1.19"
maven-plugin: "3.15"
metrics: "4.0.2.8"
momentjs: "1.1.1"
okhttp-api: "3.14.9"
parameterized-trigger: "2.41"
pipeline-build-step: "2.15"
pipeline-graph-analysis: "1.11"
pipeline-input-step: "2.12"
pipeline-milestone-step: "1.3.2"
pipeline-model-api: "1.9.2"
pipeline-model-definition: "1.9.2"
pipeline-model-extensions: "1.9.2"
pipeline-rest-api: "2.19"
pipeline-stage-step: "2.5"
pipeline-stage-tags-metadata: "1.9.2"
pipeline-stage-view: "2.19"
pipeline-utility-steps: "2.10.0"
plain-credentials: "1.7"
plugin-util-api: "2.5.0"
popper-api: "1.16.1-2"
popper2-api: "2.10.2-1"
pubsub-light: "1.16"
resource-disposer: "0.16"
run-condition: "1.5"
scm-api: "2.6.5"
script-security: "1.78"
snakeyaml-api: "1.29.1"
sse-gateway: "1.24"
ssh-credentials: "1.19"
ssh-steps: "2.0.0"
sshd: "3.1.0"
structs: "1.23"
swarm: "3.28"
thinBackup: "1.10"
timestamper: "1.13"
token-macro: "266.v44a80cf277fd"
trilead-api: "1.0.13"
variant: "1.4"
webhook-step: "1.4"
workflow-aggregator: "2.6"
workflow-api: "2.47"
workflow-basic-steps: "2.24"
workflow-cps: "2.94"
workflow-cps-global-lib: "2.21"
workflow-durable-task-step: "2.40"
workflow-job: "2.42"
workflow-multibranch: "2.26"
workflow-scm-step: "2.13"
workflow-step-api: "2.24"
workflow-support: "3.8"
ws-cleanup: "0.39"
```
Contributor guide
Research direction
Start in CpsFlowExecution.cleanUpObjectStreamClassCaches, following its callers cleanUpLoader and cleanUpHeap, then compare the reported ClassCastException with the restart path through CpsThreadGroup. Reproduce the failure around a restarted running pipeline and verify that affected pipelines resume without this exception or cleanup warning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ci-cd
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100