jenkinsci / jenkinsci/authorize-project-plugin

[JENKINS-75711] Significant Declarative Pipeline Performance Issues

Open
#458 9 comments 0 reactions 0 assignees View on GitHub
component:authorize-project-plugin imported-jira-issue priority:major resolution:unresolved
Dominant language
Java
Stars
18
Forks
40
PR merge metrics
No merged PRs in 30d

Description

Attached an old list of plugins (old-plugins-to-install.yaml) and a new list of plugins (new-plugins-to-install.yaml) set of files. Both can be installed into 2.504.1 LTS jenkins, but the new list of plugins runs significantly slower.

The performance problem is most easily demonstrated demonstrating running with a simple pipeline:

pipeline {

   agent any

   stages {
       stage('Stage1') {
           steps {
               echo 'Hello World'
           }
       }
       stage('Stage2') {
           steps {
               echo 'Hello World'
           }
       }
       stage('Stage3') {
           steps {
               echo 'Hello World'
           }
       }
   }
}


With a newer list of plugins (see attachment), a Jenkins job defined with the code directly in the job takes roughly 10 seconds to execute, where the pipeline view shows each stage taking 3-4 seconds to run.

With the older list of plugins (see attachment), a Jenkins job setup the same way takes well under 1 second to execute, where the pipeline view shows roughly 100 ms per stage to execute.

First noticed this issue while trying to standard pipeline code from SCM using a shared global library. After much troubleshooting and disabling many extraneous plugins in a fresh install, was able to come up with the list above. After digging and bumping log levels and comparing release notes on various items, I was unable to come up with the root cause of this problem.

In environments where others are using it, I will be holding off on upgrade until we can get a fix because the performance impact at scale is impactful. At this time, have not truly isolated the specific plugins impacted.

Some other notes:



  • When using pipeline from SCM (used in tandem with cloudbees-bitbucket-branch-source and an organization job) , the first build for a branch is significantly faster than subsequent builds

    • Example: First build can be 17 seconds whereas 2nd, 3rd, 4th, etc. builds take closer to 2 minutes to complete



  • Freestyle jobs continue to execute just fine

  • Imperative jobs execute much faster than declarative, but still likely slower than ideal

    • Saw roughly 1 second per stage instead of 3-4 seconds per stage, but much higher than the ~100ms per stage given above example code



  • Changing pipeline durability (performance or otherwise) does not have a major impact the overall performance of the job

---
Originally reported by sweecm, imported from: Significant Declarative Pipeline Performance Issues


  • status: Open
  • priority: Major
  • component(s): authorize-project-plugin
  • resolution: Unresolved
  • votes: 0
  • watchers: 4
  • imported: 20260312-112651

Raw content of original issue

Attached an old list of plugins (old-plugins-to-install.yaml) and a new list of plugins (new-plugins-to-install.yaml) set of files. Both can be installed into 2.504.1 LTS jenkins, but the new list of plugins runs significantly slower.

The performance problem is most easily demonstrated demonstrating running with a simple pipeline:



pipeline {

   agent any

   stages {
       stage('Stage1') {
           steps {
               echo 'Hello World'
           }
       }
       stage('Stage2') {
           steps {
               echo 'Hello World'
           }
       }
       stage('Stage3') {
           steps {
               echo 'Hello World'
           }
       }
   }
}



With a newer list of plugins (see attachment), a Jenkins job defined with the code directly in the job takes roughly 10 seconds to execute, where the pipeline view shows each stage taking 3-4 seconds to run.

With the older list of plugins (see attachment), a Jenkins job setup the same way takes well under 1 second to execute, where the pipeline view shows roughly 100 ms per stage to execute.

First noticed this issue while trying to standard pipeline code from SCM using a shared global library. After much troubleshooting and disabling many extraneous plugins in a fresh install, was able to come up with the list above. After digging and bumping log levels and comparing release notes on various items, I was unable to come up with the root cause of this problem.

In environments where others are using it, I will be holding off on upgrade until we can get a fix because the performance impact at scale is impactful. At this time, have not truly isolated the specific plugins impacted.

Some other notes:



  • When using pipeline from SCM (used in tandem with cloudbees-bitbucket-branch-source and an organization job) , the first build for a branch is significantly faster than subsequent builds

    • Example: First build can be 17 seconds whereas 2nd, 3rd, 4th, etc. builds take closer to 2 minutes to complete



  • Freestyle jobs continue to execute just fine

  • Imperative jobs execute much faster than declarative, but still likely slower than ideal

    • Saw roughly 1 second per stage instead of 3-4 seconds per stage, but much higher than the ~100ms per stage given above example code



  • Changing pipeline durability (performance or otherwise) does not have a major impact the overall performance of the job


environment

```
Jenkins 2.504.1 LTS JDK21 installed in Kubernetes

Installed 5.8.49 jenkins helm chart

Plugins installed via jenkins-plugin-cli in base container extending jenkins 2.504.1-lts-jdk21 container image
```

12 attachments

- [image-2025-05-27-22-39-56-988.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-last/refs/heads/main/attachments/64444/image-2025-05-27-22-39-56-988.png)
> ![image-2025-05-27-22-39-56-988.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-last/refs/heads/main/attachments/64444/image-2025-05-27-22-39-56-988.png)
- [image-2025-05-27-22-41-31-371.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-last/refs/heads/main/attachments/64445/image-2025-05-27-22-41-31-371.png)
> ![image-2025-05-27-22-41-31-371.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-last/refs/heads/main/attachments/64445/image-2025-05-27-22-41-31-371.png)
- [image-2025-05-28-08-33-22-589.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64446/image-2025-05-28-08-33-22-589.png)
> ![image-2025-05-28-08-33-22-589.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64446/image-2025-05-28-08-33-22-589.png)
- [image-2025-05-28-08-42-01-019.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64449/image-2025-05-28-08-42-01-019.png)
> ![image-2025-05-28-08-42-01-019.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64449/image-2025-05-28-08-42-01-019.png)
- [new-plugins-RHEL-8.10.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64441/new-plugins-RHEL-8.10.png)
> ![new-plugins-RHEL-8.10.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64441/new-plugins-RHEL-8.10.png)
- [new-plugins-performance.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64438/new-plugins-performance.png)
> ![new-plugins-performance.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64438/new-plugins-performance.png)
- [new-plugins-to-install.yaml](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64437/new-plugins-to-install.yaml)
- [old-plugins-RHEL-8.10.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64442/old-plugins-RHEL-8.10.png)
> ![old-plugins-RHEL-8.10.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64442/old-plugins-RHEL-8.10.png)
- [old-plugins-performance.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64439/old-plugins-performance.png)
> ![old-plugins-performance.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64439/old-plugins-performance.png)
- [old-plugins-to-install.yaml](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-last/refs/heads/main/attachments/64436/old-plugins-to-install.yaml)
- [run-jenkins.sh](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-last/refs/heads/main/attachments/64440/run-jenkins.sh)
- [test-job-ldap-enabled.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64443/test-job-ldap-enabled.png)
> ![test-job-ldap-enabled.png](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-misc/refs/heads/main/attachments/64443/test-job-ldap-enabled.png)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the simple three-stage Declarative Pipeline on Jenkins 2.504.1 LTS with JDK 21, using old-plugins-to-install.yaml and new-plugins-to-install.yaml and the supplied run-jenkins.sh. Compare plugin behavior and logs while checking the attached performance results, including the SCM/shared-library case. Done means the responsible plugin or interaction is isolated and Declarative Pipeline execution no longer shows the reported multi-second stage delays.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kubernetes
Domain
devops, performance, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.