[JENKINS-41722] Job folder loading performance scalability
- Dominant language
- Java
- Stars
- 26.5k
- Forks
- 9.8k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 54
Description
I have a tree-like structure job layout as below:
"Projects" Cloudbees folder
contains 34 subproject folder
subproject: "Project-A" Cloudbees folder
contains 6 branch folders ( 5 multibranch job folder and 1 cloudbees folder )
multibranch-job1
contains 295 matrix jobs (75 x 2 matrix combination)
multibranch-job2
contains 317 matrix jobs (75 x 2 matrix combination)
multibranch-job3
contains 189 matrix jobs (75 x 2 matrix combination)
multibranch-job4
contains 15 matrix jobs (75 x 2 matrix combination)
multibranch-job5
contains 5 matrix jobs (75 x 2 matrix combination)
Cloudbees folder
contains 4 matrix jobs (75 x 2 matrix combination)
subproject: Project-B Cloudbees folder
....
multibranch-job1
...
...
...
...
...
...
I found somehow no matter what kind of authorization strategy I choose,
it's always taking longer time
to open "Project-A" cloudbees folder (contains 6 items)
than "Projects" folder (contains 34 items).
Below is my benchmark:
Chrome
Chrome
Firefox
Firefox
Authorization Method
Open Projects Folder
Open Project-A Folder
Open Projects Folder
Open Project-A Folder
Anyone can do anything
2.10 sec
8.43 sec
5.99 sec
17.05 sec
Logged-in users can do anything
3.97 sec
9.94 sec
7.97 sec
20.06 sec
Initial default Role-based configuration - admin role
8.22 sec
14.54 sec
16.47 sec
28.55 sec
Role-based configuration - admin role with 119 Project roles
18.81 sec
31.95 sec
37.7 sec
65
I'm expecting it should be faster to open "Project-A" folder because it needs to process permission and load only 6 items,
and slower to open "Projects" folder because it needs to process permissions and load for 34 items.
---
Originally reported by totoroliu, imported from: Job folder loading performance scalability
Raw content of original issue
I have a tree-like structure job layout as below:
"Projects" Cloudbees folder
contains 34 subproject folder
subproject: "Project-A" Cloudbees folder
contains 6 branch folders ( 5 multibranch job folder and 1 cloudbees folder )
multibranch-job1
contains 295 matrix jobs (75 x 2 matrix combination)
multibranch-job2
contains 317 matrix jobs (75 x 2 matrix combination)
multibranch-job3
contains 189 matrix jobs (75 x 2 matrix combination)
multibranch-job4
contains 15 matrix jobs (75 x 2 matrix combination)
multibranch-job5
contains 5 matrix jobs (75 x 2 matrix combination)
Cloudbees folder
contains 4 matrix jobs (75 x 2 matrix combination)
subproject: Project-B Cloudbees folder
....
multibranch-job1
...
...
...
...
...
...I found somehow no matter what kind of authorization strategy I choose,
it's always taking longer time
to open "Project-A" cloudbees folder (contains 6 items)
than "Projects" folder (contains 34 items).Below is my benchmark:
Chrome
Chrome
Firefox
FirefoxAuthorization Method
Open Projects Folder
Open Project-A Folder
Open Projects Folder
Open Project-A FolderAnyone can do anything
2.10 sec
8.43 sec
5.99 sec
17.05 secLogged-in users can do anything
3.97 sec
9.94 sec
7.97 sec
20.06 secInitial default Role-based configuration - admin role
8.22 sec
14.54 sec
16.47 sec
28.55 secRole-based configuration - admin role with 119 Project roles
18.81 sec
31.95 sec
37.7 sec
65I'm expecting it should be faster to open "Project-A" folder because it needs to process permission and load only 6 items,
and slower to open "Projects" folder because it needs to process permissions and load for 34 items.
environment
```
Ubuntu 14.04.5 LTS
OpenJDK 8u111
Jenkins v2.32.1
Folders Plugin v5.16
Multi-Branch Project Plugin v0.5.1
Role-based Authorization Strategy v2.3.2
Intel Xeon E5-2630
24 GB RAM
500 GB SSD x 5 (RAID-5)
1897 jobs in total on the server – (1799 Matrix jobs)
Role Strategy Project Role: 119
Total Users: 464 users
Client tested on Ubuntu 16.04 (64-bit)
Chromium-browser 55.0.2883.87
Firefox 50.1.0
```
10 attachments
- [jenkins-open-folder.tar.gz](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36132/jenkins-open-folder.tar.gz)
- [jenkins-startup-repeat01-0627.log](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36115/jenkins-startup-repeat01-0627.log)
- [jenkins-startup-repeat02-0628.log](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36116/jenkins-startup-repeat02-0628.log)
- [jenkins-startup-repeat03-0629.log](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36117/jenkins-startup-repeat03-0629.log)
- [jenkins-startup-repeat04-0630.log](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36118/jenkins-startup-repeat04-0630.log)
- [jenkins-startup-repeat05-0631.log](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36119/jenkins-startup-repeat05-0631.log)
- [jenkins-startup.log.gz](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36114/jenkins-startup.log.gz)
- [jenkins-startup.tar.gz](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36131/jenkins-startup.tar.gz)
- [jenkins_open_AC_folder.log.tgz](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36146/jenkins_open_AC_folder.log.tgz)
- [jenkins_open_AC_folder_Current_Thread.log](https://raw.githubusercontent.com/jenkinsci/attachments-from-jira-issues-core-cli/refs/heads/main/attachments/36147/jenkins_open_AC_folder_Current_Thread.log)
Contributor guide
Assessment
This issue has not been assessed yet.