[JENKINS-37345] Throttle automatic git checkout to prevent timeouts/overload on first indexing

Problem

I have a huge repository (~500MB) with many branches (~30) with each containing a Jenkinsfile. The first indexing is quite painful.

Jenkins will detect all branches, and trigger a build for each one. Each build will start by cloning the git repository on the master to fetch the Jenkinsfile. But this tasks will not use/require a free executor on the master. So, the 30 git clone will be launched at the same time, trying to download 30 times the same 500MB repository. It will end with a really slow Jenkins and a timeout.

I'll have to manually relaunch every branch, one or two at a time to fix this problem.

Possible solution

Is it possible to use an executor on the master during the checkout ? To throttle the git clones ?

How to reproduce it

• Create a git repository


• Create a Jenkinsfile with something in it (echo 'hello world')


• Add a huge file (an ISO of your favorite Linux distribution or something else)


• Commit it


• Create 30 branches


• Add this project in your Jenkins as a Multibranch Pipeline.

---
Originally reported by superboum, imported from: Throttle automatic git checkout to prevent timeouts/overload on first indexing

• status: Reopened
  
• priority: Major
  
• component(s): workflow-multibranch-plugin
  
• resolution: Unresolved
  
• votes: 4
  
• watchers: 7
  
• imported: 20251212-090250
  

Raw content of original issue

Problem


I have a huge repository (~500MB) with many branches (~30) with each containing a Jenkinsfile. The first indexing is quite painful.


Jenkins will detect all branches, and trigger a build for each one. Each build will start by cloning the git repository on the master to fetch the Jenkinsfile. But this tasks will not use/require a free executor on the master. So, the 30 git clone will be launched at the same time, trying to download 30 times the same 500MB repository. It will end with a really slow Jenkins and a timeout.


I'll have to manually relaunch every branch, one or two at a time to fix this problem.


Possible solution


Is it possible to use an executor on the master during the checkout ? To throttle the git clones ?


How to reproduce it




	
Create a git repository

	
Create a Jenkinsfile with something in it (echo 'hello world')

	
Add a huge file (an ISO of your favorite Linux distribution or something else)

	
Commit it

	
Create 30 branches

	
Add this project in your Jenkins as a Multibranch Pipeline.