jenkinsci / jenkinsci/github-branch-source-plugin
[JENKINS-65725] Too many calls to the GitHub API cause throttling
- Dominant language
- Java
- Stars
- 217
- Forks
- 398
- Avg merge
- 30m
- Merged PRs (30d)
- 1
Description
We have a multibranch pipeline configured with a GitHub App credential. We use the SSH checkout trait with another ssh credential to perform the checkout.
For one repository only, we hit about 1000 calls, excluding our shared library, to the github API with our GitHub App token. Here is the details of these calls for 1h in a single repository. We can see that there is quite a high amount of calls to the /repos/{owner}/{repository}/git/refs/heads/{branch} and /repos/{owner}/{repository} endpoints.
The high amount of calls cause our builds to sleep so we avoid being rate limited by GitHub. I believe Github branch source plugin is making most of these calls. The repo is not configured for polling and it is configured with the Discover All branches strategy.
Our Shared Library is also configured to use the GitHub SCM with a GitHub App and checkout over SSH, but we only have 1 call to /repos/{owner}/{repository} and 1 to /repos/{owner}/{repository}/branches/{branch} per build which seems normal. Although, some caching could be done there as well to further reduce the amount of calls.
---
Originally reported by agaudreault, imported from: Too many calls to the GitHub API cause throttling
Raw content of original issue
We have a multibranch pipeline configured with a GitHub App credential. We use the SSH checkout trait with another ssh credential to perform the checkout.
For one repository only, we hit about 1000 calls, excluding our shared library, to the github API with our GitHub App token. Here is the details of these calls for 1h in a single repository. We can see that there is quite a high amount of calls to the /repos/{owner}/{repository}/git/refs/heads/{branch} and /repos/{owner}/{repository} endpoints.
The high amount of calls cause our builds to sleep so we avoid being rate limited by GitHub. I believe Github branch source plugin is making most of these calls. The repo is not configured for polling and it is configured with the Discover All branches strategy.
Our Shared Library is also configured to use the GitHub SCM with a GitHub App and checkout over SSH, but we only have 1 call to /repos/{owner}/{repository} and 1 to /repos/{owner}/{repository}/branches/{branch} per build which seems normal. Although, some caching could be done there as well to further reduce the amount of calls.
environment
```
Jenkins 2.289
GitHub Api 1.117
GitHub Branch Source 2.9.3
```
1 attachment
- [screenshot-1.png](https://issues.jenkins.io/secure/attachment/54881/screenshot-1.png)
> 
Contributor guide
Research direction
No source files or tests are named. Start by tracing GitHub Branch Source handling of GitHub App requests for branch discovery and repository metadata, then compare the reported endpoint-call patterns with the shared-library behavior. Done means reducing unnecessary calls so the multibranch pipeline no longer approaches GitHub rate limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, java
- Domain
- api, ci-cd, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100