jenkinsci / jenkinsci/github-branch-source-plugin

[JENKINS-54495] Better handling of GitHub Organization folder scan to avoid API quota

Open
#1,221 3 comments 0 reactions 0 assignees View on GitHub
component:github-branch-source-plugin enhancement imported-jira-issue performance priority:major resolution:unresolved ux
Dominant language
Java
Stars
217
Forks
398
Avg merge
30m
Merged PRs (30d)
1

Description

When having a big GitHub organization, with hundreds of repos, each with hundreds of branches and tags, refreshing the whole organization is not possible (or it takes ages) due to GitHub API quota being hit.

This is particularly bad when trying to add a new repo, it could take days, which is completely impractical.

There are several solutions to this issue that I can think of:



  • Use GitHub GraphQL API to query the whole thing in one (or very few) request(s)

  • Make a "shallow scan", that only discovers repos. Then each repo can be refreshed separately, which can 1. enable the quick addition of new repos and 2. distribute the refresh API bursts in time making hitting the API quota less likely

  • Add a separate function to only discover one repo specified by the user

---
Originally reported by lucasocio, imported from: Better handling of GitHub Organization folder scan to avoid API quota


  • status: Open
  • priority: Major
  • component(s): github-branch-source-plugin
  • label(s): api, github, performance, quota, user-experience
  • resolution: Unresolved
  • votes: 4
  • watchers: 7
  • imported: 2025-12-02

Raw content of original issue

When having a big GitHub organization, with hundreds of repos, each with hundreds of branches and tags, refreshing the whole organization is not possible (or it takes ages) due to GitHub API quota being hit.

This is particularly bad when trying to add a new repo, it could take days, which is completely impractical.

There are several solutions to this issue that I can think of:



  • Use GitHub GraphQL API to query the whole thing in one (or very few) request(s)

  • Make a "shallow scan", that only discovers repos. Then each repo can be refreshed separately, which can 1. enable the quick addition of new repos and 2. distribute the refresh API bursts in time making hitting the API quota less likely

  • Add a separate function to only discover one repo specified by the user


environment

```
Jenkins ver. 2.138.2

GitHub Branch Source Plugin 2.4.1
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.