actions / actions/actions-runner-controller
Add Warning Log for High Pagination Size in Runner Controller to Prevent GitHub API Rate Limiting Issues
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.5k
- Forks
- 1.5k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 27
Description
What would you like added?
I would like a warning message to be added to the logs of the GitHub Action Runner Controller. This warning should notify that the Runner Controller has a high pagination size when it's contacting the GitHub API.
Why is this needed?
We recently faced a production incident where the Runner Controller kept spawning new runners on our cluster. It was continually attempting to contact GitHub to check if the new runner had registered. Due to the high pagination size and the presence of offline runners in the GitHub API, we hit the rate limit, causing operational issues.
Additional context
This issue is critical as it led to a production incident and could potentially affect others who might be in a similar setup. Adding a warning message will serve as an alert to preemptively catch this condition and help avoid hitting the API rate limit. Diagnosing the root cause of this issue was time-consuming. A simple log message would have saved us a lot of time. Specifically, we had to set the log level to at least -3 to notice that the controller was experiencing a rate limit when contacting GitHub.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Runner Controller code that contacts the GitHub API and handles pagination, then inspect its existing logging. Add a warning for a high pagination size that helps explain possible rate-limit conditions, and verify that the warning appears at the relevant log level.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go, kubernetes
- Domain
- api, devops, infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100