actions / actions/actions-runner-controller

Add Warning Log for High Pagination Size in Runner Controller to Prevent GitHub API Rate Limiting Issues

Open
#3,001 1 comment 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

community enhancement needs triage
Dominant language
Go
Stars
6.5k
Forks
1.5k
Avg merge
2d 2h
Merged PRs (30d)
27

Description

What would you like added?

I would like a warning message to be added to the logs of the GitHub Action Runner Controller. This warning should notify that the Runner Controller has a high pagination size when it's contacting the GitHub API.

Why is this needed?

We recently faced a production incident where the Runner Controller kept spawning new runners on our cluster. It was continually attempting to contact GitHub to check if the new runner had registered. Due to the high pagination size and the presence of offline runners in the GitHub API, we hit the rate limit, causing operational issues.

Additional context

This issue is critical as it led to a production incident and could potentially affect others who might be in a similar setup. Adding a warning message will serve as an alert to preemptively catch this condition and help avoid hitting the API rate limit. Diagnosing the root cause of this issue was time-consuming. A simple log message would have saved us a lot of time. Specifically, we had to set the log level to at least -3 to notice that the controller was experiencing a rate limit when contacting GitHub.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the Runner Controller code that contacts the GitHub API and handles pagination, then inspect its existing logging. Add a warning for a high pagination size that helps explain possible rate-limit conditions, and verify that the warning appears at the relevant log level.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, go, kubernetes
Domain
api, devops, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.