actions / actions/actions-runner-controller
HRA scale down ignores non-idling Runners
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.5k
- Forks
- 1.5k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 27
Description
Checks
- I've already read https://github.com/actions/actions-runner-controller/blob/master/TROUBLESHOOTING.md and I'm sure my issue is not covered in the troubleshooting guide.
- I'm not using a custom entrypoint in my runner image
Controller Version
0.27.4
Helm Chart Version
0.23.3
CertManager Version
1.7.2
Deployment Method
Helm
cert-manager installation
yes
Checks
- This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
- I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
- My actions-runner-controller version (v0.x.y) does support the feature
- I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
- I've migrated to the workflow job webhook event (if you using webhook driven scaling)
Resource Definitions
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: gh-xx
namespace: ghrunner-xx
spec:
template:
spec:
ephemeral: true
labels:
- Linux
- X64
- self-hosted
- mylabel
organization: myorg
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
name: gh-xx
namespace: ghrunner-x
spec:
maxReplicas: 10
minReplicas: 0
scaleDownDelaySecondsAfterScaleOut: 300
scaleTargetRef:
kind: RunnerDeployment
name: gh-xx
scaleUpTriggers:
- duration: 10m
githubEvent:
workflowJob: {}
To Reproduce
1. Start 1 Github Workflows which sleeps for 30 Minutes
2. Verify:
- Github UI: 1 running job
- HRA desired 1
- runnerdeployment desired 1, current 1, up-to-date 1, available 1
3. Wait 15 Minutes (time larger than hra.spec.scaleUpTriggers.duration):
- Github UI: 1 still running job
- HRA desired 0
- runnerdeployment desired 0, current 1, up-to-date 1, available 1
4. When you now trigger another workflow, then it will be in queuing state for 15 Minutes
- Github UI: 1 running and 1 queuing job
- HRA desired 1
- runnerdeployment desired 1, current 1, up-to-date 1, available 1
Describe the bug
I would expect that desired would include the currently non-idling runners.
Describe the expected behavior
No queuing time
Whole Controller Logs
na
Whole Runner Pod Logs
na
Additional Context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the provided HorizontalRunnerAutoscaler and RunnerDeployment definitions on controller version 0.27.4, using a workflow that sleeps for 30 minutes. Trace the HRA scale-down decision when the running runner is non-idling and compare desired replicas with the active job count. Done means active runners remain covered by the desired count and a newly triggered workflow does not wait for scale-up.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go, kubernetes
- Domain
- ci-cd, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100