Invalid values for the metrics gha_registered_runners and gha_idle_runners in ghalistener

Checks

• I've already read https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/troubleshooting-actions-runner-controller-errors and I'm sure my issue is not covered in the troubleshooting guide.
• I am using charts that are officially provided

Controller Version

0.9.2

Deployment Method

Helm

Checks

• This isn't a question or user support case (For Q&A and community support, go to Discussions).
• I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

-

Describe the bug

We have a GitHub Action that runs once a day. A special type of runners is allocated specifically for it. During the execution of the GitHub Action, we receive the latest batch of messages about task execution. In this message, the statistics.totalIdleRunners and statistics.totalRegisteredRunners contain non-zero values.

These values are published by the controller as a prometheus metrics. After this last message, the metric values do not change until the next runner execution the following day.

Is it possible to fix this behavior, or does it require changes on the GitHub side?

Describe the expected behavior

The value of the Prometheus metrics ghalistener should reflect the actual state of the runners.

Additional Context

-

Controller Logs

2024-05-26T07:24:29Z	INFO	listener-app.listener	Getting next message	{"lastMessageID": 1089}
2024-05-26T07:24:37Z	INFO	listener-app.listener	Processing message	{"messageId": 1090, "messageType": "RunnerScaleSetJobMessages"}
2024-05-26T07:24:37Z	INFO	listener-app.listener	New runner scale set statistics.	{"statistics": {"totalAvailableJobs":0,"totalAcquiredJobs":3,"totalAssignedJobs":3,"totalRunningJobs":3,"totalRegisteredRunners":4,"totalBusyRunners":3,"totalIdleRunners":0}}
2024-05-26T07:24:37Z	INFO	listener-app.listener	Job completed message received.	{"RequestId": 669571, "Result": "succeeded", "RunnerId": 83622, "RunnerName": "terraform-drift-checker-hxppm-runner-9q2hx"}
2024-05-26T07:24:37Z	INFO	listener-app.listener	Deleting last message	{"lastMessageID": 1090}
2024-05-26T07:24:38Z	INFO	listener-app.worker.kubernetesworker	Calculated target runner count	{"assigned job": 3, "decision": 3, "min": 0, "max": 30, "currentRunnerCount": 3, "jobsCompleted": 1}
2024-05-26T07:24:38Z	INFO	listener-app.worker.kubernetesworker	Compare	{"original": "{\"metadata\":{\"creationTimestamp\":null},\"spec\":{\"replicas\":-1,\"patchID\":-1,\"ephemeralRunnerSpec\":{\"metadata\":{\"creationTimestamp\":null},\"spec\":{\"containers\":null}}},\"status\":{\"currentReplicas\":0,\"pendingEphemeralRunners\":0,\"runningEphemeralRunners\":0,\"failedEphemeralRunners\":0}}", "patch": "{\"metadata\":{\"creationTimestamp\":null},\"spec\":{\"replicas\":3,\"patchID\":6917,\"ephemeralRunnerSpec\":{\"metadata\":{\"creationTimestamp\":null},\"spec\":{\"containers\":null}}},\"status\":{\"currentReplicas\":0,\"pendingEphemeralRunners\":0,\"runningEphemeralRunners\":0,\"failedEphemeralRunners\":0}}"}
2024-05-26T07:24:38Z	INFO	listener-app.worker.kubernetesworker	Preparing EphemeralRunnerSet update	{"json": "{\"spec\":{\"patchID\":6917,\"replicas\":3}}"}
2024-05-26T07:24:38Z	INFO	listener-app.worker.kubernetesworker	Ephemeral runner set scaled.	{"namespace": "github-actions-runner", "name": "terraform-drift-checker-hxppm", "replicas": 3}
2024-05-26T07:24:38Z	INFO	listener-app.listener	Getting next message	{"lastMessageID": 1090}
2024-05-26T07:24:50Z	INFO	listener-app.listener	Processing message	{"messageId": 1091, "messageType": "RunnerScaleSetJobMessages"}
2024-05-26T07:24:50Z	INFO	listener-app.listener	New runner scale set statistics.	{"statistics": {"totalAvailableJobs":0,"totalAcquiredJobs":0,"totalAssignedJobs":0,"totalRunningJobs":0,"totalRegisteredRunners":2,"totalBusyRunners":0,"totalIdleRunners":1}}
2024-05-26T07:24:50Z	INFO	listener-app.listener	Job completed message received.	{"RequestId": 669572, "Result": "succeeded", "RunnerId": 83625, "RunnerName": "terraform-drift-checker-hxppm-runner-6dmvl"}
2024-05-26T07:24:50Z	INFO	listener-app.listener	Job completed message received.	{"RequestId": 669573, "Result": "succeeded", "RunnerId": 83623, "RunnerName": "terraform-drift-checker-hxppm-runner-lcc6k"}
2024-05-26T07:24:50Z	INFO	listener-app.listener	Job completed message received.	{"RequestId": 669574, "Result": "succeeded", "RunnerId": 83624, "RunnerName": "terraform-drift-checker-hxppm-runner-d2bv7"}
2024-05-26T07:24:50Z	INFO	listener-app.listener	Deleting last message	{"lastMessageID": 1091}

Runner Pod Logs

-

Start in the ghalistener path that processes RunnerScaleSetJobMessages and publishes the Prometheus metrics, using the supplied controller logs to compare successive statistics messages. Determine whether metrics are updated after each message or remain cached after runner activity; done means gha_registered_runners and gha_idle_runners reflect the latest runner statistics, with coverage for the observed transition.