AutoscalingRunnerSet deletion wedges forever when githubConfigSecret is gone

Checks

• I've already read https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/troubleshooting-actions-runner-controller-errors and I'm sure my issue is not covered in the troubleshooting guide.
• I am using charts that are officially provided

Controller Version

0.14.2 (gha-runner-scale-set-controller); same code on master at 03328aa14f6435462f6a2add1f5ba79247da6e09

Deployment Method

Helm

Checks

• This isn't a question or user support case (For Q&A and community support, go to Discussions).
• I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

1. Install a gha-runner-scale-set release with minRunners: 0 and wait until the AutoscalingRunnerSet (ARS) carries the runner-scale-set-id annotation, i.e. the scale set is registered with the Actions service.
2. Remove the configured secret: kubectl delete secret <githubConfigSecret> -n <ns> (drop its autoscalingrunnerset.actions.github.com/cleanup-protection finalizer first if the chart created it).
3. kubectl delete autoscalingrunnerset <name> -n <ns> (or helm uninstall, or delete the namespace).
4. The ARS stays in Terminating indefinitely. The controller logs, on every requeue:

ERROR AutoscalingRunnerSet Failed to initialize Actions service client for updating a existing runner scale set {"error": "failed to resolve app config: failed to get kubernetes secret: \"<ns>/<githubConfigSecret>\""}
ERROR AutoscalingRunnerSet Failed to delete runner scale set
ERROR AutoscalingRunnerSet Failed to clean up resources during deletion

The finalizer autoscalingrunnerset.actions.github.com/finalizer is never removed, so kubectl delete / helm uninstall hang and a namespace containing the ARS can never finish terminating. Only manual intervention (kubectl patch ... --type=merge -p '{"metadata":{"finalizers":null}}') or recreating a secret with the old name unblocks it.

The same happens whenever the secret is gone for good rather than temporarily: the namespace is being torn down and the secret went first, the secret was rotated to a new name and the old one deleted before the ARS, or Helm removed the secret before the ARS during uninstall.

Describe the bug

During deletion, Reconcile calls cleanUpResources, which ends with deleteRunnerScaleSet:

• deleteRunnerScaleSet builds an Actions client from githubConfigSecret via GetActionsService and returns the error if that fails: https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/autoscalingrunnerset_controller.go#L768-L772
• cleanUpResources propagates it: https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/autoscalingrunnerset_controller.go#L446-L450
• Reconcile returns the error without touching the finalizer: https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/autoscalingrunnerset_controller.go#L86-L89

A missing secret is a permanent condition, so the object is requeued forever with controller-runtime's exponential backoff. With many such objects (e.g. a namespace holding thousands of ARS objects being torn down) the workqueue fills with long-delayed items and every other reconcile in the controller slows down.

The deletion path already treats the two neighbouring "nothing to deregister" cases as terminal:

• If the runner-scale-set-id annotation is absent, deleteRunnerScaleSet returns nil, documenting that "manual deletion of the scale set is required": https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/autoscalingrunnerset_controller.go#L742-L757
• If the Actions service reports the scale set as already gone (scaleset.NotFoundError), it logs and continues to release the finalizer (added in #4571): https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/autoscalingrunnerset_controller.go#L775-L779
• removeGitHubSecretFinalizer tolerates NotFound/Forbidden on the very same secret ("GitHub secret has already been deleted"): https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/autoscalingrunnerset_controller.go#L1140-L1142

Only the "secret is NotFound" case blocks forever.

One detail for the fix: the resolver currently discards the underlying Kubernetes error. k8sResolver.appConfig formats it with %q of the key rather than wrapping the error (https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/secretresolver/secret_resolver.go#L221-L231), and GetActionsService wraps with %v (https://github.com/actions/actions-runner-controller/blob/03328aa14f6435462f6a2add1f5ba79247da6e09/controllers/actions.github.com/secretresolver/secret_resolver.go#L70-L79). So kerrors.IsNotFound(err) cannot currently see through the chain; both sites need %w (or a typed sentinel) before the controller can distinguish NotFound from transient failures.

Related but distinct: #4093 covers a secret that never existed (the ARS never registered, so the annotation-absent guard above already releases the ARS finalizer; the remaining wedge there is on EphemeralRunner, addressed by #4619). This report is about an ARS that did register and whose secret was removed afterwards, which neither of those covers.

Describe the expected behavior

When githubConfigSecret is definitively NotFound during deletion, the controller should:

1. log at Error level that the runner scale set (with its id) could not be deregistered from the Actions service and must be removed manually, matching the contract already documented for the missing-annotation case;
2. remove the runner-scale-set-id annotation;
3. return nil so the finalizer is released and the object, Helm release and namespace can finish deleting.

Transient errors (API server unavailable, 5xx from the Actions service, Forbidden on the secret while RBAC is being torn down) should keep returning the error and requeueing as today. Optionally emit a Kubernetes Event so the leaked scale set is visible.

Sketch of the change in deleteRunnerScaleSet:

actionsClient, err := r.GetActionsService(ctx, autoscalingRunnerSet)
switch {
case kerrors.IsNotFound(err): // requires %w in secretresolver
    logger.Error(err, "GitHub config secret no longer exists; runner scale set cannot be deregistered and must be deleted manually from the Actions service", "runnerScaleSetId", runnerScaleSetID)
    // fall through to the annotation removal below and return nil
case err != nil:
    logger.Error(err, "Failed to initialize Actions service client for updating a existing runner scale set")
    return err
default:
    // existing DeleteRunnerScaleSet call
}

Test: in autoscalingrunnerset_controller_test.go, create an ARS with the runner-scale-set-id annotation and a valid secret, delete the secret, delete the ARS, and Eventually assert the ARS is gone and the fake client's DeleteRunnerScaleSet was not called; a second case with a non-NotFound secret error asserts the ARS remains Terminating.

I am happy to open a PR with the envtest case if maintainers agree that releasing the finalizer is the right contract here.

Additional Context

# gha-runner-scale-set values (nothing unusual)
githubConfigUrl: https://github.com/<org>
githubConfigSecret: <name of a pre-created secret>
minRunners: 0
maxRunners: 5

Controller Logs

# repeated on every requeue, with growing backoff, until the finalizer is patched away by hand:
INFO  AutoscalingRunnerSet Deleting resources
INFO  AutoscalingRunnerSet Deleting the listener
INFO  AutoscalingRunnerSet Listener is deleted
INFO  AutoscalingRunnerSet deleting ephemeral runner sets
INFO  AutoscalingRunnerSet Ephemeral runner set is deleted
INFO  AutoscalingRunnerSet deleting runner scale set
INFO  AutoscalingRunnerSet Deleting the runner scale set from Actions service
ERROR AutoscalingRunnerSet Failed to initialize Actions service client for updating a existing runner scale set {"error": "failed to resolve app config: failed to get kubernetes secret: \"<ns>/<githubConfigSecret>\""}
ERROR AutoscalingRunnerSet Failed to delete runner scale set
ERROR AutoscalingRunnerSet Failed to clean up resources during deletion
ERROR Reconciler error {"controller": "autoscalingrunnerset", ... , "error": "failed to resolve app config: failed to get kubernetes secret: \"<ns>/<githubConfigSecret>\""}

Runner Pod Logs

N/A - no runner pods exist at this point (minRunners: 0; EphemeralRunnerSet already deleted before the wedge).

Analysis prepared with an AI agent operated by KR-Ravindra, who verified the code paths.

Start in controllers/actions.github.com/autoscalingrunnerset_controller.go, especially deleteRunnerScaleSet and the deletion path, then inspect controllers/actions.github.com/secretresolver/secret_resolver.go for error wrapping. Read autoscalingrunnerset_controller_test.go and add the described envtest cases for a missing secret and a transient error. Done means a definitively missing secret releases the ARS finalizer without calling DeleteRunnerScaleSet, while transient errors still requeue.