actions / actions/actions-runner-controller

When using githubAPICredentialsFrom Controller fails with "authentication failed: using private key of size 0" on startup

Open
#2,994 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug community needs triage
Dominant language
Go
Stars
6.5k
Forks
1.5k
Avg merge
2d 2h
Merged PRs (30d)
27

Description

Checks
Controller Version

0.27.5

Helm Chart Version

0.23.5

CertManager Version

v1.11.0

Deployment Method

Helm

cert-manager installation

Cluster already contains a cert-manager that works fine.

Checks
  • This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
  • I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
  • My actions-runner-controller version (v0.x.y) does support the feature
  • I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
  • I've migrated to the workflow job webhook event (if you using webhook driven scaling)
Resource Definitions
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: arc
  namespace: arc
spec:
  interval: 60m
  releaseName: arc
  chart:
    spec:
      chart: actions-runner-controller
      version: "0.23.5"
      sourceRef:
        kind: HelmRepository
        name: arc
  values:
    githubEnterpriseServerURL: "https://github.company.com/"
    authSecret:
      enabled: false
To Reproduce
1. Install arc as given above.
   (We need to handle multiple github orgs/apps, therefore we do *not* give any credentials here, but use githubAPICredentialsFrom with the actual runners later.)
2. Notice the pod fail.
3. Check the logs to find: `authentication failed: using private key of size 0 (...): could not parse private key: invalid key: Key must be a PEM encoded PKCS1 or PKCS8 key`
Describe the bug

Without any runners I don't expect the controller to connect to github at all.
With runners present I expect the controller to take the credentials from an existing githubAPICredentialsFrom.

For some reason the controller from the very beginning tries to connect to github using some global credentials not present.

Is there any reason to do so?
If no, I would consider this a bug.
If yes, why is that connection needed, and what kind of credentials are needed for this?

Describe the expected behavior

When providing runners for multiple orgs/apps, I expect the credentials to be taken from githubAPICredentialsFrom only, i.e. no credentials needed for the controller itself.

Whole Controller Logs
Defaulted container "manager" out of: manager, kube-rbac-proxy
2023-10-16T12:13:56Z    ERROR   unable to create client {"error": "authentication failed: using private key of size 0 (...): could not parse private key: invalid key: Key must be a PEM encoded PKCS1 or PKCS8 key"}
main.main
        ./main.go:158
runtime.main
        runtime/proc.go:250
Whole Runner Pod Logs
N/A
Additional Context

We are currently running on GHES 3.8.3, so unfortunately we cannot use the new API, yet.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at main.go:158 and reproduce the startup failure with the HelmRelease values shown, using GHES 3.8.3 and no controller-level credentials. Trace why the controller creates a GitHub client before runners provide githubAPICredentialsFrom; done means the intended credential flow is established and the reported startup error is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, go, helm, kubernetes
Domain
authentication, backend, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.