actions / actions/runner-container-hooks

Missing directories in workflow pods when running selfhosted runners in kubernetes-novolume mode

Open
#347 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
153
Forks
112
Avg merge
6m
Merged PRs (30d)
1

Description

We have been using self-hosted runners managed by ARC in our Kubernetes cluster for our GHA workflow needs. In our current setup of ARC we are configuring Kubernetes mode with persistent volumes by providing below values to the helm chart:

containerMode:
  type: "kubernetes"
  kubernetesModeWorkVolumeClaim:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 3Gi

Our plan is to migrate to a kubernetes-novolume mode that was introduced by https://github.com/actions/actions-runner-controller/pull/4250, but we are hitting a blocker at the moment when we are configuring ARC with below values to it's helm chart :

containerMode:
  type: "kubernetes-novolume"

The blocker is as follows:
Any marketplace actions like actions/checkout that are Javascript based and that need node binaries at path /__e/ are failing with error as :
[0](https://github.com/Egencia/orawli-test/actions/runs/25205597320/job/73905379626#step:6:22) env: can't execute '/__e/node24_alpine/bin/node': No such file or directory.

On investigation it was found that Node binaries are available in "Kubernetes volume" mode because of the following volumeMount on the workflow pod:

    - mountPath: /__e
      name: work
      subPath: externals 

In "Kubernetes volume" mode node binaries are available at runner pod's source path /home/runner/_work/externals which are then shared with workflow pod through the above volume mount, but in "Kubernetes novolume mode" they are not copied over by the hook, though they are available in the runner pod at a different path: /home/runner/externals
The custom container image that we have been using to create the workflow pods has node binaries in it but that path can't be overridden to run JS marketplace actions.
Another directory that is missing from the workflow pod is /github, though the subdirectories such as workflow and home which should be there inside /github dir are present at /__w/_temp/_github_* in the same pod. As a result files such as /github/workflow/event.json that are referred by $GITHUB_EVENT_PATH are missing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the kubernetes-novolume hook handling and compare it with Kubernetes volume mode, especially how /home/runner/externals and the /github paths reach workflow pods. Reproduce the failure with actions/checkout and verify that /__e/node24_alpine/bin/node and /github/workflow/event.json are available in the workflow pod.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, kubernetes, node.js, typescript
Domain
ci-cd, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.