actions / actions/runner-container-hooks

k8s 0.8.x: hashFiles() always returns an empty string, silently freezing every actions/cache key

Open
#448 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
153
Forks
112
Avg merge
6m
Merged PRs (30d)
1

Description

Summary

In kubernetes-novolume mode, and in any runner image built from main, ${{ hashFiles(...) }} always evaluates to the empty string — for every pattern, including a file that certainly exists such as README.md.

Nothing fails. The job stays green. But every cache key derived from hashFiles becomes a constant, and actions/cache writes only on an exact-key miss. So the entry is written once and never updates again.

Affected versions

Hook Workspace hashFiles()
0.7.0 shared volume works
0.8.0, 0.8.1, current main per-pod emptyDir + exec cp returns ""

actions/runner's images/Dockerfile pins RUNNER_CONTAINER_HOOKS_VERSION=0.7.0 for /home/runner/k8s and v0.8.1 for /home/runner/k8s-novolume. So users reach this by opting into containerMode: kubernetes-novolume, or by building a runner image from main.

Reproduction

Two otherwise identical scale sets on one cluster — same chart (gha-runner-scale-set 0.14.2), same pod template. The only difference is the hook bundle at /home/runner/k8s/index.js.

- name: Probe hashFiles
  run: echo "hashFiles(README.md) = '${{ hashFiles('README.md') }}'"
Hook Output
0.7.0 (stock image) a 64-character digest
main empty string

Every other pattern we tried behaved the same way.

Root cause

hashFiles() is an expression, so the runner evaluates it before the step runs — not the workflow pod.

Since #244 the workflow pod's /__w is a pod-local emptyDir:

// packages/k8s/src/k8s/index.ts
appPod.spec.volumes = [
  { name: WORK_VOLUME, emptyDir: {} },
  ...
]

actions/checkout runs as a step, so the repo lands in that emptyDir. runScriptStep copies only _temp in, and only _temp/_runner_file_commands back:

// packages/k8s/src/hooks/run-script-step.ts
await execCpToPod(state.jobPod, runnerTemp, containerTempSrc)
// ...
await execCpFromPod(state.jobPod, `${containerTemp}/_runner_file_commands`, `${workdir}/_temp`)

The workspace is never copied back, so the runner's $GITHUB_WORKSPACE stays empty and every glob matches zero files.

Same root cause as #280, reported there from the other end: "Looking at the runner, /home/runner/_work/<repo>/<repo> directory is empty." #299 / #300 (event.json) and #347 are further symptoms.

Why this one needs separate tracking

Every other symptom of this root cause is a hard error somebody notices — #280 Can't find 'action.yml', #299 / #300 missing $GITHUB_EVENT_PATH, #423 PrepareJob failure.

hashFiles() returning "" is a legal result. There is no warning. A workflow using

key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
restore-keys: npm-${{ runner.os }}-

pins itself to a cache entry written on its first ever run. And via the restore-keys prefix, that frozen entry can also be restored by jobs running on healthy 0.7.0 runners in the same repository.

How we hit it

We are affected by #228. #333 is merged but unreleased, so we built a runner image from main to pick it up. That also moved us from the 0.7.0 shared-volume workspace to the 0.8.x exec-cp one — our scale set still declares containerMode: kubernetes with a work volume claim, which the main hook ignores.

So the architecture change is easy to import unintentionally while chasing an unreleased fix.

Suggestion

#339 reverts to volumes and would fix this. Two smaller asks meanwhile:

  1. Document the constraint in packages/k8s/README.md. Under the exec-cp implementation the runner cannot see the workspace, so hashFiles(), local composite actions, and anything else resolved against GITHUB_WORKSPACE do not work. The README currently documents the opposite — "The runner pod should map a persistent volume claim into the _work directory".
  2. Consider whether hashFiles() matching zero files should warn instead of returning "". That is actions/runner territory, but the silence is what makes this dangerous.

Root-cause analysis assisted by Claude. The A/B measurement, the code references and the version pins were verified by hand.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/k8s/README.md and the referenced workspace setup in packages/k8s/src/k8s/index.ts and packages/k8s/src/hooks/run-script-step.ts. Reproduce the README.md hashFiles case with the 0.8.x or main hook and compare it with 0.7.0. Done means the workspace constraint is accurately documented, or the reproduction no longer returns an empty hash after the workspace handling is addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, kubernetes, typescript
Domain
ci-cd, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.