actions / actions/runner-container-hooks
Pod template `imagePullSecrets` silently overwrites `container.credentials` — should union the lists
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 153
- Forks
- 112
- Avg merge
- 6m
- Merged PRs (30d)
- 1
Description
Summary
In the Kubernetes hook, when a job specifies container.credentials and the pod
template (ACTIONS_RUNNER_CONTAINER_HOOK_TEMPLATE) also defines spec.imagePullSecrets,
the template's value completely overwrites the image-pull secret generated from
container.credentials. The user-supplied job credentials are silently discarded — no
warning, no error. The $job container is then pulled using only the template's secret,
which fails with ImagePullBackOff if that secret has no auth entry for the job image's
registry.
imagePullSecrets is additive in Kubernetes (the kubelet unions all entries and matches
per registry host), so these two secrets should be merged (union + dedup by name), the
same way volumes already is — not overwritten.
Environment
actions/runner-container-hooks(k8s hook),main- ARC
gha-runner-scale-set,containerMode: kubernetes/kubernetes-novolume - The workflow pod runs an ARC-injected
fs-initinit container using the runner image
(on registry host A), while the$jobcontainer image is on a different registry
host B provided viacontainer.credentials.
Repro
-
Set a hook template with a pod-level
imagePullSecrets(e.g. to authenticate the
fs-initrunner image on a private mirror, registry host A):spec: imagePullSecrets: - name: infra-pull-secret # covers registry host A (runner/fs-init image) -
Run a job whose container image is on a different private registry (host B), supplying
credentials inline:container: image: registryB.example.com/toolbox:tag credentials: username: ${{ secrets.REG_B_USER }} password: ${{ secrets.REG_B_TOKEN }} -
Observe the created workflow pod:
spec.imagePullSecretscontains only
infra-pull-secret. The secret generated fromcontainer.credentialsis gone, and the
$jobimage pull fails withImagePullBackOff(no auth entry for registry host B).
Root cause
createJobPod sets the container.credentials secret first, then merges the pod-template
extension:
// packages/k8s/src/k8s/index.ts (createJobPod)
if (registry) {
const secret = await createDockerSecret(registry)
const secretReference = new k8s.V1LocalObjectReference()
secretReference.name = secret.metadata.name
appPod.spec.imagePullSecrets = [secretReference] // (1) job creds set here
}
...
if (extension?.spec) {
mergePodSpecWithOptions(appPod.spec, extension.spec) // (2) template merged here
}
mergePodSpecWithOptions only merges containers and volumes; every other field
(including imagePullSecrets) hits the generic overwrite branch:
// packages/k8s/src/k8s/utils.ts
export function mergePodSpecWithOptions(base, from): void {
for (const [key, value] of Object.entries(from)) {
if (key === 'containers') {
base.containers.push(...)
} else if (key === 'volumes' && value) {
base.volumes = mergeLists(base.volumes, value as k8s.V1Volume[])
} else {
base[key] = value // imagePullSecrets clobbered here
}
}
}
So step (2) overwrites the array set in step (1), and container.credentials is lost.
Impact
- Any setup where the runner/
fs-initimage and the$jobimage live on different
private registries cannot authenticate both on the same pod. You are forced to bake a
single multi-host secret into the hook template covering every registry any workflow
might ever use, andcontainer.credentialsbecomes dead config. - The failure is silent: valid, user-provided
container.credentialsare dropped with no
log or warning.
Proposed fix
Treat imagePullSecrets like volumes in mergePodSpecWithOptions — union and dedup by
name rather than overwrite:
} else if (key === 'imagePullSecrets' && value) {
base.imagePullSecrets = mergeLists(
base.imagePullSecrets,
value as k8s.V1LocalObjectReference[]
)
}
Because createJobPod sets the container.credentials secret before the merge, the result
becomes [jobCredSecret, ...templateSecrets]: the kubelet uses the job creds for the
$job image and the template creds for the fs-init / infra image, matching per host.
Why this is safe
The union is a superset — every secret the template declared is still present. The only
behavioral change is that container.credentials is no longer discarded. Kubernetes already
treats imagePullSecrets as additive and dedups host matches, so extra entries are
harmless.
Alternatives considered
- Multi-host secret in the hook template (current workaround): requires enumerating
every registry up front and makescontainer.credentialsinert — unworkable when jobs
may use arbitrary registries. - ServiceAccount
imagePullSecrets: the ServiceAccount admission controller only
injects when the pod has noimagePullSecrets; since the hook already sets one, the SA
entry is ignored — so this cannot supply the second secret.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with mergePodSpecWithOptions in packages/k8s/src/k8s/utils.ts and then inspect createJobPod in packages/k8s/src/k8s/index.ts to confirm the merge order. Verify that generated pods retain the credential secret and template imagePullSecrets, with duplicate names removed; existing Kubernetes hook tests should cover the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, kubernetes, typescript
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100