actions / actions/actions-runner-controller
cgroupv2 is not respecting dockerdContainerResources
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.5k
- Forks
- 1.5k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 27
Description
Resource Definitions
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: ghrunner
spec:
replicas: 1
template:
spec:
dockerdContainerResources:
limits:
cpu: "4"
memory: 1000Mi
requests:
cpu: 50m
memory: 1000Mi
ephemeral: true
image: ""
labels:
- Linux
- X64
- self-hosted
- mylabel
organization: xyz
resources:
limits:
cpu: "4"
memory: 800Mi
requests:
cpu: 50m
memory: 800Mi
To Reproduce
Execute following in a workflow or directly on the runner:
docker info // make sure it has "Cgroup Version: 2" and "Cgroup Driver: cgroupfs"*
docker run -it ubuntu bash
cat /dev/zero | head -c 2000000000 | tail
*which is the default, when dind is started on a kind v0.17.0 (k8s v1.25.3) cluster on Ubuntu 22.04.1 LTS.
Describe the bug
You can consume more memory than in the limits specified.
And kubectl top <runner pod> is missing the memory/cpu of the nested containers.
Describe the expected behavior
The dind container/process should have been killed.
(this works fine if the node has support for cgroup v1, e.g. with kind on ubuntu 18)
EDIT: killing works fine again with newer Docker versions, but kubectl top still shows the wrong data. Please see linked issues.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the workflow on kind v0.17.0 with Kubernetes 1.25.3, Ubuntu 22.04.1, cgroupv2, and the cgroupfs driver; start by checking docker info and kubectl top for the runner pod. Trace how dockerdContainerResources are applied to nested containers. Done means nested container CPU and memory are enforced and included in the runner pod metrics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, kubernetes
- Domain
- infrastructure, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100