opencontainers / opencontainers/runc

Runc fails to run in systemd-nspawn container due to unsual cgroups hierarchy

Open
#3,653 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
13.5k
Forks
2.3k
Avg merge
2d 8h
Merged PRs (30d)
30

Description

systemd-nspawn mounts cgroups in unusual manner, the only writeable part of the cgroupfs is buried deep in the hierarchy:

cgroup /sys/fs/cgroup cgroup2 ro,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/machine.slice/container@docker.service/payload cgroup2 rw,nosuid,nodev,noexec,relatime 0 0

So, runc tries to use a wrong path and fails to run a container:

[root@docker:/mycontainer]# runc --debug run mycontainerid
WARN[0000]libcontainer/process_linux.go:376 libcontainer.(*initProcess).start.func1() unable to get oom kill count                  error="openat2 /sys/fs/cgroup/user.slice/user-0.slice/mycontainerid/memory.events: no such file or directory"
DEBU[0000] nsexec[537]: => nsexec container setup
ERRO[0000]utils.go:61 main.fatalWithCode() runc run failed: unable to start container process: unable to apply cgroup configuration: mkdir /sys/fs/cgroup/user.slice/user-0.slice/mycontainerid: read-only file system

Probably nspawn containers should be automatically detected and the paths should be fixed accordingly.

More details here: https://github.com/moby/moby/issues/44402

But probably this is a runc issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the cgroup-related path handling around libcontainer/process_linux.go:376 and the failure reported from utils.go:61. Reproduce runc --debug run inside a systemd-nspawn container using the cgroup mounts shown in the issue, then determine the expected behavior for the writable hierarchy and verify that the container starts without targeting the read-only path.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go, linux
Domain
infrastructure, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.