nestybox / nestybox/sysbox

Sysbox Install - Pods stuck in ContainerCreating or Podinitializing state - cri-o configured with systemd cgroup manager, but did not receive slice as parent

Open
#567 30 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
3.9k
Forks
230
Avg merge
7h 48m
Merged PRs (30d)
3

Description

Rancher 2.6.5 - RKE2 v1.23.6+rke2r1 - Sysbox-CE 0.5.2 - Ubuntu 20.04 - Kernel 5.13.0-39-generic - x86_64

An issue was observed when attempting to install Sysbox on an RKE2 kubernetes cluster.
After the install, the pods in each worker node fail to come back online, and are stuck in the ContainerCreating or Podinitializing state.

The pods show the following error.

Failed to create pod sandbox: rpc error: code = Unknown desc = cri-o configured with systemd cgroup manager, but did not receive slice as parent: /kubepods/burstable/pod95c5e3b0-0305-4bd3-8cf7-019d19f9af0b

Below are logs from the sysbox-deploy-k8s-xxxxx pod.

Detected Kubernetes version v1.23
Adding K8s label "crio-runtime=installing" to node ...
node/xxxxx labeled
Deploying CRI-O installer agent on the host (v1.23) ...
Running CRI-O installer agent on the host (may take several seconds) ...
Removing CRI-O installer agent from the host ...
Configuring CRI-O ...
Adding K8s label "sysbox-runtime=installing" to node ...
node/xxxxx labeled
Installing Sysbox dependencies on host ...
Copying shiftfs sources to host ...
Kernel version 5.13 is >= 5.13
Deploying Sysbox installer helper on the host ...
Running Sysbox installer helper on the host (may take several seconds) ...
Stopping the Sysbox installer helper on the host ...
Removing Sysbox installer helper from the host ...
Installing Sysbox on host ...
Created symlink from /etc/systemd/system/multi-user.target.wants/sysbox.service to /lib/systemd/system/sysbox.service.
Created symlink from /etc/systemd/system/sysbox.service.wants/sysbox-mgr.service to /lib/systemd/system/sysbox-mgr.service.
Created symlink from /etc/systemd/system/sysbox.service.wants/sysbox-fs.service to /lib/systemd/system/sysbox-fs.service.
Configuring host sysctls ...
kernel.unprivileged_userns_clone = 1
fs.inotify.max_queued_events = 1048576
fs.inotify.max_user_watches = 1048576
fs.inotify.max_user_instances = 1048576
kernel.keys.maxkeys = 20000
kernel.keys.maxbytes = 1400000
kernel.pid_max = 4194304
Starting Sysbox ...
Adding Sysbox to CRI-O config ...
Restarting CRI-O ...
Deploying Kubelet config agent on the host ...
Running Kubelet config agent on the host (will restart Kubelet and temporary bring down all pods on this node for ~1 min) ...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the sysbox-deploy-k8s logs and the Failed to create pod sandbox error, then inspect how the installer configures CRI-O and the kubelet on the RKE2 worker nodes. Reproduce with the listed Rancher, RKE2, Sysbox, Ubuntu, and kernel versions if possible; done means the worker pods return from ContainerCreating or Podinitializing without the cgroup-parent error.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.