nestybox / nestybox/sysbox

sysbox-runc with containerd > 2.0 must use new containerd config scheme

Open
#997 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
3.9k
Forks
230
Avg merge
7h 48m
Merged PRs (30d)
3

Description

When running sysbox with containerd > 2.0, it doesn't have to install CRI-O, which is great. However, if you pin the containerd config scheme version to "2" (which is the case for AKS, for example), then there is no backwards compatibility for the containerd v1 scheme ("plugins."io.containerd.grpc.v1.cri").

Wrong (current result):

oom_score = -999
version = 2

[metrics]
  address = "0.0.0.0:10257"

[plugins]

  [plugins."io.containerd.cri.v1.images"]

    [plugins."io.containerd.cri.v1.images".pinned_images]
      sandbox = "mcr.microsoft.com/oss/v2/kubernetes/pause:3.6"

    [plugins."io.containerd.cri.v1.images".registry]
      config_path = "/etc/containerd/certs.d"

      [plugins."io.containerd.cri.v1.images".registry.headers]
        X-Meta-Source-Client = ["azure/aks"]

  [plugins."io.containerd.cri.v1.runtime"]

    [plugins."io.containerd.cri.v1.runtime".containerd]
      default_runtime_name = "runc"

      [plugins."io.containerd.cri.v1.runtime".containerd.runtimes]

        [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runc]
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runc.options]
            BinaryName = "/usr/bin/runc"
            SystemdCgroup = true

        [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.untrusted]
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.untrusted.options]
            BinaryName = "/usr/bin/runc"

  [plugins."io.containerd.grpc.v1.cri"]

    [plugins."io.containerd.grpc.v1.cri".containerd]

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.sysbox-runc]
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.sysbox-runc.options]
            BinaryName = "/usr/bin/sysbox-runc"
            SystemdCgroup = true

Working:

oom_score = -999
version = 2

[metrics]
  address = "0.0.0.0:10257"

[plugins]

  [plugins."io.containerd.cri.v1.images"]

    [plugins."io.containerd.cri.v1.images".pinned_images]
      sandbox = "mcr.microsoft.com/oss/v2/kubernetes/pause:3.6"

    [plugins."io.containerd.cri.v1.images".registry]
      config_path = "/etc/containerd/certs.d"

      [plugins."io.containerd.cri.v1.images".registry.headers]
        X-Meta-Source-Client = ["azure/aks"]

  [plugins."io.containerd.cri.v1.runtime"]

    [plugins."io.containerd.cri.v1.runtime".containerd]
      default_runtime_name = "runc"

      [plugins."io.containerd.cri.v1.runtime".containerd.runtimes]

        [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runc]
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runc.options]
            BinaryName = "/usr/bin/runc"
            SystemdCgroup = true

        [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.untrusted]
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.untrusted.options]
            BinaryName = "/usr/bin/runc"

        [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.sysbox-runc]
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.sysbox-runc.options]
            BinaryName = "/usr/bin/sysbox-runc"
            SystemdCgroup = true

I do not know if the working config is complete / secure - but technically, this runs in AKS running Kubernetes 1.33 with Ubuntu 24.04.

Relates to:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating where sysbox-runc generates or installs the containerd configuration, then compare the shown failing and working configurations for containerd versions above 2.0 and scheme version 2. Reproduce with the AKS or Ubuntu 24.04 setup described; done means sysbox-runc is registered under the active v2 runtime configuration without breaking the existing runtimes.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.