Cannot enforce NoNewPrivs: illegal version string v1

After switching Kubernetes v1.23.17 cluster nodes from dockershim to cri-dockerd, kubelet blocks a single Pod (the MetalLB controller) while all other Pods seem to run fine:

Mar 20 15:55:51 n0217 kubelet[1225]: E0320 15:55:51.965802    1225 pod_workers.go:965] "Error syncing pod, skipping" err="pod cannot be run: Cannot enforce NoNewPrivs: illegal version string \"v1\"" pod="metallb-system/controller-85dd57bdc-wh29s" podUID=578c94ac-0f44-40e2-bf74-9729c7806f18

Reverting to dockershim or setting securityContext.allowPrivilegeEscalation to true let the Pod start without issues. The Deployment does not look special:

        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - all
          readOnlyRootFilesystem: true

Using cri-dockerd also raises object not registered warnings.

  Warning  FailedMount  114s (x10 over 6m4s)  kubelet            MountVolume.SetUp failed for volume "cert" : object "metallb-system"/"webhook-server-cert" not registered
  Warning  FailedMount  114s (x10 over 6m4s)  kubelet            MountVolume.SetUp failed for volume "kube-api-access-pt487" : object "metallb-system"/"kube-root-ca.crt" not registered

However, they disappear after a while, and I wonder if they are related.

Is Kubernetes v1.23 compatible with cri-dockerd? Any hints on what’s missing or incompatible?

Versions:

• cri-dockerd v0.3.1
• OS: Ubuntu Server 22.04.2
• Linux kernel 5.19.0-35-generic
• Docker v23.0.1 (also tested with v20.10.23)

Start by reproducing the failure with Kubernetes v1.23.17, cri-dockerd v0.3.1, Docker 23.0.1 or 20.10.23, Ubuntu 22.04.2, and the listed kernel. Trace the kubelet and cri-dockerd handling of NoNewPrivs and the object-not-registered warnings; done means determining whether the versions are compatible and documenting a verified fix or workaround.