coder / coder/envbox

Regression in 0.6.8 on Bottlerocket: /sys/fs/cgroup umount fails with EBUSY

Closed
#179 1 comment 1 reaction 1 assignee Claimed by @geokat View on GitHub
Dominant language
Go
Stars
73
Forks
22
Avg merge
14h 50m
Merged PRs (30d)
1

Description

After upgrading Envbox from 0.6.7 to 0.6.8, workspaces no longer start on AWS Bottlerocket nodes.

0.6.7 works correctly with the same cluster and workspace configuration.

Environment:

AWS EKS
Bottlerocket OS 1.64.0 (aws-k8s-1.36)
Kubernetes 1.36
containerd 2.2.5+bottlerocket
cgroup v2

Error:

Waiting for dockerd to startup...

Failed to create Container-based Virtual Machine:
umount: /sys/fs/cgroup: target is busy.
envbox: failed to umount /sys/fs/cgroup

This looks related to the cgroup changes introduced in #169 and adjusted in #173.

0.6.8 added wrap_dockerd.sh, which can execute:

umount /sys/fs/cgroup
mount -t cgroup2 cgroup /sys/fs/cgroup

The failure is coming directly from that new code path.

#173 already mentions EBUSY problems with the new cgroup remount logic, but it looks like Bottlerocket/containerd exposes another mount topology where:

cgroup_mount_root != "/"

while /sys/fs/cgroup still cannot safely be unmounted.

Regression range:

0.6.7: working
0.6.8: broken

I can provide /proc/self/mountinfo, findmnt -R /sys/fs/cgroup, and other cgroup details from the failing pod if useful.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.