bottlerocket-os / bottlerocket-os/bottlerocket

Kubevirt fails to run on bottlerocket

Open
#4,229 3 comments 1 reaction 0 assignees View on GitHub
status/needs-triage type/bug
Dominant language
Rust
Stars
9.7k
Forks
586
Avg merge
1d 11h
Merged PRs (30d)
11

Description

**Image I'm using:**
OS Image: Bottlerocket OS 1.24.0 (aws-k8s-1.28)
Operating System: linux
Architecture: amd64
Container Runtime Version: containerd://1.7.22+bottlerocket
Kubelet Version: v1.28.10-eks-890c2ac
Kube-Proxy Version: v1.28.10-eks-890c2ac

**What I expected to happen:**
I am trying to setup [kubevirt](https://github.com/kubevirt/kubevirt) 1.3.1 on EKS with bottlerocket.

**What actually happened:**
There's one kubevirt component that is currently failing to start:

```
virt-handler {"component":"virt-handler","level":"info","msg":"Kubevirt Seccomp profile is not enabled","pos":"virt-handler.go:521","timestamp":"2024-10-02T10:27:07.747966Z"}
virt-handler {"component":"virt-handler","level":"info","msg":"set verbosity to 2","pos":"virt-handler.go:490","timestamp":"2024-10-02T10:27:07.747976Z"}
virt-handler {"component":"virt-handler","level":"info","msg":"VSOCK server is already stopped","pos":"server.go:32","timestamp":"2024-10-02T10:27:07.747985Z"}
virt-handler Error: error relabeling file /proc/self/fd/4 with label system_u:object_r:container_file_t:s0. Reason: operation not supported
virt-handler Usage:
virt-handler virt-chroot selinux relabel [flags]
virt-handler
virt-handler Examples:
virt-handler virt-chroot selinux relabel
virt-handler
virt-handler Flags:
virt-handler -h, --help help for relabel
virt-handler --root string safe root path which will be prepended to passed in files (default "/")
virt-handler
virt-handler Global Flags:
virt-handler --cpu uint cpu time in seconds for the process
virt-handler --memory uint memory in bytes for the process
virt-handler --mount string mount namespace to use
virt-handler --user string switch to this targetUser to e.g. drop privileges
virt-handler
virt-handler error relabeling file /proc/self/fd/4 with label system_u:object_r:container_file_t:s0. Reason: operation not supported
virt-handler panic: error relabeling required files: error relabeling file root: /, relative: /dev/net/tun with label system_u:object_r:container_file_t:s0. Reason: exit status 1
virt-handler
virt-handler goroutine 1 [running]:
virt-handler main.(*virtHandlerApp).Run(0xc000682c40)
virt-handler cmd/virt-handler/virt-handler.go:406 +0x271c
virt-handler main.main()
virt-handler cmd/virt-handler/virt-handler.go:675 +0x49
```

This kubevirt component is managed by a DaemonSet. It was running with
```
securityContext:
privileged: true
```

I've changed it to
```
securityContext:
privileged: true
seLinuxOptions:
role: system_r
user: system_u
type: super_t
```
But it made no difference.

Kubevirt install docs mention that it requires [container-selinux](https://github.com/containers/container-selinux) installed. I'm trying to figure out if bottlerocket has it, but it's unclear to me. I checked the bottlerocket node via the admin-container and can't see the `container_file_t` type.

```
seinfo -t

Types: 34
clock_exec_t
container_t
runtime_exec_t
system_t
any_t
api_t
bus_t
cache_t
clock_t
data_t
etc_t
lease_t
os_t
proc_t
state_t
mount_exec_t
csi_exec_t
bus_exec_t
cni_exec_t
api_exec_t
network_t
measure_t
runtime_t
private_t
network_exec_t
init_exec_t
api_socket_t
kernel_t
secret_t
init_t
local_t
mount_t
super_t
control_t
```

I found this [PR](https://github.com/bottlerocket-os/bottlerocket/pull/1316/files) that hints that bottlerocket does have container-selinux installed though. Or is it that bottlerocket used names that matches the ones that container-selinux uses, but are different types?

Given that the DaemonSet is running in privileged mode with `super_t` as type the problem then must be that Bottlerocket does not have container-selinux types installed, right?
Is this case, is my only option to build my own bottlerocket image with this dependency included?

**How to reproduce the problem:**
Install kubevirt on a bottlerocket node.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.