bottlerocket-os / bottlerocket-os/bottlerocket-kernel-kit
kernel-kit v8.0.0 (6.12.100) reintroduces SELinux overlayfs execmod denial: UPX-packed binaries (e.g. hasura/graphql-engine) SIGSEGV on EKS Auto Mode AMI 2026.8.24
- Dominant language
- Rust
- Stars
- 13
- Forks
- 38
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 21
Description
**Platform:** EKS Auto Mode, Kubernetes 1.35, `Bottlerocket (EKS Auto, Standard) 2026.8.24 (aws-k8s-1.35-standard)` — kernel 6.12.100-125.179, containerd 2.2.7. Works on `2026.8.10` (kernel 6.12.95-124.187, containerd 2.2.5). Reproduces on Intel (m8i) and AMD (c6a, m5a) instances.
**What happens:** `hasura/graphql-engine` (every version tested: v2.8.4, v2.36.0, v2.48.3) exits 139 immediately at startup — `Segmentation fault (core dumped)` before any output — on 2026.8.24 nodes. The identical images run fine on 2026.8.10 nodes.
**Repro:**
```
kubectl run hasura-probe --image=hasura/graphql-engine:v2.8.4 --restart=Never \
--overrides='{"spec":{"nodeName":"<2026.8.24 node>"}}' --command -- graphql-engine --help
# -> exit 139 on 2026.8.24; prints usage on 2026.8.10
```
**Root cause (gdb inside the container):** the binary is UPX-packed. The UPX stub maps its own file, decompresses in place, then calls `mprotect(addr, len, PROT_READ|PROT_EXEC)`, which returns `-EACCES`; the stub then jumps into the still non-executable page and faults. Registers at the fault: `rdi=0x7ffff7ff7000 rsi=0x1563 rdx=5 rax=-13`, RIP inside the stub, no shared libraries loaded yet. A write-then-exec `mprotect` on *anonymous* memory still succeeds on the same node; only file-backed (overlayfs) mappings are denied.
**Why:** kernel-kit v8.0.0 (#526, cherry-picked in #527) removed `1009-Revert-selinux-fix-overlayfs-mmap-and-mprotect-acces.patch`, which #496 had added for exactly this class of failure (#494, dotnet `execmem`). Upstream 6.12.97's "selinux: fix incorrect execmem checks on overlayfs" skips only `process:execmem` in the mounter-credential check; `file:execmod` (and `file:execute`) are still evaluated against the mounter, i.e. containerd's `runtime_t`, which the policy `neverallow`s (`neverallow host_s global (files (relax))` in `rules.cil`). So any container binary that needs `execmod` on an overlayfs-backed mapping — UPX-packed executables, and likely JIT/runtime code that patches its own file mappings — is denied regardless of the container's own `container_t` permissions.
No pod-level setting affects it (`privileged`, `seLinuxOptions`, `seccompProfile: Unconfined` all tested), and EKS Auto Mode exposes no way to pin or roll back the AMI, so affected workloads break as nodes drift.
**Workaround we applied:** rebuilt the image with the binary unpacked (`upx -d`) — it runs normally on 2026.8.24. Not an option for users who don't control the image.
**Ask:** restore the revert (or carry a kernel/policy fix that covers `file:execmod`/`file:execute` in the mounter check, not just `execmem`) until upstream handles the full case. #494 was .NET and this is a Haskell/UPX binary, so the blast radius is broader than one runtime.
Contributor guide
Assessment
This issue has not been assessed yet.