bottlerocket-os / bottlerocket-os/bottlerocket-kernel-kit

kernel-kit v8.0.0 (6.12.100) reintroduces SELinux overlayfs execmod denial: UPX-packed binaries (e.g. hasura/graphql-engine) SIGSEGV on EKS Auto Mode AMI 2026.8.24

Open
#534 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
13
Forks
38
Avg merge
1d 16h
Merged PRs (30d)
21

Description

**Platform:** EKS Auto Mode, Kubernetes 1.35, `Bottlerocket (EKS Auto, Standard) 2026.8.24 (aws-k8s-1.35-standard)` — kernel 6.12.100-125.179, containerd 2.2.7. Works on `2026.8.10` (kernel 6.12.95-124.187, containerd 2.2.5). Reproduces on Intel (m8i) and AMD (c6a, m5a) instances.

**What happens:** `hasura/graphql-engine` (every version tested: v2.8.4, v2.36.0, v2.48.3) exits 139 immediately at startup — `Segmentation fault (core dumped)` before any output — on 2026.8.24 nodes. The identical images run fine on 2026.8.10 nodes.

**Repro:**
```
kubectl run hasura-probe --image=hasura/graphql-engine:v2.8.4 --restart=Never \
--overrides='{"spec":{"nodeName":"<2026.8.24 node>"}}' --command -- graphql-engine --help
# -> exit 139 on 2026.8.24; prints usage on 2026.8.10
```

**Root cause (gdb inside the container):** the binary is UPX-packed. The UPX stub maps its own file, decompresses in place, then calls `mprotect(addr, len, PROT_READ|PROT_EXEC)`, which returns `-EACCES`; the stub then jumps into the still non-executable page and faults. Registers at the fault: `rdi=0x7ffff7ff7000 rsi=0x1563 rdx=5 rax=-13`, RIP inside the stub, no shared libraries loaded yet. A write-then-exec `mprotect` on *anonymous* memory still succeeds on the same node; only file-backed (overlayfs) mappings are denied.

**Why:** kernel-kit v8.0.0 (#526, cherry-picked in #527) removed `1009-Revert-selinux-fix-overlayfs-mmap-and-mprotect-acces.patch`, which #496 had added for exactly this class of failure (#494, dotnet `execmem`). Upstream 6.12.97's "selinux: fix incorrect execmem checks on overlayfs" skips only `process:execmem` in the mounter-credential check; `file:execmod` (and `file:execute`) are still evaluated against the mounter, i.e. containerd's `runtime_t`, which the policy `neverallow`s (`neverallow host_s global (files (relax))` in `rules.cil`). So any container binary that needs `execmod` on an overlayfs-backed mapping — UPX-packed executables, and likely JIT/runtime code that patches its own file mappings — is denied regardless of the container's own `container_t` permissions.

No pod-level setting affects it (`privileged`, `seLinuxOptions`, `seccompProfile: Unconfined` all tested), and EKS Auto Mode exposes no way to pin or roll back the AMI, so affected workloads break as nodes drift.

**Workaround we applied:** rebuilt the image with the binary unpacked (`upx -d`) — it runs normally on 2026.8.24. Not an option for users who don't control the image.

**Ask:** restore the revert (or carry a kernel/policy fix that covers `file:execmod`/`file:execute` in the mounter check, not just `execmem`) until upstream handles the full case. #494 was .NET and this is a Haskell/UPX binary, so the blast radius is broader than one runtime.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.