feat(supervisor): NFLOG-based bypass detection to drop the CAP_SYSLOG requirement
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 8.7k
- Forks
- 1.3k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 253
Description
Problem Statement
Sandbox pods/containers currently request CAP_SYS_ADMIN, CAP_NET_ADMIN, CAP_SYS_PTRACE, and CAP_SYSLOG. With Kubernetes user namespaces (GA in 1.36, already supported via enable_user_namespaces) the first three become namespaced and lose their host-level power — but CAP_SYSLOG does not. The kernel checks syslog(2)//dev/kmsg access against the initial user namespace (capable(), not ns_capable()), so in a hostUsers: false pod the granted CAP_SYSLOG is a no-op and the bypass detection monitor silently degrades.
The only consumer of CAP_SYSLOG is the bypass detection monitor in openshell-supervisor-process, which tails dmesg --follow for nftables log entries (prefix openshell:bypass:<ns>:) emitted by the per-workload netns REJECT rules. Reading the host-global kernel ring buffer from inside a sandbox is also a mild information leak, and requires the nf_log_all_netns sysctl on the node.
Proposed Design
Switch bypass log delivery from kernel-printk to nfnetlink_log (NFLOG):
- During netns setup (privileged startup, before supervisor seccomp hardening), bind an
AF_NETLINK/NETLINK_NETFILTERsocket to an NFLOG group inside the workload netns. Netns affinity is fixed atsocket()time; binding needs onlyCAP_NET_ADMINover the netns owner, which works under user namespaces. - Generate the nftables log rules as
log prefix <p> group <N>when the socket bind succeeds; keep the existinglog ... flags skuidprintk rules as an automatic fallback when it fails (e.g. kernel withoutnfnetlink_log). - The bypass monitor consumes NFLOG packet messages (prefix attribute for filtering,
NFULA_UIDfor workload UID, raw IP payload for dst/ports/proto) and reports through the existing OCSF dual-emit path. Thedmesgpath remains as fallback (still useful in the VM driver, which runs with full capabilities). - Drop
CAP_SYSLOGfrom the Kubernetes, Docker, and Podman driver capability sets.
Enforcement is unaffected: the nftables REJECT rules do not depend on the log delivery mechanism. NFLOG delivery is netns-scoped, so the monitor no longer reads host-global kernel logs, and nf_log_all_netns is no longer needed on the NFLOG path.
Alternatives Considered
- Keep dmesg, relax
kernel.dmesg_restricton nodes — exposes host kernel logs to every pod on the node; strictly worse. - eBPF-based detection —
bpf()is also initial-userns-gated (CAP_BPF/CAP_SYS_ADMINin init ns), so it has the same problem asCAP_SYSLOG, plus a new dependency. - Netlink crate dependency (
netlink-packet-netfilter/libnetfilter_logbindings) — the required protocol subset (group bind, copy mode, packet attr parsing) is ~stable kernel ABI and small; a hand-rolledlibcimplementation avoids new dependencies in a security-sensitive crate and matches its existing raw-FFI style.
Agent Investigation
CAP_SYSLOG's only consumer isbypass_monitor::spawn(crates/openshell-supervisor-process/src/bypass_monitor/mod.rs); drivers request it inopenshell-driver-kubernetes/src/driver.rs,openshell-driver-docker/src/lib.rs,openshell-driver-podman/src/container.rs.- No eBPF usage anywhere in the workspace; netns/nftables management shells out to
iproute2/nft, so NFLOG covers the whole detection path. - The supervisor seccomp prelude does not restrict
socket(); the workload runtime filter blocks non-NETLINK_ROUTEnetlink sockets, so the sandboxed workload cannot bind the NFLOG group itself. - The sidecar topology has no bypass log listener today (no
CAP_SYSLOGthere either); it keeps printk rules unchanged. - Implementation is complete on a branch with unit tests (netlink message layout, datagram parsing, IPv4/IPv6 packet parsing, ruleset generation); Linux
cargo check/clippy(pedantic+nursery)/tests pass. PR to follow.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with crates/openshell-supervisor-process/src/bypass_monitor/mod.rs and the driver capability files in openshell-driver-kubernetes/src/driver.rs, openshell-driver-docker/src/lib.rs, and openshell-driver-podman/src/container.rs. Review the existing branch implementation and its unit tests, then run Linux cargo check, clippy, and tests. Done means NFLOG delivery works with printk fallback and CAP_SYSLOG is removed from the listed drivers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, kubernetes, linux, rust
- Domain
- networking, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100