apple / apple/containerization
[Request]: Enable Cilium's base kernel requirements in the kernel configs
- Dominant language
- Swift
- Stars
- 8.9k
- Forks
- 359
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 13
Description
### Feature or enhancement request details
`kernel/config-arm64` (and `config-x86_64`) is the documented way to build a custom kernel for `container`, and apple/container discussion #1673 asks for Cilium in `container k8s` clusters and links Cilium's requirements page. Cilium's documented base requirements (https://docs.cilium.io/en/stable/operations/system_requirements/#base-requirements):
```
CONFIG_BPF=y CONFIG_BPF_EVENTS=y CONFIG_BPF_SYSCALL=y CONFIG_NET_CLS_BPF=y CONFIG_BPF_JIT=y
CONFIG_NET_CLS_ACT=y CONFIG_NET_SCH_INGRESS=y CONFIG_DEBUG_INFO_BTF=y CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_USER_API_HASH=y CONFIG_CGROUPS=y CONFIG_CGROUP_BPF=y CONFIG_PERF_EVENTS=y CONFIG_SCHEDSTATS=y
```
Five are not enabled in `config-arm64` on `main` (the same five in `config-x86_64`):
| Option | State today | Note |
|---|---|---|
| `CONFIG_BPF_JIT` | absent | the Cilium agent needs a JIT |
| `CONFIG_SCHEDSTATS` | `# ... is not set` | one-line flip |
| `CONFIG_CRYPTO_USER_API_HASH` | `# ... is not set` | one-line flip |
| `CONFIG_BPF_EVENTS` | absent | needs `FTRACE` and `KPROBES`/`KPROBE_EVENTS`, otherwise olddefconfig drops it |
| `CONFIG_DEBUG_INFO_BTF` | absent (`DEBUG_INFO_NONE=y`) | needs `pahole` (dwarves) in `kernel/image` |
Everything else on the list, including the tunneling, netfilter, L7 and WireGuard sets, is already `=y`. `CONFIG_NETKIT=y` (Cilium's netkit device mode, optional) is also absent. For comparison, the kernel `container` recommends today (Kata 3.32.0 debug, 6.18.35) has `BPF_JIT`, `BPF_EVENTS` and `DEBUG_INFO_BTF` but not `SCHEDSTATS`, `NETKIT` or `NETFILTER_XT_MATCH_SOCKET`. A Cilium 1.19.4 agent starts on it but keeps failing to install its proxy iptables rules (`-m socket --transparent`). This request is only about `kernel/config-arm64`.
**Measured.** I built linux-6.18.5 (the version `kernel/Makefile` pins) three times from `config-arm64`, same toolchain for all three (gcc 13.3 in ubuntu:24.04, not the gcc 9.4 `kernel/image` toolchain the file was generated with, so absolute sizes will differ slightly from an in-tree build; the deltas should not):
| Variant | arm64 `Image` bytes | Delta |
|---|---|---|
| `config-arm64` as-is | 29,354,496 | |
| + `BPF_JIT`, `SCHEDSTATS`, `CRYPTO_USER_API_HASH`, `NETKIT` | 29,424,128 | +69,632 (+0.24%) |
| + the above and `BPF_EVENTS` with `FTRACE`, `KPROBES`, `KPROBE_EVENTS` | 33,769,984 | +4,415,488 (+15.0%) |
`make olddefconfig` on the hand-edited small variant adds `BPF_JIT_DEFAULT_ON=y` and `EXECMEM=y` and surfaces `# CONFIG_BPF_JIT_ALWAYS_ON is not set`; nothing else moves relative to the regenerated baseline. The tracing variant additionally pulls in `TRACING`, `EVENT_TRACING`, `TRACEPOINTS`, `RING_BUFFER`, `KRETPROBES`, `UPROBES`/`UPROBE_EVENTS`, `EPROBE_EVENTS`, `DYNAMIC_EVENTS`, `NOP_TRACER`, `CONTEXT_SWITCH_TRACER`, `GLOB`, `TRACE_CLOCK` (all tracers stay off).
Functional check, both variants, on container CLI 1.0.0: booted with `container run --kernel` (`bpf_jit_enable=1`, `/proc/schedstat` present, an `AF_ALG` hash socket binds, `ip link add ... type netkit` succeeds where 6.12.28 and 6.18.35 both say "Unknown device type"), then a two-node kubeadm 1.37 cluster on kindest/node with Cilium 1.19.4 via `cilium install --wait`: `cilium status` OK with 2/2 agents, nodes Ready, and from a curl pod ClusterIP, pod IP and cluster DNS answer 200, including pod-to-pod across nodes with `cilium-dbg status` reporting `Routing: Tunnel [vxlan]`. The small variant is enough; the tracing variant behaved the same, and its tracing features themselves were not exercised at runtime (size is the only thing measured for it).
**Proposal**
1. Enable `CONFIG_BPF_JIT`, `CONFIG_SCHEDSTATS`, `CONFIG_CRYPTO_USER_API_HASH` and `CONFIG_NETKIT`, hand-edited in place the way #681 and #797 were: +68 KB, and enough for the Cilium agent. PR attached, both arches (only arm64 built and tested).
2. Decide separately on the two heavier items Cilium also lists: `CONFIG_BPF_EVENTS` (+4.4 MB with its tracing dependencies; needed by kprobe- and tracepoint-based BPF programs such as Tetragon) and `CONFIG_DEBUG_INFO_BTF` (needs `pahole` in `kernel/image` and flips `DEBUG_INFO_NONE`; needed by CO-RE tooling like `bpftool` and libbpf programs). Neither was needed for Cilium 1.19.4 here, so they fit a follow-up with measured boot-time cost, or a documented custom-build fragment.
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Contributor guide
Research direction
Start with kernel/config-arm64 and compare kernel/config-x86_64, using the linux-6.18.5 version pinned in kernel/Makefile. Run make olddefconfig after the configuration changes, then use container run --kernel and the described Cilium/kubeadm checks; done means the four proposed options are enabled and the documented Cilium checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, linux
- Domain
- networking, operating-systems
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100