boxlite-ai / boxlite-ai/boxlite
Jailer integration tests fail on Ubuntu 24+ default kernel/AppArmor (unprivileged userns blocked)
- Dominant language
- Rust
- Stars
- 2.3k
- Forks
- 179
- Avg merge
- 23h 25m
- Merged PRs (30d)
- 121
Description
## Summary
Two integration tests in `src/boxlite/tests/jailer.rs` fail on hosts where unprivileged user namespaces are blocked by the kernel or AppArmor:
- `jailer::jailer_creates_isolated_mount_namespace`
- `jailer::jailer_enabled_box_starts_and_executes`
The failure surfaces during the box start preflight as:
```
called `Result::unwrap()` on an `Err` value:
Config(\"Sandbox preflight failed: bwrap cannot create user namespaces.
failed to run bwrap: Permission denied (os error 13)\")
```
Kernel audit log on a fresh Ubuntu 24.04 dev host:
```
audit: type=1400 apparmor=\"DENIED\" operation=\"capable\"
profile=\"unprivileged_userns\" capability=21 capname=\"sys_admin\"
```
## Reproduction
```bash
# fresh Ubuntu 24.04 EC2 host (kernel 7.0.0-1004-aws)
git clone https://github.com/boxlite-ai/boxlite
cd boxlite
make setup
make runtime:debug
make test:integration:rust FILTER=jailer_enabled_box
```
`bwrap --unshare-user --bind / / true` succeeds in isolation, but the libcontainer-driven preflight in the jailer requires `CAP_SYS_ADMIN` inside the userns and that is denied by the `unprivileged_userns` AppArmor profile that ships in `apparmor` ≥ 4.0 (Ubuntu 24+ default).
Setting:
- `kernel.unprivileged_userns_clone=1`
- `kernel.apparmor_restrict_unprivileged_userns=0`
- `apparmor_parser -R /etc/apparmor.d/unprivileged_userns`
does not unblock the preflight on this host. The kernel/AppArmor stack appears to require additional unconfinement for `bwrap`.
## Decision (this PR)
These two tests are marked `#[ignore]` until a portable fix lands. The skip reason is encoded in the `#[ignore]` attribute so it shows up in test output. Run them explicitly with:
```bash
cargo nextest run -p boxlite --features krun,gvproxy --test jailer --run-ignored only
```
## Options for a real fix
1. **Test-side preflight** — detect userns availability at test start, skip with clear logging on unsupported hosts. Keeps the signal in CI runners with permissive kernels.
2. **Run jailer tests in a privileged container** in CI (the runc / youki convention).
3. **Document a setup script** that installs an AppArmor allowlist for the test bwrap binary, similar to `bubblewrap-userns-restrict` overrides for snap.
Tracking under this issue. Likely going with (1) + (2) for CI.
## Context
These two tests were among the originally-reported 14 failures in `make test:all`; the other 12 had different root causes (engine registration + PID validation) addressed in [PR for `fix/shim-required-features-and-pid-validation`](https://github.com/boxlite-ai/boxlite/pull/0). With that fix, 9/14 pass on a clean dev box; the remaining 5 are environment-dependent (these 2 jailer tests + 3 flaky zygote concurrency tests, see separate issue).
Contributor guide
Research direction
Start with src/boxlite/tests/jailer.rs and reproduce with make test:integration:rust FILTER=jailer_enabled_box on an affected Ubuntu 24+ host. Compare the two named tests and the libcontainer-driven preflight behavior, then use the stated test-side and CI options as constraints. Done means unsupported hosts are handled with a clear skip while permissive CI still exercises the jailer tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, rust
- Domain
- ci-cd, operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100