boxlite-ai / boxlite-ai/boxlite

Jailer integration tests fail on Ubuntu 24+ default kernel/AppArmor (unprivileged userns blocked)

Open
#468 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2.3k
Forks
179
Avg merge
23h 25m
Merged PRs (30d)
121

Description

## Summary

Two integration tests in `src/boxlite/tests/jailer.rs` fail on hosts where unprivileged user namespaces are blocked by the kernel or AppArmor:

- `jailer::jailer_creates_isolated_mount_namespace`
- `jailer::jailer_enabled_box_starts_and_executes`

The failure surfaces during the box start preflight as:

```
called `Result::unwrap()` on an `Err` value:
Config(\"Sandbox preflight failed: bwrap cannot create user namespaces.

failed to run bwrap: Permission denied (os error 13)\")
```

Kernel audit log on a fresh Ubuntu 24.04 dev host:

```
audit: type=1400 apparmor=\"DENIED\" operation=\"capable\"
profile=\"unprivileged_userns\" capability=21 capname=\"sys_admin\"
```

## Reproduction

```bash
# fresh Ubuntu 24.04 EC2 host (kernel 7.0.0-1004-aws)
git clone https://github.com/boxlite-ai/boxlite
cd boxlite
make setup
make runtime:debug
make test:integration:rust FILTER=jailer_enabled_box
```

`bwrap --unshare-user --bind / / true` succeeds in isolation, but the libcontainer-driven preflight in the jailer requires `CAP_SYS_ADMIN` inside the userns and that is denied by the `unprivileged_userns` AppArmor profile that ships in `apparmor` ≥ 4.0 (Ubuntu 24+ default).

Setting:
- `kernel.unprivileged_userns_clone=1`
- `kernel.apparmor_restrict_unprivileged_userns=0`
- `apparmor_parser -R /etc/apparmor.d/unprivileged_userns`

does not unblock the preflight on this host. The kernel/AppArmor stack appears to require additional unconfinement for `bwrap`.

## Decision (this PR)

These two tests are marked `#[ignore]` until a portable fix lands. The skip reason is encoded in the `#[ignore]` attribute so it shows up in test output. Run them explicitly with:

```bash
cargo nextest run -p boxlite --features krun,gvproxy --test jailer --run-ignored only
```

## Options for a real fix

1. **Test-side preflight** — detect userns availability at test start, skip with clear logging on unsupported hosts. Keeps the signal in CI runners with permissive kernels.
2. **Run jailer tests in a privileged container** in CI (the runc / youki convention).
3. **Document a setup script** that installs an AppArmor allowlist for the test bwrap binary, similar to `bubblewrap-userns-restrict` overrides for snap.

Tracking under this issue. Likely going with (1) + (2) for CI.

## Context

These two tests were among the originally-reported 14 failures in `make test:all`; the other 12 had different root causes (engine registration + PID validation) addressed in [PR for `fix/shim-required-features-and-pid-validation`](https://github.com/boxlite-ai/boxlite/pull/0). With that fix, 9/14 pass on a clean dev box; the remaining 5 are environment-dependent (these 2 jailer tests + 3 flaky zygote concurrency tests, see separate issue).

Contributor guide

Open the contributing guide

Research direction

Start with src/boxlite/tests/jailer.rs and reproduce with make test:integration:rust FILTER=jailer_enabled_box on an affected Ubuntu 24+ host. Compare the two named tests and the libcontainer-driven preflight behavior, then use the stated test-side and CI options as constraints. Done means unsupported hosts are handled with a clear skip while permissive CI still exercises the jailer tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
ci-cd, operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.