arc-dind: hardcoded CAP_SYS_MODULE in cap_drop breaks AWF on hosts with a trimmed capability bounding set (Talos)
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 541
- Avg merge
- 5h 46m
- Merged PRs (30d)
- 760
Description
### What happened
`runner.topology: arc-dind` gets a Copilot workflow almost all the way through on an ARC runner, but the run dies when AWF brings up its sandbox:
```
Error response from daemon: invalid CapDrop: capability not supported by your kernel or not available in the current environment: "CAP_SYS_MODULE"
[ERROR] Fatal error: Error: Command failed with exit code 1: docker compose up -d --no-deps --pull never squid-proxy
```
The nodes run [Talos Linux](https://www.talos.dev/), which removes `CAP_SYS_MODULE` and `CAP_SYS_BOOT` from the bounding set of every container on the host — kernel module loading is not part of its threat model:
```
CapBnd: 000001ffffbeffff # Talos
CapBnd: 000001ffffffffff # stock
```
A privileged DinD sidecar inherits that bounding set and cannot exceed it, so the daemon has no `CAP_SYS_MODULE` to reason about. Docker validates `cap_drop` entries against the capabilities it can see and rejects the whole container create rather than treating the drop as satisfied.
### Where it comes from
`awf-bundle.js` hardcodes the list for both the squid proxy and the agent container:
```js
cap_drop:["NET_RAW","SYS_ADMIN","SYS_PTRACE","SYS_MODULE","MKNOD","AUDIT_WRITE","SETFCAP"]
```
```js
cap_add:["SYS_CHROOT","SYS_ADMIN"],cap_drop:["NET_RAW","SYS_PTRACE","SYS_MODULE","SYS_RAWIO","MKNOD"]
```
Still present in `v0.28.8`. I could not find an env var or config key that influences it — `buildContainerSecurityHardening` builds the list unconditionally.
### Why this is safe to change
Dropping a capability that is already outside the bounding set is a no-op. On a host like this, `SYS_MODULE` cannot be held by any container whether AWF names it or not, so omitting it costs exactly zero isolation. The request is only that AWF not *insist* on naming it.
### Suggested fix
Filter `cap_drop` against what the daemon reports before writing the compose file — `docker info` exposes the host's capability view, and `/proc/self/status` `CapBnd` is readable in the runner. Anything not present is already dropped. A narrower alternative would be a documented escape hatch such as `AWF_SKIP_CAP_DROP`, though auto-detection seems strictly better since it needs no configuration and cannot be used to weaken a host that *does* have the capability.
### Impact
This affects any hardened or minimal host that trims the bounding set, not just Talos — the same reasoning applies to distributions that drop `SYS_MODULE` for immutability. `arc-dind` is documented as a supported topology, and everything else in that path now works correctly; this is the only remaining blocker.
### Workaround in use
A `pre-agent-steps` step that strips `"SYS_MODULE"` from the installed bundle after `install_awf_binary.sh` runs, asserting that every other drop survives. Happy to withdraw it the moment this is handled upstream.
### Environment
| | |
|---|---|
| gh-aw | `v0.85.4` |
| AWF | `v0.27.44` (reproduced against `v0.28.8`) |
| Topology | `runner.topology: arc-dind`, ARC `gha-runner-scale-set` 0.14.2 |
| Host | Talos Linux, kernel `6.18.36-talos`, Kubernetes 1.36 |
| Sidecar | `docker:29-dind`, privileged, `DOCKER_HOST=tcp://localhost:2376` |
Contributor guide
Research direction
Start with awf-bundle.js and buildContainerSecurityHardening, then inspect how the cap_drop lists are assembled before docker compose up runs. Reproduce the arc-dind path on the reported Talos environment and verify that containers start there while the complete capability drops remain on hosts that expose them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, javascript, kubernetes, linux
- Domain
- devops, infrastructure, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100