coder / coder/registry

Coder boundary: audit records in the final flush window are dropped from the coderd forward on shutdown (local log keeps them)

Open
#1,019 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HCL
Stars
79
Forks
161
Avg merge
1d 10h
Merged PRs (30d)
38

Description

**Environment**: Coder v2.34.2, agent-firewall via built-in coder boundary (`module v0.0.3`, `use_agent_firewall_directly=false`), nsjail, Kubernetes (3 coderd replicas)

**Summary**: When a boundary-wrapped command exits shortly after a request, that request's audit record reaches the local `--log-dir` file but is not forwarded to coderd (`coderd.agentrpc: boundary_request`). The coderd forward is an async periodic (~5s) batch with no flush/drain on shutdown, so any record in the final interval before the wrapped command exits is silently lost centrally. Independent of decision (allow/deny).

**Steps to reproduce**
Allowlist = `domain=www.docker.com` only.
```bash
# Variant A — trailing record has a flush window (sleep 5)
boundary --jail-type=nsjail --config= -- bash -c '
curl -sk -o /dev/null -w "kubernetes(deny): %{http_code}\n" https://kubernetes.io/docs/tutorials/ || true # DENY
sleep 5
curl -sk -o /dev/null -w "docker(allow): %{http_code}\n" https://www.docker.com/get-started/ || true # ALLOW (trailing)
echo done'
```
Image

```bash
# Variant B — no flush window (back-to-back, immediate exit)
boundary --jail-type=nsjail --config= -- bash -c '
curl -sk -o /dev/null -w "kubernetes(deny): %{http_code}\n" https://kubernetes.io/docs/tutorials/ || true # DENY
curl -sk -o /dev/null -w "docker(allow): %{http_code}\n" https://www.docker.com/get-started/ || true # ALLOW
echo done'
```
Image

**Compare the local `--log-dir` file against the `coderd boundary_request` logs on all replicas.**
**Variant A** — trailing ALLOW lands 40 ms before shutdown (DENY had a ~5s window):
| Record | Local log | coderd |
|---|---|---|
| DENY `kubernetes.io` @ `14:23:51.600Z` | ✅ | ✅ present |
| ALLOW `www.docker.com` @ `14:23:56.664Z` | ✅ | ❌ missing (all 3 replicas) |

**Variant B** — both records within ~110 ms of shutdown, no window:
| Record | Local log | coderd |
|---|---|---|
| DENY `kubernetes.io` @ `14:42:18.273Z` | ✅ | ❌ missing |
| ALLOW `www.docker.com` @ `14:42:18.321Z` | ✅ | ❌ missing |

**Impact**
Short-lived invocations are the canonical use case for Agent Firewall (boundary -- ). For any such command the trailing audit record — and for very short commands all records — never reach the central coderd audit stream, while the local file log is complete. Centralized audit is therefore silently incomplete for exactly the workloads Agent Firewall is designed to wrap.

**Suggested fix**: on the wrapped command's exit, flush/drain the pending audit buffer to the log-proxy socket and await ack before boundary exits (optionally with a bounded drain timeout).

Contributor guide

Open the contributing guide

Research direction

Start at the boundary command's shutdown path and trace how pending audit records are batched and sent through the coderd.agentrpc boundary_request forward. Compare the local --log-dir output with the central stream for an immediate-exit command. Done means the final records are drained and acknowledged before boundary exits, including records produced immediately before shutdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, shell
Domain
backend, cli, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.