agent-substrate / agent-substrate/substrate

Worker datapath does not drop non-tunneled actor egress, bypassing the egress gateway

Aperta
#1,332 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area/network area/security kind/bug
Lingua principale
Go
Stelle
1.8k
Fork
316
Merge medio
2g 43m
PR unite (30g)
287

Descrizione

## Summary

When an egress gateway is configured, actor TCP goes through atunnel, but non-TCP traffic still gets forwarded and masqueraded directly out of the worker. That traffic never reaches the gateway, so gateway policy cannot control it.

There’s already a TODO in `internal/ateomnet/net.go` to restrict the masquerade to resolver DNS and drop other non-tunneled egress. This issue proposes doing that, depending on the direction agreed in #1342.

## What happens today

`InstallActorNftablesRules` installs:

- A TCP redirect to atunnel when `egressPort != 0`, including TCP to in-cluster addresses.
- A masquerade rule for **all** traffic from the actor IP.
- A forward chain with an accept policy and an unconditional accept rule.

Both gVisor and microVM use this setup.

I tested the current setup in disposable Linux network namespaces with tunneled egress enabled:

- Non-DNS UDP reached a separate destination, with the worker’s IP as its source.
- TCP to the same destination was redirected to the local listener.
- Removing masquerade still allowed UDP through when the destination had a return route to the actor subnet.
- Replacing the forwarding accept with a drop blocked UDP while leaving TCP redirection working.

So removing NAT alone isn’t enough: we need an explicit drop. Actual reachability still depends on routing, CNI policy, and firewalls.

These were local datapath tests, not full gVisor/microVM e2e tests. The three scenarios and four existing setup tests passed across three runs; `go vet ./internal/ateomnet` also passed.

## Proposed change

When a gateway is configured:

- Allow direct UDP DNS only to the configured resolver(s), plus replies.
- Restrict masquerade to that DNS exception and explicitly drop other actor-originated forwarded traffic.
- Scope the rules to the actor-facing interface, coordinating source validation with #175.
- Don’t fall back to unrestricted forwarding if resolver configuration is missing or invalid.

Keep the existing direct-egress behavior when no gateway is configured.

Tests should cover blocked non-DNS traffic, DNS (including TCP fallback), tunneled HTTP(S)/gRPC, and ingress, across activation, restore, and worker reuse in both runtimes.

This would cover IPv4 forwarding only. Worker-local traffic takes the INPUT path, so this alone would not provide complete actor network isolation; that belongs with #175.

## Related work / questions

- **#1342** already covers non-TCP egress behavior. Should this be a separate implementation task or folded into that issue?
- **#1339** says broader protocol-policy support is outside the initial milestone. Is dropping unsupported traffic something we want sooner, or should it wait too?
- **PR #985** narrows masquerade but leaves forwarding accept-all, so it doesn’t guarantee a drop.
- **#1325 / #1323** cover gateway policy; this addresses traffic that bypasses it.
- **#960 / #1430** cover in-cluster services and telemetry. Those paths need to keep working through whichever policy mechanism we choose.

Before implementing, we should agree on the DNS exception—resolver selection, TCP fallback, and missing configuration—and what to do with existing connections when stricter rules are installed.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.