[Bug]: Container subnet routing breaks when a competing default route appears (e.g. VPN/exit-node) and doesn't self-heal
- Dominant language
- Swift
- Stars
- 49.9k
- Forks
- 1.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 22
Description
### Summary
`container`'s vmnet-backed network only ever installs an interface-scoped **default** route (`RTF_IFSCOPE`) for the container subnet (e.g. `bridge100`), never an explicit non-scoped route for the subnet itself (e.g. `192.168.64.0/24`). Because of that, routing to containers is entirely dependent on which *global* default route currently wins on the host. Any other tool that installs or changes the system's default route (a VPN client, a Tailscale exit node, etc.) can silently break all container connectivity — and once broken, it does **not** self-heal when the competing route is removed, or even when `container system` is fully restarted. Only manually adding an explicit subnet route restores it.
This is a different bug from #856 — that one is about `container-apiserver` failing to *bind* a listener to the gateway IP (`EADDRNOTAVAIL`). This is about outbound *routing* to already-running containers breaking due to route table interference, discovered while investigating #856/#1813 (it isn't the same root cause — I confirmed binding to the gateway IP for listening still succeeds even while this routing bug is active).
**Update:** I initially assumed this was a routing *priority* problem (a competing default route "winning" over the more-specific `192.168.64.0/24` route). Testing further, that's not quite right — I manually installed an explicit `192.168.64.0/24 -interface bridge100` route and confirmed it correctly restores connectivity on its own, but when a Tailscale exit node is subsequently enabled/disabled, that route is **removed from the table outright**, not merely outranked. This matches [tailscale/tailscale#18653](https://github.com/tailscale/tailscale/issues/18653) ("Exit node overrides existing connected RFC1918 routes"), which explicitly calls out `apple/container`'s `bridge100` interface and is already being investigated on their side — it looks like a Tailscale-side behavior at the System Extension layer, not something `container` can prevent outright.
Given that, this issue is really two things:
1. `container` never installs an explicit subnet route in the first place, only relying on the interface-scoped default (fixable here, see below).
2. A well-behaved third-party tool (any VPN/route-manipulating tool, not just Tailscale) can still remove that route later, which `container` can't unilaterally prevent — that's tracked upstream at tailscale/tailscale#18653 for the Tailscale-specific case.
### Environment
```
- OS: macOS 26.5 (25F71)
- container: 1.0.0 (release, commit: ee848e3)
- Trigger used: Tailscale 1.90.9 exit node (any tool that installs a competing default route should reproduce this)
- Architecture: arm64
```
### Steps to reproduce
1. Start `container system` and run a container on the default network:
```
container run -d --name web nginx
container list # note the container's IP, e.g. 192.168.64.47
```
2. Confirm it's reachable:
```
nc -zv 192.168.64.47 80 # succeeds
```
3. Introduce a competing default route. I used a Tailscale exit node:
```
tailscale set --exit-node=
```
4. Inspect routing to the container subnet:
```
route -n get 192.168.64.5
```
Before step 3, this resolves via `bridge100`. After, it resolves via the *other* default route (in my case `en10` via the LAN gateway), because there's never been an explicit `192.168.64.0/24` route — only `bridge100`'s own interface-scoped default, which isn't consulted unless something explicitly scopes to that interface.
5. Confirm connectivity is now broken:
```
nc -zv 192.168.64.47 80 # times out
```
6. Remove the competing route:
```
tailscale set --exit-node=
```
Routing to the container subnet **remains broken** — `route -n get 192.168.64.5` still resolves via the wrong interface.
7. Restart container networking entirely:
```
container system stop
container system start
```
Routing **still remains broken** after this.
8. Manually installing an explicit subnet route does restore connectivity:
```
sudo route add -net 192.168.64.0 -netmask 255.255.255.0 -interface
```
(e.g. `-interface 192.168.64.1` for the default network's `bridge100` gateway)
9. However, this manually-added route is **not** durable against the original trigger: re-enabling and then disabling the Tailscale exit node again removes it from the table entirely (not just outranks it) — see tailscale/tailscale#18653.
### Current behavior
Container subnet connectivity can be permanently broken by any third-party tool that manipulates the default route, and neither removing that tool's route nor restarting `container system` repairs it. `container` also never installs an explicit subnet route on its own, so it has no baseline resilience against this class of interference at all.
### Expected behavior
`container` should install an explicit, non-scoped subnet route (e.g. `192.168.64.0/24 -interface bridge100`) when the network is created, rather than relying solely on the interface-scoped default. This won't fully eliminate exposure to tools like Tailscale that actively remove routes they don't own (that's tracked at tailscale/tailscale#18653), but it does close the gap for the simpler and more common case — anything that merely adds a competing default route without deleting others — and gives `container` a correct baseline instead of none.
### Workaround in the meantime
Tools built on top of `container`/`container-compose` that need to survive this (e.g. avoiding an exit node's effects on local dev) currently work around it by explicitly binding sockets to the bridge gateway IP before connecting out, forcing the kernel to use the interface-scoped route directly rather than relying on the general routing table.
Contributor guide
Research direction
Start by reproducing the issue with the listed `container system`, `container run`, `route -n get`, and Tailscale commands, then inspect the network-creation path that installs the interface-scoped default route. Done means container networking installs an explicit non-scoped subnet route such as `192.168.64.0/24` while preserving the documented limitation around routes removed by Tailscale.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100