Local dev: containers never start on hosts with net.ipv4.conf.all.src_valid_mark=1 (e.g. Tailscale) — sidecar handshake packets dropped as martians

## Summary

On Linux hosts where `net.ipv4.conf.all.src_valid_mark=1` is set in the init
netns — which **Tailscale does by default** — `wrangler dev` with a `containers`
config can never start the application container. workerd creates the
`cloudflare/proxy-everything` egress-interceptor sidecar, but the TCP handshake
between workerd and the sidecar never completes: the kernel inside the sidecar
netns drops every packet that the sidecar's own TPROXY rules have fwmark-marked,
because the inherited `src_valid_mark=1` makes source validation use the fwmark
and classify the peer as a martian source. workerd times out after ~20 s
(`kj/timer.c++:30: overloaded: operation timed out`), destroys the sidecar,
retries, and the Durable Object eventually fails with
`Container failed to start` (`outcome: 'unrecognized_error'`) / HTTP 500
`"Container is starting. Please retry in a moment."` after ~151 s.

The application container is **never** even created (no `container create` in
`docker events` for the app image over the whole retry budget — only the
`…-proxy` sidecar churns create/destroy every ~20–45 s).

A one-line fix is possible on workerd's side: pass
`HostConfig.Sysctls: {"net.ipv4.conf.all.src_valid_mark": "0"}` when creating
the sidecar (namespaced `net.*` sysctls are settable via the Docker API without
extra privileges). Optionally also `net.ipv4.conf.all.rp_filter: "0"`, which is
inherited the same way and can interfere with TPROXY setups too.

## Environment

- workerd `1.20260701.1` (via wrangler `4.107.0`, miniflare `4.20260701.0`)
- `@cloudflare/sandbox` 0.12.3 (but any `containers` config reproduces it)
- Sidecar image: `cloudflare/proxy-everything:3cb1195@sha256:0ef6716c…`
- Linux 7.1.3 (CachyOS/Arch), Docker 29.6.1 (iptables backend), iptables 1.8.11 (nf_tables)
- Tailscale installed and running on the host (this is what sets
`net.ipv4.conf.all.src_valid_mark=1` in the init netns)

## Reproduction

Tailscale is not required — the sysctl is the trigger:

```bash
# on any Linux host with working `wrangler dev` containers:
sudo sysctl -w net.ipv4.conf.all.src_valid_mark=1

# any Worker project with a `containers` config:
wrangler dev
curl -X POST http://localhost:8787/... # anything that starts the container DO
```

Result: sidecar create/destroy loop, ~20 s per attempt, app container never
created, request fails after the retry budget. Setting the sysctl back to `0`
(and recreating the containers) fixes it immediately.

Note on inheritance: with the kernel default
`net.core.devconf_inherit_init_net=0`, **new network namespaces inherit all
IPv4 `conf/{all,default}` values from init_net**, so a host-level
`src_valid_mark=1` silently propagates into every container netns that Docker
creates.

## Root cause analysis

The sidecar sets up transparent interception inside its netns:

```
# mangle PREROUTING (inside the sidecar netns)
-A PREROUTING -p tcp -m socket -j DIVERT # existing-socket match → mark
-A DIVERT -j MARK --set-xmark 0x1/0xffffffff
-A DIVERT -j ACCEPT
...
# policy routing
32765: from all fwmark 0x1 lookup 100
# table 100
local default dev lo scope host
```

When workerd connects from the docker bridge gateway (e.g. `172.17.0.1`) to the
sidecar's ingress (`:39001`) to push the egress configuration:

1. The **SYN** does not match `-m socket` (xt_socket ignores non-transparent
wildcard listeners), stays unmarked, and is delivered normally — the
listener replies SYN-ACK and goes to `SYN_RECV`.
2. The **final ACK and the request payload** match `-m socket` (the request
socket for the 4-tuple now exists), get fwmark `0x1`, and then hit source
validation. With `src_valid_mark=1`, `fib_validate_source()` includes the
fwmark in the reverse-path lookup, which therefore resolves via table 100
(`local default dev lo`) — the source `172.17.0.1` is classified as a
local/martian source and the packet is **dropped**.
3. The sidecar's TCP stack never sees the ACK: it stays in `SYN_RECV`
retransmitting SYN-ACK; workerd's side sits in `FIN-WAIT-1` with ~164 bytes
unacked. After ~20 s the kj timer fires, workerd tears the sidecar down and
retries.

Evidence gathered while debugging:

- `docker events`: only `…-proxy` create/start/kill/destroy cycles; zero
`create` events for the app image.
- Docker API capture (socat proxy on the docker socket): per attempt exactly
`GET /networks/bridge` → `POST /containers/create?name=…-proxy` →
`POST /containers/…-proxy/start` → `GET /containers/…-proxy/json`, then ~20 s
of silence, then force-DELETEs — i.e. workerd blocks between "inspect
sidecar" and "create app container".
- tcpdump on both ends: ACK+data visible on the sidecar's `eth0`, never
processed by TCP; `/proc/net/tcp6` inside the sidecar shows the connection
stuck in `SYN_RECV`; host side stuck in `FIN-WAIT-1` with Send-Q > 0.
- Routing verdict inside the sidecar netns:
`ip route get from 172.17.0.1 iif eth0 mark 0x1` →
`RTNETLINK answers: Invalid argument`, while the same query **without**
`mark 0x1` returns a normal local route.
- Live fix: `nsenter -t -n sysctl -w net.ipv4.conf.all.src_valid_mark=0`
→ the very next attempt succeeds; app container is created and the DO
request returns HTTP 200 in ~1.6 s.

## Suggested fix

Set the sysctl explicitly when creating the sidecar container (and, if it uses
mark-based routing too, the app container), so local dev does not depend on
host-inherited values:

```json
"HostConfig": {
"Sysctls": {
"net.ipv4.conf.all.src_valid_mark": "0"
}
}
```

`rp_filter` may deserve the same treatment (`"net.ipv4.conf.all.rp_filter": "0"`),
since strict rp_filter in the container netns (also inherited from hosts that
set it, e.g. via ufw's `/etc/ufw/sysctl.conf`) is the classic TPROXY breaker.

## Workarounds for affected users (until fixed)

- `net.core.devconf_inherit_init_net=2` (new netns get kernel defaults instead
of inheriting init_net's IPv4 conf values) — verified working:
```bash
echo 'net.core.devconf_inherit_init_net = 2' | sudo tee /etc/sysctl.d/99-docker-netns-defaults.conf
sudo sysctl -w net.core.devconf_inherit_init_net=2
```
- Not recommended: clearing `net.ipv4.conf.all.src_valid_mark` on the host —
Tailscale needs it and re-sets it.

## Related, but distinct

While debugging this we also hit an independent local-dev pitfall with
identical symptoms: host firewalls with a default-deny INPUT policy (e.g. plain
ufw) block the sidecar's dial-back to workerd's egress listener on the bridge
gateway (`172.17.0.1:`), because that's container→host INPUT
traffic. `ufw allow in on docker0` fixes it. Possibly worth a note in the local
development docs — both issues produce the same "sidecar churns, app container
never created, 20 s timeout" signature that is otherwise hard to attribute.

Related issues (similar symptoms, different causes as far as I can tell):
cloudflare/workers-sdk#14242, cloudflare/workerd#5996.

The issue does not name a source file or test; start by locating the code that creates the cloudflare/proxy-everything sidecar and builds its Docker HostConfig. Reproduce with net.ipv4.conf.all.src_valid_mark=1, then verify the sidecar handshake succeeds, the application container is created, and the container request returns HTTP 200.

Written by the indexing model from the issue text.