tailscale / tailscale/tailscale

TUN mode fails on UDM Pro: LinuxBypassMark 0x80000 collides with UniFi WireGuard VPN fwmark

Open
#18,443 1 comment 1 reaction 0 assignees View on GitHub
bug OS-linux
Dominant language
Go
Stars
36.5k
Forks
3.2k
Avg merge
2d 3h
Merged PRs (30d)
123

Description

### What is the issue?

# TUN mode fails on UDM Pro: LinuxBypassMark 0x80000 collides with UniFi WireGuard VPN fwmark

## Summary

Tailscale's bypass mark (`LinuxBypassMarkNum = 0x80000`) causes TUN mode to fail on UDM Pro when WireGuard VPN tunnels are configured. The bypass mark value exactly matches UniFi OS's fwmark for **routing table 003**, causing Tailscale control plane traffic to be misrouted through the VPN tunnel instead of WAN.

**Userspace-networking mode works correctly.** This issue only affects TUN mode.

## Environment

| Property | Value |
|----------|-------|
| Device | UDM Pro |
| UniFi OS | 4.4.6 |
| Tailscale Version | 1.92.5 |
| Installation Method | [tailscale-udm](https://github.com/SierraSoftworks/tailscale-udm) |
| TAILSCALED_FLAGS | (empty - TUN mode) |

## Symptoms

1. `tailscale up` hangs indefinitely with no auth URL
2. Verbose logs show connection timeouts to control plane:
```
dial tcp 199.165.136.101:443: i/o timeout
dial tcp [2606:b740:49::105]:443: connect: network is unreachable
```
3. `DefaultRouteInterface` is empty in logs:
```
link state: interfaces.State{defaultRoute= ifs={...}
Rebind; defIf="", ips=[]
```

## Root Cause

### The Collision

Tailscale hardcodes `LinuxBypassMarkNum = 0x80000` in [`tsconst/linuxfw.go`](https://github.com/tailscale/tailscale/blob/main/tsconst/linuxfw.go#L41-L42).

UniFi OS uses the same value `0x80000` for **routing table 003** (assigned to WireGuard VPN tunnels):

```
32515: from all fwmark 0x80000/0x7e0000 lookup 003.wgclt
```

The collision is with **table 003**, not a specific tunnel interface. Any VPN assigned to this routing table will trigger the collision. The tunnel interface name (wgclt5, wgclt7, etc.) is irrelevant - what matters is the routing table assignment.

### Evidence: Packet Routing Test

```bash
# Without mark - correct routing via WAN
$ ip route get 199.165.136.101
199.165.136.101 via dev eth8 table 201.eth8 ✓

# With Tailscale's bypass mark - WRONG routing via VPN tunnel!
$ ip route get 199.165.136.101 mark 0x80000
199.165.136.101 dev wgclt table 003.wgclt ✗
```

(Where `` is whichever WireGuard tunnel is currently assigned to table 003)

### UniFi OS IP Rules

```
0: from all lookup local
5270: not from all fwmark 0x80000/0xff0000 lookup 52 ← Tailscale's rule
32000: from all lookup main
32505: from all fwmark 0x1a0000/0x7e0000 lookup 201.eth8 ← WAN
32507: from all fwmark 0x1c0000/0x7e0000 lookup 202.eth9 ← WAN2
32511: from all fwmark 0x40000/0x7e0000 lookup 001.wgclt3 ← VPN (table 001)
32513: from all fwmark 0x60000/0x7e0000 lookup 002.wgclt4 ← VPN (table 002)
32515: from all fwmark 0x80000/0x7e0000 lookup 003.wgclt ← VPN (table 003 - COLLISION!)
32766: from all lookup 201.eth8
```

**Key insight:** The fwmark is tied to the **routing table number**, not the tunnel interface name:
- Table 001 → fwmark 0x40000
- Table 002 → fwmark 0x60000
- Table 003 → fwmark 0x80000 (collides with Tailscale!)

### Failure Chain

1. `tailscaled` starts in TUN mode
2. Creates control plane socket with `SO_MARK = 0x80000`
3. Packet enters ip rule evaluation:
- Rule 5270: `not fwmark 0x80000` → SKIP (has mark)
- Rule 32000: lookup main → no default route → continue
- Rule 32505-32514: other fwmarks → no match → continue
- **Rule 32515: `fwmark 0x80000/0x7e0000` → MATCH!**
4. Packet routed to VPN tunnel (table 003) instead of eth8 WAN
5. Connection times out

### FWMARK Collision Analysis

UniFi OS uses mask `0x7e0000` (bits 17-22) for WAN/VPN selection. The fwmark is determined by the **routing table number**:

| UniFi Mark | Routing Table | Route | Collision with Tailscale 0x80000? |
|------------|---------------|-------|-----------------------------------|
| 0x1a0000 | 201 | eth8 (WAN) | NO |
| 0x1c0000 | 202 | eth9 (WAN2) | NO |
| 0x40000 | 001 | VPN tunnel | NO |
| 0x60000 | 002 | VPN tunnel | NO |
| **0x80000** | **003** | **VPN tunnel** | **YES - EXACT MATCH** |

Tailscale chose bit 19 (`0x80000`). UniFi uses the same value for **any VPN assigned to routing table 003**.

## Additional Notes

- This issue affects users with WireGuard VPN tunnels configured in UniFi OS
- The collision occurs when **any VPN is assigned to routing table 003**
- The tunnel interface name (wgclt5, wgclt7, etc.) is irrelevant - it's the routing table that matters
- UniFi assigns routing tables dynamically; users cannot control which table a VPN gets
- Pausing/deleting a VPN may cause another VPN to be reassigned to table 003
- The issue is NOT related to `DefaultRouteInterface` detection (that's a separate symptom)

## Related Issues

- #4038 - UDM Pro default route detection (acknowledged but not the root cause)

## System Information

```
$ /usr/bin/ubnt-device-info firmware_detail
4.4.6

$ tailscale version
1.92.5

$ ip rule show | grep -E "5270|0x80000"
5270: not from all fwmark 0x80000/0xff0000 lookup 52
32515: from all fwmark 0x80000/0x7e0000 lookup 003.wgclt

$ ip rule show | grep wgclt
32511: from all fwmark 0x40000/0x7e0000 lookup 001.wgclt3
32513: from all fwmark 0x60000/0x7e0000 lookup 002.wgclt4
32515: from all fwmark 0x80000/0x7e0000 lookup 003.wgclt7 ← table 003 = collision
```

### Steps to reproduce

1. Set up Ubiquiti UDM Pro with WireGuard VPN tunnels configured
2. Ensure at least one VPN is assigned to **routing table 003** (check with `ip rule show | grep "003\."`)
3. Install Tailscale using [tailscale-udm](https://github.com/SierraSoftworks/tailscale-udm) in TUN mode
4. Run `tailscale up`
5. Observe timeout - no auth URL returned
6. Verify with: `ip route get mark 0x80000` - will show VPN tunnel instead of WAN

### Are there any recent changes that introduced the issue?

_No response_

### OS

Linux

### OS version

UniFi OS 4.4.6

### Tailscale version

1.92.5

### Other software

_No response_

### Bug report

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.