tsnet magicsock allocates peer-relay VNI to a third-party peer but never handshakes; data stays on DERP

Operating system

macOS

System version

macOS (SFM / sing-box-for-apple Tailscale endpoint). Same userspace tsnet stack is also used on iOS SFI.

Installation type

sing-box for macOS Graphical Client

If you are using a graphical client, please provide the version of the client.

SFM (sing-box-for-apple) Tailscale endpoint. The node advertises 198.18.0.1 as a magicsock endpoint (fake-ip), Hostinfo has no IPNVersion, PeerAPI only. This matches userspace tsnet, not Tailscale.app.

github.com/sagernet/tailscale currently used by sing-box dev is v1.102.1-sing-box-1.14-mod.4 (go.mod). Issues are disabled on https://github.com/SagerNet/tailscale, so this is filed here. Related: #3657 (closed stale; that report was against 1.12.14, before relay_server_port).

Version

# Client is SFM tsnet, not `sing-box` CLI on this Mac.
# Relay and NAS are official tailscaled (control experiment):
#   relay host: 1.98.10 linux
#   NAS:        1.102.2 linux

Description

Tailscale peer-relay server on an official tailscaled node works. The SFM/tsnet client discovers it, and will even complete a relay handshake when the other magicsock is the relay node itself. When the other magicsock is a third-party official client (NAS), tsnet allocates a VNI on the relay and then never sends the UDP handshake. Overlay traffic stays on DERP.

This is not DERP, not an exit node, and not a missing ACL. Official tailscaled on the NAS lists the same relay (debug peer-relay-servers → relay's Tailscale IP). cap/relay is in the netmap.

Expected path order: direct, then peer-relay, then DERP.
Observed for SFM ↔ NAS: DERP only (relay "hkg", 300ms+), direct connection not established, peer-relay VNI stuck at <no handshake> with 0 packets.

Topology

Three nodes, one tailnet:

Node	Client	Role
R (linux)	official tailscaled 1.98.10	peer-relay server only: tailscale set --relay-server-port=40000 --relay-server-static-endpoints=<public-v4>:40000. Also tagged as the campus host. UDP 40000 open on the public NIC.
N (linux NAS)	official tailscaled 1.102.2	overlay destination (SMB). UDP true, public IPv4, MappingVariesByDestIP: false.
C (macOS)	SFM Tailscale endpoint (tsnet)	overlay client. Endpoints include campus LAN 10.x:port, a campus public mapping, and 198.18.0.1:port (fake-ip).

Grant (capability only, no overlay ip on the relay UDP port):

{
  "src": ["tag:nas", "tag:elon"],
  "dst": ["tag:peer-relay"],
  "app": { "tailscale.com/cap/relay": [] }
}

R carries tag:peer-relay. N is tag:nas. C is a personal tagged device. Official docs: src is the accessed stable host, dst is the relay.

R↔N is already direct (~15ms). The slow path is C↔N (SMB overlay).

What works

On R:

$ sudo tailscale debug peer-relay-servers
["100.x.x.R"]

$ sudo tailscale debug peer-relay-sessions
Server port: 40000
Sessions count: 1

VNI: 1
  <C-lan>:port(d:3e93da16…) --> <R>:41641(d:b19cb73e…), Packets: 7 Bytes: 882
  <R>:41641(d:b19cb73e…) --> <C-lan>:port(d:3e93da16…), Packets: 8 Bytes: 1000

tcpdump on R udp port 40000 saw C send UDP length 160 to <R-public>:40000. So tsnet can emit peer-relay probes, at least toward R.

tailscale ping C from R while campus UDP works: direct ~4ms.

What fails (the SMB overlay)

From N (official 1.102.2):

$ tailscale debug peer-relay-servers
["100.x.x.R"]

$ tailscale ping C
pong from C via DERP(hkg) in 300–1800ms
… (8/8)
direct connection not established

$ tailscale status
C   macOS  active; relay "hkg"

On R at the same time, a new session appears for C↔N:

VNI: 5
  <C-lan>:port(d:3e93da16…) --> <no handshake>(d:c6ad8c7c…), Packets: 0 Bytes: 0
  <no handshake>(d:c6ad8c7c…) --> <C-lan>:port(d:3e93da16…), Packets: 0 Bytes: 0

d:c6ad8c7c is N's disco key. The relay allocated the VNI. Handshake never starts.

Simultaneous tcpdump:

• R ens65f1 filter udp port 40000 and not port 3478: 0 packets during N→C ping
• N udp and host <R-public> and port 40000: 0 packets

So neither C nor N sends relay UDP for that pair. ICMP/data stays on DERP. magicsock_send_peer_relay on R stays 0 (R is the server, not the client; expected).

Extra: blocking direct UDP between R and C

On R, INPUT+OUTPUT DROP of C's magicsock UDP endpoints (TCP/SSH left alone). Then tailscale ping C from R:

timeout, timeout,
pong via DERP(hkg) in ~131ms  (×10)
direct connection not established

12 pings, never peer-relay. C still sent a couple of 160-byte probes to :40000; the data path did not switch. After removing the DROP rules, ping returned to direct ~4ms.

Reproduction (minimal)

1. Official tailscaled on host R: tailscale set --relay-server-port=40000 plus a static public ip:port, firewall allow UDP 40000.
2. ACL grant tailscale.com/cap/relay from the node you want to reach (N) to R's relay tag. Do not use src: "*".
3. Official tailscaled on N (any NAT that still has UDP). Confirm tailscale debug peer-relay-servers lists R.
4. sing-box Tailscale endpoint on C (SFM or CLI). Empty extra relay options; C is a client of R, not a relay server.
5. From N: tailscale ping C while C cannot punch direct to N.
6. On R: tailscale debug peer-relay-sessions and tcpdump UDP 40000.

Expected: sessions show a completed handshake for C↔N; ping becomes peer-relay before DERP.
Actual: VNI exists with <no handshake> / 0 bytes; ping stays DERP(...).

CLI-shaped client config (same endpoint type SFM uses):

{
  "log": { "level": "trace" },
  "endpoints": [
    {
      "type": "tailscale",
      "tag": "ts",
      "hostname": "repro-client"
    }
  ]
}

Route 100.64.0.0/10 to ts as usual. Relay server is official tailscaled, not this process.

Logs

No SFM crash. Packet-level evidence is above (peer-relay-sessions, tcpdump, tailscale ping). SFM Hostinfo on C:

Hostname: (OS hostname)
OS: macOS
Services: peerapi4/peerapi6 only
IPNVersion: absent
Endpoints: [campus-public:port, lan:port, 198.18.0.1:port]

Notes

• Official Tailscale has similar “grant correct, never probe / never select peer-relay” reports (tailscale/tailscale#19784, #20785, #18851 with exit node). Here the split is sharper: tsnet completes relay UDP toward the relay node, and does not start it toward a third-party peer.
• R also offers exit node. The C↔N failure was measured from N, which is not using an exit node.
• Please route this to magicsock/peerrelay in github.com/sagernet/tailscale (tsnet); sing-box only constructs tsnet.Server in protocol/tailscale/endpoint.go.

Supporter

• I am a sponsor

Integrity requirements

• I confirm that I have read the documentation, understand the meaning of all the configuration items I wrote, and did not pile up seemingly useful options or default values.
• I confirm that I have provided the server and client configuration files and process that can be reproduced locally, instead of a complicated client configuration file that has been stripped of sensitive data.
• I confirm that I have provided the simplest configuration that can be used to reproduce the error I reported, instead of depending on remote servers, TUN, graphical interface clients, or other closed-source software.
• I confirm that I have provided the complete configuration files and logs, rather than just providing parts I think are useful out of confidence in my own intelligence.

The unchecked item is intentional: the failing client is SFM's tsnet endpoint (graphical). SagerNet/tailscale has issues disabled. A CLI endpoints[].type=tailscale snippet is included so this can be re-run without SFM.

Start at protocol/tailscale/endpoint.go to confirm how sing-box constructs the tsnet endpoint, then trace the tsnet magicsock/peer-relay path for connections to third-party peers. Reproduce the CLI-shaped configuration and inspect peer-relay sessions and UDP captures. Done means the client initiates the relay handshake for C↔N and selects peer-relay instead of remaining on DERP.