tailscale / tailscale/ts-plug

Hangs on HTTP requests bodies larger than one tsnet MTU

Open
#11 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
60
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Summary

ts-unplug reverse-proxies fine for small requests, but any request whose body
exceeds roughly one tsnet path-MTU (~1200 bytes of HTTP body) hangs forever
— the
proxy never returns response headers and the client eventually times out. Small
requests through the exact same proxy, tsnet node, and backend succeed in <1 s.

This makes ts-unplug unusable as a proxy for real API clients. Concretely, it
cannot proxy an LLM gateway for the pi coding agent (and
by extension the Anthropic/OpenAI SDKs): every chat request carries a system prompt
plus tool definitions (>1.2 KB), so every request hangs. From the user's side
pi just sits on "Working…" and then reports Operation aborted (its client-side
timeout firing), having received zero bytes.

Environment

  • ts-unplug from github.com/tailscale/ts-plug@v0.0.0-20260627053921-1937455808f0/cmd/ts-unplug/ts-unplug.go
  • Built with go1.26.8, embeds tailscale.com@v1.94.2
  • Host: WSL2 (5.15.167.4-microsoft-standard-WSL2, Ubuntu 24.04), userspace tsnet node
  • tsnet reports a 1360-byte peer MTU to the backend and upgrades to a direct
    connection: magicsock: disco: node ... now using <backend>:16573 mtu=1360
  • Invocation: ts-unplug -dir <dir> -port 8899 <backend-tailscale-ip>

Reproduction

Point ts-unplug at any tailnet HTTP backend and send POST bodies of increasing
size. Everything below goes to the same backend; only the request-body size varies.

probe() {
  pad=$(head -c "$1" /dev/zero | tr '\0' x)
  # build a JSON body of a known size
  body=$(printf '{"pad":"%s"}' "$pad")
  printf 'body=%dB  ->  ' "${#body}"
  timeout 12 curl -s -o /dev/null -w '%{http_code}\n' \
    -H 'content-type: application/json' \
    --data "$body" \
    http://127.0.0.1:8899/<any-endpoint> || echo TIMEOUT
}
for n in 900 1000 1100 1200 1300 1400 2000 4000; do probe "$n"; done
Observed (through ts-unplug)
body= 1064B  ->  200
body= 1164B  ->  200
body= 1214B  ->  TIMEOUT   <-- hangs, no response headers ever arrive
body= 1264B  ->  TIMEOUT
body= 1364B  ->  TIMEOUT
body= 2174B  ->  TIMEOUT
body= 4174B  ->  TIMEOUT

The cliff sits at ~1.2 KB of body — i.e. once the request no longer fits in a
single tsnet MTU (1360) segment.

Control: same large bodies straight to the backend (bypassing ts-unplug)

Sending the identical large bodies directly to the backend over the host's own
Tailscale (kernel WireGuard) path succeeds every time:

body=  1674B  ->  200  0.99s
body=  2174B  ->  200  0.76s
body=  4174B  ->  200  0.69s
body= 20000B  ->  200  0.93s

So the backend and the tailnet are fine; the failure is exclusively in
ts-unplug's userspace tsnet reverse-proxy path
for multi-segment request bodies.

Real-world impact (the pi case)

pi's Anthropic-style request captured off the wire is POST /v1/messages?beta=true
with a 22,835-byte body (system prompt + 4 tool schemas). Through ts-unplug
the proxy logs the inbound request but never produces a response:

REQ  POST /v1/messages          <- request reaches the proxy
(no RESP — upstream never returns headers)
http: proxy error: context canceled   <- 30 s later, client gives up

The same request sent directly to the backend returns a normal streaming
text/event-stream 200 in ~1 s. Stripping the body down to <1.2 KB is the only way
to get a response through ts-unplug, which is obviously not viable for a real
client.

Likely cause

cmd/ts-unplug/ts-unplug.go builds a stock
httputil.NewSingleHostReverseProxy over a tsnet.Server. The hang correlates
exactly with the tsnet peer MTU (1360). This looks like a path-MTU / segmentation
issue in the userspace (gVisor/netstack) datapath: requests that fit in one segment
pass; anything that must be split stalls. tsnet logs also show
magicsock: peermtu: updating peer MTU status ... failed ... disabling.

What I tried

  • Forcing identity content-encoding (ruled out response gzip as the cause).
  • TS_DEBUG_MTU=1280 — could not get a clean run to confirm; a documented, working
    way to clamp the tsnet MTU (or fix PMTUD) from ts-unplug would help.

Ask

  1. Confirm whether this is a tsnet PMTU/segmentation bug or a ts-unplug datapath
    configuration gap.
  2. Expose a flag/env to clamp the tsnet MTU (e.g. 1280) from ts-unplug, and/or fix
    large-body handling so it works out of the box.

Happy to provide full -debug-tsnet netmap/packet-filter captures on request.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cmd/ts-unplug/ts-unplug.go and reproduce the failure with the provided curl body-size loop, then inspect how the tsnet.Server is used with httputil.NewSingleHostReverseProxy and how MTU settings are exposed. Done means multi-segment request bodies, including the 22,835-byte example, receive normal response headers and complete without a timeout; document whether MTU clamping or a datapath fix resolves it.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.