cloudflare / cloudflare/cloudflared
2026.9.0 segfaults in quic-go QueueProbePacket (nil receiver) when running in a Docker bridge network — 2026.8.2 unaffected
- Dominant language
- Go
- Stars
- 15.6k
- Forks
- 1.4k
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
`cloudflared` 2026.9.0 crashes with SIGSEGV roughly every 75 seconds when running
in a Docker container on a bridge network. The panic is a nil-receiver dereference
inside the bundled quic-go fork, on the probe-timeout path.
The same container on 2026.8.2 is completely stable. Notably, a **second tunnel
running the same 2026.9.0 binary on the same machine**, as a host systemd service,
has **zero** crashes — the only meaningful difference is that it uses the host
network directly instead of the Docker bridge/NAT.
Since `QueueProbePacket` is only reached on a PTO (probe timeout), the NAT path
seems to make packet loss / timeouts frequent enough to hit a latent race that the
direct path rarely triggers.
**To Reproduce**
1. Run `cloudflare/cloudflared:2026.9.0` in a Docker container on a user-defined
bridge network (container IP `172.20.0.2`, NAT via iptables MASQUERADE).
2. Command: `tunnel --no-autoupdate --loglevel info run --token `
(remotely-managed tunnel, 3 ingress rules pointing at an internal service).
3. Let it run. It starts cleanly every time — all connectivity pre-checks PASS,
4 connections register (tpe01 / khh01, quic) — then panics after roughly a
minute and is restarted by `restart: unless-stopped`.
Observed: **145 crashes in ~6 hours**.
4. Tunnel ID: <填入或留空>
5. cloudflared config: token-based (remotely managed), no local config file.
**Expected behavior**
The tunnel stays connected without crashing, as it does on 2026.8.2.
**Environment and versions**
- OS: Ubuntu 24.04.5 LTS, kernel 6.8.0-139-generic (Docker container)
- Architecture: AMD64 — Intel Core 2 Duo P8800, 2 cores (old hardware, no AES-NI;
noting it in case goroutine timing is relevant to the race)
- Version: 2026.9.0 (built 2026-09-09-18:43 UTC), GoVersion go1.26.8
- Last known good: 2026.8.2 (built 2026-08-14-12:28 UTC)
**Logs and errors**
[signal SIGSEGV: segmentation violation code=0x1 addr=0x4c pc=0xdae18d]
goroutine 128 [running]:
github.com/quic-go/quic-go/internal/ackhandler.(*sentPacketHandler).getPacketNumberSpace(...)
/go/pkg/mod/github.com/chungthuang/quic-go@v0.45.1-0.20260529212404-a9fddf436fc4/internal/ackhandler/sent_packet_handler.go:374
github.com/quic-go/quic-go/internal/ackhandler.(*sentPacketHandler).QueueProbePacket(0x0?, 0xfc?)
/go/pkg/mod/github.com/chungthuang/quic-go@v0.45.1-0.20260529212404-a9fddf436fc4/internal/ackhandler/sent_packet_handler.go:1042 +0x12d
github.com/quic-go/quic-go.(*Conn).sendProbePacket(0x1524c553b208, 0xd8?, 0x35a126441a6)
/go/pkg/mod/github.com/chungthuang/quic-go@v0.45.1-0.20260529212404-a9fddf436fc4/connection.go:2722 +0x104
github.com/quic-go/quic-go.(*Conn).triggerSending(0x1524c553b208, 0x35a126441a6)
/go/pkg/mod/github.com/chungthuang/quic-go@v0.45.1-0.20260529212404-a9fddf436fc4/connection.go:2486 +0x16a
github.com/quic-go/quic-go.(*Conn).run(0x1524c553b208)
/go/pkg/mod/github.com/chungthuang/quic-go@v0.45.1-0.20260529212404-a9fddf436fc4/connection.go:741 +0x705
github.com/quic-go/quic-go.(*Transport).doDial.func1()
/go/pkg/mod/github.com/chungthuang/quic-go@v0.45.1-0.20260529212404-a9fddf436fc4/transport.go:326 +0x3a
created by github.com/quic-go/quic-go.(*Transport).doDial in goroutine 107
/go/pkg/mod/github.com/chungthuang/quic-go@v0.45.1-0.20260529212404-a9fddf436fc4/transport.go:325 +0x737
The `pc` is identical on every crash (`pc=0xdae18d`), with `addr` varying only
between 0x4b–0x4d — a deterministic nil dereference, not memory corruption.
**Additional context**
- **Not OOM:** `OOMKilled=false`, memory usage 17.4 MiB against a 128 MiB limit.
- **Same binary, same host, no crash:** the host-side systemd tunnel runs the same
2026.9.0 with **0** SIGSEGV over 8+ hours.
- Differences between the two processes on this machine:
- container: Docker bridge NAT (172.20.0.2), ICMP proxy **enabled**
- host: direct on the physical NIC, ICMP proxy **disabled** (ping_group_range)
- `net.core.rmem_max` / `wmem_max` are raised to 7500000 on this host, so both
processes get the full 7 MiB UDP receive buffer they ask for. The stable host
process runs with the same setting, so this does not appear to be the trigger.
- **Rolling the container back to 2026.8.2 resolves it completely:** 0 crashes in
over an hour, where the observed rate would have predicted ~48.
Contributor guide
Research direction
Start with the bundled quic-go fork's internal/ackhandler/sent_packet_handler.go and connection.go at the QueueProbePacket and sendProbePacket stack locations shown in the panic. Compare the 2026.9.0 and 2026.8.2 dependency revisions, then reproduce the tunnel on a Docker bridge network; done means the probe-timeout path no longer produces the SIGSEGV and the container remains connected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100