Management interface: epoll busy loop when remote client dies without closing connection
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 14.6k
- Forks
- 3.4k
- PR merge metrics
- No merged PRs in 30d
Description
Describe the bug
When a remote management interface client dies (e.g. crash, power loss) without cleanly closing its TCP
connection, the OpenVPN server process enters a busy loop consuming 100% of a CPU core. The process spins
indefinitely in an epoll_wait / recvfrom cycle on the dead management connection's file descriptor.
The root cause is that the management connection socket enters an inconsistent state where epoll_wait
(level-triggered EPOLLIN) reports readiness, but recvfrom returns EAGAIN. Since OpenVPN uses
level-triggered epoll, this creates an infinite loop:
epoll_wait(9, [{events=EPOLLIN, data={u32=4, u64=4}}], 205, 10000) = 1
recvfrom(10, 0x7ffe05885ec0, 256, MSG_NOSIGNAL, NULL, NULL) = -1 EAGAIN
epoll_wait(9, [{events=EPOLLIN, data={u32=4, u64=4}}], 205, 10000) = 1
recvfrom(10, 0x7ffe05885ec0, 256, MSG_NOSIGNAL, NULL, NULL) = -1 EAGAIN
... (~18,800 iterations/sec)
This continues until the service is manually restarted.
To Reproduce
- Configure OpenVPN server with
management 0.0.0.0 <port>(externally accessible) - Connect a management client from a remote host
- Poweroff the client(not use command, click button) or unplug the network cable
- Observe the OpenVPN server process consuming 100% CPU on one core
The issue persists until the service is restarted. System TCP keepalive (tcp_keepalive_time defaults to 7200s)
does not resolve the socket state quickly enough, and even after keepalive would have fired, the socket may
remain in the broken state.
Expected behavior
OpenVPN should handle EAGAIN from recvfrom on the management socket gracefully — either by:
- Removing the fd from the epoll interest set after repeated
EAGAINand closing the stale connection, or - Using
SO_KEEPALIVEwith reasonable timeouts on management connections to detect dead peers, or - Providing a
management-timeoutconfiguration option to allow administrators to set an idle timeout for
management connections
Version information
- OS: Ubuntu 24.04.1 LTS (Noble Numbat), kernel 6.8.0-51-generic
- OpenVPN version: 2.6.12 (
openvpn 2.6.12-0ubuntu0.24.04.3) - OpenVPN build:
x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO] - OpenSSL: 3.0.13
Additional context
strace -c output over a 3-second sample during the busy loop:
% time seconds usecs/call calls errors syscall
--- 51.05 0.266354 4 56291 epoll_wait
48.95 0.255355 4 56291 56291 recvfrom
---100.00 0.521709 4 112582 56291 total
ss output showing the problematic socket (fd 10) plus two additional stale CLOSE-WAIT connections on the
management port:
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 2 1 0.0.0.0:7507 0.0.0.0:* users:(("openvpn",pid=1783203,fd=3))
CLOSE-WAIT 8 0 ********:7507 ********:51738
CLOSE-WAIT 8 0 ********:7507 ********:57004
ESTAB 0 0 ********:7507 ********:48060 users:(("openvpn",pid=1783203,fd=10))
The server had been running for 407 days and is a single-vCPU VM, so the busy loop effectively made the entire
server unresponsive. Restarting openvpn@server-tcp immediately resolved the issue (CPU idle went from 0% to
91%).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the management-interface socket handling around epoll_wait and recvfrom, using the reproduction steps and strace output in this issue. Reproduce the dead-client case and verify that the server no longer spins at 100% CPU and handles the stale connection without requiring a restart.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100