Orchestrator HTTP server spins at 100% CPU forever: node:http never reaps CLOSE_WAIT sockets (EPOLLRDHUP storm)

## Summary

The orchestrator's HTTP server burns **exactly one full CPU core, continuously and indefinitely**, once any upstream proxy half-closes a keep-alive connection. The process never recovers on its own.

Observed on `chrisleekr/github-app:1.15.0-orchestrator` (Bun 1.3.14) running behind ingress-nginx. The pod had burned **13.93 days of CPU over 13.95 days of wall-clock (99.88%)** before it was restarted.

This is **not** a timer/poller problem, and not a proxy misconfiguration. It is a socket-lifecycle defect in Bun's `node:http` layer that this app is exposed to because `src/app.ts` never sets a server timeout.

## Impact

- ~1 CPU core wasted per orchestrator pod, permanently, from ~20 minutes after startup.
- The spin is **binary, not cumulative**: a *single* stuck socket pegs the event-loop thread at 100%. Seven stuck sockets cost exactly the same as one. This is why CPU is flat rather than ramping, and why it is easy to mistake for "normal load".
- Restarting the pod is **not** a workaround. The pod was idle for only ~22 minutes of its 13.95-day life, i.e. the spin re-established itself within ~20 minutes of boot.

## Evidence

`strace` on the main thread (PID 1), ~8 seconds:

```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
62.75 2.732801 5 517658 clock_gettime
36.99 1.610774 6 258776 epoll_pwait2
0.09 0.003796 36 104 13 futex
0.07 0.003124 72 43 sched_yield
0.02 0.000804 15 53 recvfrom
0.00 0.000031 15 2 close
0.00 0.000013 6 2 timerfd_settime
```

`epoll_pwait2` is called **~32,000 times per second with zero errors**. `clock_gettime` runs at exactly 2.0002x that rate (the event loop timing each iteration).

The raw calls show exactly what it is spinning on:

```
epoll_pwait2(4, [{events=EPOLLRDHUP, data=0x575dfe20f00},
{events=EPOLLRDHUP, data=0x575dfe20400},
{events=EPOLLRDHUP, data=0x575dfe22700},
{events=EPOLLRDHUP, data=0x575dfe22b00},
{events=EPOLLRDHUP, data=0x575dfe20900},
{events=EPOLLRDHUP, data=0x575dfe22e00},
{events=EPOLLRDHUP, data=0x575dfe22200}],
1024, {tv_sec=0, tv_nsec=78481990}, [], 8) = 7
```

Every call returns the **same 7 fds**, all `EPOLLRDHUP`, with identical `data` pointers. Bun asks to wait ~78ms (`tv_nsec` visibly counts down toward its next timer deadline across successive calls) but epoll returns **instantly**, because those 7 fds are permanently ready.

Those 7 fds correspond one-for-one with 7 sockets stuck in `CLOSE_WAIT`:

```
State Recv-Q Local Address:Port Peer Address:Port
CLOSE-WAIT 1 [::ffff:10.0.0.206]:3000 [::ffff:10.0.0.67]:48232
CLOSE-WAIT 1 [::ffff:10.0.0.206]:3000 [::ffff:10.0.0.67]:42786
... (7 total)
```

`10.0.0.67` is the ingress-nginx controller. `Recv-Q=1` is the undrained FIN.

Supporting measurements:

- Exactly **1.000 cores**, flat for 24h (Prometheus `rate(container_cpu_usage_seconds_total[5m])`).
- `nr_throttled 0`, `throttled_usec 0` against `cpu.max = 200000 100000` (a 2-core limit) — the process *could* use 2 cores and is not being capped. The spin is genuine.
- `user_usec` 56.0% / `system_usec` 44.0% — the kernel half is the `epoll_pwait2` storm.
- Only **2.9 voluntary context switches/sec** while burning a full core: the thread essentially never sleeps.
- Per-thread accounting: thread 1 holds 120,353,370 of 120,387,653 ticks (99.97%). Every other thread is idle.

**Timers are excluded:** `timerfd_settime` was called **2 times** in 8 seconds. `proposal-poller` runs at `intervalMs=90000` and `scheduler` at `intervalMs=300000`. Neither can produce a 32kHz loop.

## Mechanism

1. ingress-nginx is configured with `upstream-keepalive-timeout: 5` — it closes idle upstream connections after 5s. This is normal, correct proxy behaviour.
2. Bun's `node:http` server uses a default idle timeout of **10s** (uWS `HTTP_IDLE_TIMEOUT_S`, see oven-sh/bun#12446). Because 10s > 5s, **nginx always closes first**.
3. The peer's FIN arrives. Our socket enters `CLOSE_WAIT`. Bun registered `EPOLLRDHUP` interest on it.
4. Bun **never** handles the `EPOLLRDHUP`: it does not `close()` the fd, nor `epoll_ctl(EPOLL_CTL_DEL)` it. (`close` fired 2x and `epoll_ctl` 6x across 8 seconds — nowhere near the 7 needed.)
5. epoll is **level-triggered**, so the fd stays permanently ready. `epoll_pwait2` returns immediately, forever. One core, gone.

It is a **race, not a rule** — nginx closed thousands of connections over 14 days and only 7 stuck. So timeout tuning changes the odds, not the mechanism.

## Suggested fix

`src/app.ts:326` never sets a server timeout:

```ts
const server = http.createServer((req, res) => { ... }); // line 244
server.listen(config.port, () => { ... }); // line 326
```

Per a Bun maintainer in [oven-sh/bun#12446](https://github.com/oven-sh/bun/issues/12446), the supported knob for `node:http` servers in Bun is **`server.setTimeout()`** (since v1.1.27) — **not** `server.keepAliveTimeout`, which Bun does not honour the same way as Node.

Options, roughly in order of preference:

1. **Set `server.setTimeout()` below the proxy's idle timeout** so *we* close first and never enter `CLOSE_WAIT`. Note this inverts the usual advice (for AWS ALB you want the server timeout *longer* than the LB's, to avoid a 502 race). nginx's `proxy_next_upstream error` retries connection-level failures by default, which makes the trade more acceptable here than it would be behind an ALB — but it is a real trade and should be verified under load.
2. **Move the server to `Bun.serve()`**, which supports `idleTimeout` natively and avoids the `node:http` shim entirely. Larger change; the app is already on Hono, which has a native Bun adapter.
3. **Report upstream to Bun.** I could not find an existing issue for `EPOLLRDHUP`/`CLOSE_WAIT` spin in `node:http`. The closest is [oven-sh/bun#33699](https://github.com/oven-sh/bun/issues/33699) (`node:http` leaking on client disconnect, confirmed on 1.3.14, closed 2026-07-07) — same subsystem, different symptom. Bun 1.3.14 is the latest release (2026-05-13), so there is no version to upgrade to.

Whatever the app does, the underlying Bun defect should be fixed upstream: a half-closed peer connection must not be able to peg an event loop forever.

## Environment

| | |
|---|---|
| Image | `chrisleekr/github-app:1.15.0-orchestrator` |
| Bun | 1.3.14 (latest release, 2026-05-13) |
| Server | `node:http` via `http.createServer` (`src/app.ts:244`), Hono 4.12.25 |
| Platform | Kubernetes 1.33.4, Linux x64, Debian 13 (trixie) base |
| Proxy | ingress-nginx, `upstream-keepalive-timeout: 5`, `upstream-keepalive-time: 30s` |
| Container | `requests: cpu=500m/512Mi`, `limits: cpu=2/2Gi`, Burstable, caps dropped ALL, uid 1000 |

## Reproduction sketch

Not yet reduced to a standalone case. Expected shape:

1. `http.createServer` under Bun 1.3.14, listening, no `setTimeout` set.
2. A client that opens keep-alive connections and half-closes them after a shorter idle period than Bun's 10s default (mimicking `upstream-keepalive-timeout: 5`).
3. Drive traffic until a connection lands in `CLOSE_WAIT` (it is a race; may take a while).
4. Observe: `ss -tan state close-wait` non-empty, and CPU pinned at exactly 1.00 core with `strace` showing `epoll_pwait2` returning `EPOLLRDHUP` in a tight loop.

A minimal repro is the main thing needed before filing upstream with Bun.