ChainSafe / ChainSafe/lodestar
[bun] Networking errors escape as uncaught exceptions and N-API teardown can abort
- Dominant language
- TypeScript
- Stars
- 1.4k
- Forks
- 483
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 150
Description
## Describe the bug
A five-node Lodestar fleet running under Bun is healthy enough to advance and maintain peers, but it shows several runtime-specific failures in networking error handling and shutdown paths. A same-window comparison against all non-Bun beacon instances found none of the corresponding errors.
Observed on 2026-08-21 from approximately 15:35–16:18 UTC:
### 1. Expected network aborts escape as global uncaught exceptions
```text
uncaughtException: The operation was aborted
```
- 427 occurrences across all five Bun instances in a 30-minute comparison window.
- Zero occurrences across all non-Bun beacon instances in the same window.
- Representative stack:
```text
destroyWhenAborted (node:net:88:33)
@libp2p/utils queue/job.js
any-signal
uncaughtException
```
This appears related to the abort behavior previously reported in #7280, but the issue remains reproducible at fleet scale.
### 2. Bun network error shape breaks the req/resp peer-scoring path
```text
TypeError: undefined is not an object
(evaluating 'e.type.error.message.includes')
```
Representative stack:
```text
packages/reqresp/src/request/index.ts:160
onOutgoingReqRespError (.../network/reqresp/score.ts:41)
onOutgoingRequestError (.../ReqRespBeaconNode.ts:333)
```
The preceding request log renders the error as:
```text
error=error=[object], local=true, isTrusted=false
```
`onOutgoingReqRespError` assumes `e.type.error.message` exists, but that field is absent for this Bun error. This occurred 16 times across four of the five Bun instances and zero times on non-Bun instances.
### 3. Routine socket failures escape as global uncaught exceptions
```text
uncaughtException: recvmsg ECONNREFUSED
uncaughtException: recvmsg EHOSTUNREACH
```
- 13 occurrences across the Bun fleet in the 30-minute comparison window.
- Zero occurrences on non-Bun beacon instances.
These should remain handled dial/socket failures rather than reaching Lodestar's global uncaught-exception handler.
### 4. Native/N-API teardown can abort the process
During shutdown of one instance, after the network worker was closed and Lodestar logged `terminating network worker`:
```text
assertion `left == right` failed: failed to delete napi ref
thread '' ... panicked at rust/lib.rs:29:1
fatal runtime error: failed to initiate panic, error 5, aborting
```
This was observed on one Bun instance and not on non-Bun instances in the comparison window. The immediate trigger may be in Bun or a native dependency, but it makes graceful shutdown/restart unreliable from Lodestar's perspective.
### 5. Lower-confidence logger/EventEmitter signal
All five Bun instances emitted:
```text
MaxListenersExceededWarning:
Possible EventEmitter memory leak detected.
11 error listeners added to [DailyRotateFile]
```
No equivalent warning appeared on the non-Bun instances. This may be a Bun EventEmitter compatibility difference or a real listener leak exposed by Bun; it needs a focused Bun-vs-Node reproduction before assigning root cause.
## Expected behavior
- Expected aborts and routine socket errors are handled locally and do not reach `uncaughtException`.
- Req/resp scoring safely handles every runtime error shape without throwing a secondary `TypeError`.
- Closing the network worker and native modules completes without a native abort.
- Logger transports do not accumulate unexpected listeners.
## Steps to reproduce
1. Run Lodestar under Bun on a live network with normal peer churn and verbose networking logs.
2. Let the node establish a normal peer set and process req/resp traffic.
3. Observe abort and failed-dial paths for `The operation was aborted`, `recvmsg ECONNREFUSED`, and `recvmsg EHOSTUNREACH` reaching `uncaughtException`.
4. Stop/restart the node repeatedly to exercise network-worker and native/N-API teardown.
5. Compare the same workload under Node.js.
A smaller deterministic reproduction should separately cover:
- aborting a libp2p `node:net` operation;
- passing Bun's outgoing request error into `onOutgoingReqRespError`;
- repeated network-worker/native teardown;
- `DailyRotateFile` listener counts under Bun and Node.
## Additional context
- Fleet: five Bun beacon nodes (`feat1-*`), four Hoodi and one Mainnet.
- Lodestar: `v1.46.0/c746ec6`, commit [`c746ec6a07a98a92c7bdb96bb410db307934f7b8`](https://github.com/ChainSafe/lodestar/commit/c746ec6a07a98a92c7bdb96bb410db307934f7b8).
- Runtime: Bun. The exact Bun release was not included in the collected logs; Bun reported the Node compatibility version as `v26.3.0` through `process.version`/`nodejs_version_info`.
- OS: Linux.
- All five nodes remained up, advancing, and peered after the latest starts. The steady-state errors were noisy/recoverable; the teardown abort was the severe failure.
- Earlier Bun tracking issue: #7280.
Suggested priority:
1. Native/N-API teardown abort.
2. Defensive handling of the error shape in `onOutgoingReqRespError`.
3. Escaped abort and `recvmsg` exceptions.
4. Bun-vs-Node logger listener comparison.
Contributor guide
Research direction
Start with packages/reqresp/src/request/index.ts:160, network/reqresp/score.ts:41, and ReqRespBeaconNode.ts:333, then reproduce Bun versus Node for the listed abort and request-error paths. Also isolate network-worker/N-API teardown and DailyRotateFile listener counts. Done means routine errors stay handled, scoring accepts Bun error shapes, shutdown does not abort, and listener counts remain bounded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bun, node.js, typescript
- Domain
- backend, networking, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100