ChainSafe / ChainSafe/lodestar
bug: Direct-peer connections fail with 'Too many messages for missing streams' during status exchange
- Dominant language
- TypeScript
- Stars
- 1.4k
- Forks
- 483
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 156
Description
## Description
When two Lodestar nodes are configured to peer exclusively via `--directPeers` (with `--discv5=false`), the connection is established but immediately torn down during the status/metadata handshake. This prevents isolated two-node setups from syncing.
## Error Sequence
1. Node B connects to Node A (inbound connection established)
2. Node A attempts status exchange: `Req error method=status, version=2 ... Too many messages for missing streams`
3. Node A marks the request as invalid: `invalid requestStatus peer=`
4. Node A disconnects Node B with a reconnection cool-down: `onLibp2pPeerDisconnect ... coolDownMin=-1`
5. Node B range-sync batches fail with `REQUEST_ERROR_DIAL_ERROR` / `the connection is being closed`
## Reproduction
```bash
# Node A (server)
lodestar beacon --directPeers /ip4/127.0.0.1/tcp/9201/p2p/ \
--targetPeers 1 --disablePeerScoring --supernode ...
# Node B (syncer)
lodestar beacon --discv5=false \
--directPeers /ip4/127.0.0.1/tcp/9300/p2p/ \
--targetPeers 1 --disablePeerScoring ...
```
Node B stays at `connected=0` and cannot sync.
## Environment
- Branch: `test/epbs-e2e-combined` (epbs-devnet-0 base + PR #8985 + PR #8995)
- libp2p: v2.9.0 (yamux stream multiplexer)
- Node.js: v24.13.0
## Analysis
The `Too many messages for missing streams` error originates from the yamux stream multiplexer when multiple req/resp protocols (status, metadata, ping) race during the initial handshake. The rapid exchange creates messages for stream IDs that have not yet been fully opened, causing yamux to reject them.
This only manifests in small peer sets where the same peer is the target of multiple concurrent protocol requests immediately after connection. In larger networks with gradual peer discovery, the handshake completes before the next protocol request fires.
## Impact
- Blocks isolated Lodestar-to-Lodestar e2e testing setups
- Does not affect normal devnet/mainnet operation (peer discovery spreads connections over time)
- Workaround: use mixed-peer discovery instead of strict direct-peer isolation
## Related
Discovered during e2e testing of PR #8985 and PR #8995 on `epbs-devnet-0`.
Contributor guide
Research direction
The report names the libp2p/yamux handshake and the status, metadata, and ping request paths, but no Lodestar files or tests. Reproduce the two-node --directPeers setup first, then trace those protocol requests around connection establishment; done means status/metadata exchange completes without the missing-stream error and range sync succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100