perf: req/resp handler holds forkchoice exclusive lock during block processing on libp2p thread
- Dominant language
- Zig
- Stars
- 97
- Forks
- 39
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
The libp2p rust bridge runs on its own thread (`rustBridgeThread`). `onReqRespResponse` (handling blocks received from peers) fires on that thread and calls directly into `processBlockByRootChunk` → `chain.onBlock()` → `forkChoice.onBlock()` (exclusive `mutex.lock()`) + `forkChoice.updateHead()` (exclusive lock).
Meanwhile the main libxev event loop calls `forkChoice.onInterval()` (also exclusive mutex). During block sync when many blocks are incoming these exclusive locks contend and can stall both sync and the main tick.
**Serving side** (`onReqRespRequest`) is mostly fine:
- `blocks_by_root`: DB-only, no forkchoice lock ✅
- `status`: two brief `lockShared()` calls — OK but blocked if a writer holds the exclusive lock
## Fix
1. **Response handling**: `onReqRespResponse` should not call `chain.onBlock()` directly on the libp2p thread. Received blocks should be queued into a channel/ring buffer and consumed by the main event loop. This is exactly the architecture described in #700 (`xev.Async` dispatch).
2. **Serving status**: Cache an atomically-updated `Status` snapshot (updated in `onBlockFollowup` whenever head/finalized changes). The `status` req/resp handler then serves it lock-free.
3. **General**: Any read from forkchoice for serving req/resp should use a snapshot — take the lock briefly, copy the needed fields, release, then do the work.
## Related
- #700 — long-term fix: `xev.Async` lock-free dispatch (eliminates the need for any lock here)
- PR #695 — introduced `state_mutex` as interim fix for the same class of problem
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with onReqRespResponse, processBlockByRootChunk, onBlockFollowup, and onReqRespRequest, then read issue #700 and PR #695 for the intended dispatch and mutex context. Trace how blocks reach the main libxev event loop and how status is served. Done means response processing no longer holds forkchoice locks on the libp2p thread and status serving uses the described snapshot approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- zig
- Domain
- distributed-systems, networking, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100