erigontech / erigontech/erigon

exec3: parallel execution performance improvements

Open
#19,826 1 comment 0 reactions 1 assignee Claimed by @mh0lt View on GitHub
performance
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

## Overview

Track performance improvements for the parallel execution pipeline (`exec3`). The goal is to get steady-state block processing as close to flat GC allocation as possible — block processing is cyclical, so with no RPC traffic the memory profile should be flat.

## Current observations (branch: exec3-parallel-direct-finalize)

- **Parallel alloc ~13.5GB vs serial ~7.5GB** — nearly 2x memory overhead from version maps, read/write sets, and TxResult collector buffers
- **RPC tip staleness** — `eth_blockNumber` can lag 60+ blocks behind actual committed tip during batch processing. Root cause: shared domains not shared with RPC provider, and RPC waits for DB transaction rotation instead of reading from in-memory cache
- **Batch buffer overflow** — `buf=510MB/500MB` seen in logs, suggesting batch size accounting isn't tight enough

## Improvement areas

### Memory / GC
- [ ] Profile heap allocations per block — identify top allocation sources (VersionedWrite, VersionedRead, ReadSet maps, etc.)
- [ ] Pool/reuse TxResult and collector write slices across transactions instead of fresh allocation each TX
- [ ] Reduce version map overhead — explore more compact representations
- [ ] Evaluate read/write set data structure alternatives (sorted slices vs maps)
- [ ] Track alloc baseline between GC cycles — should be flat sawtooth, not ratcheting

### RPC tip latency
- [ ] RPC layer should read from committed in-memory state, not wait for MDBX transaction rotation
- [ ] Ensure forkchoice update notifies RPC provider immediately when canonical head advances
- [ ] Investigate notification path: execution commit → domain update → RPC visibility

### Batch sizing
- [ ] Tighten batch buffer accounting to avoid >500MB overflows
- [ ] Consider adaptive batch sizing based on block gas usage

### Finalize path (follow-on from #19824)
- [ ] Data structure optimization for VersionedWrites/ReadSet (currently append-only slices with linear scans)
- [ ] Reduce allocations in `finalizeTx` direct path

## Measurement

- `pprof_monitor.sh` captures periodic heap profiles + metrics CSV for parallel vs serial comparison
- `go tool pprof -diff_base=early.pb.gz late.pb.gz` to identify growing allocations
- Monitor: block number, heap alloc, sys memory, tip age — every 60s

## Related
- #19824 — Extract EEST edge cases into unit tests + finalize data structure optimization
- #19814 — Direct finalize path (exec3-parallel-direct-finalize branch)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.