erigontech / erigontech/erigon

chaintip parallel exec: ~55% of the exec+commitment window is "other" (flush-to-mem + orchestration + stalls), not exec or commitment

Open
#23,073 5 comments 0 reactions 1 assignee Claimed by @sudeepdino008 View on GitHub
performance
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

## Summary

In chaintip parallel execution, the biggest slice of the per-block **exec+commitment window** is neither EVM execution nor commitment computation. Measuring each phase directly, **~55% of the window is "other"** — per-block flush of writes into `sd.mem`, apply-loop / committer orchestration handoffs, and cold-read stalls. Commitment compute is only ~44% of the window, and EVM exec (which overlaps commitment) is ~16%.

This is worth a look because "other" is the largest term and isn't an obvious cost.

## Per-block breakdown (warm, SSD, ~200-block mean)

| Phase | time/block | share of window | notes |
|---|---|---|---|
| exec (EVM, `bdur`) | ~42 ms | ~16% | runs concurrently with commitment (overlaps) |
| commitment compute | ~117 ms | ~44% | measured in the committer (trie fold + root + its reads) |
| **"other"** | **~146 ms** | **~55%** | flush-to-`sd.mem` + apply/committer orchestration + cold-read stalls |
| **exec+commitment window** | **~263 ms** | 100% | this is what `gas/s` is normalized by |
| commit (flush + MDBX commit) | ~409 ms | — | **excluded** from the exec+commitment window / `gas/s` |
| **wall/block** | ~677 ms | — | ≈ window + commit + collate |

`"other" = window − max(exec, commitment)` (exec and commitment overlap, so they are not additive).

## Two takeaways

1. **`gas/s` measures the exec+commitment window only** — it excludes the ~409 ms/block flush+commit. That is the intended behavior (a live node defers persistence to FCU), but it means `gas/s` is not the end-to-end block rate.
2. **Within that window, real commitment compute is only ~44%.** The ~55% "other" term is the surprise and the best optimization target — it is currently unattributed and deserves a profile (flush-to-`sd.mem` cost, stage-to-stage handoff latency in the exec→apply→committer pipeline, and cold mmap-fault stalls are the suspects).

## Setup

- `integration stage_exec --sync.mode.chaintip`, mainnet, parallel executor + BAL-driven commitment, all files on NVMe, warm cache.
- Numbers from a per-block committer-level timer (commitment compute) plus per-phase timers around the exec stage and `doms.Commit`.
- Absolute window varies with block gas (~235–263 ms across ranges); the **~44% commitment / ~55% other** split is the stable result. Avg block in the sampled window: ~37M gas, ~283 txs.
- Instrumentation branch: **`bal_zero`** (`origin/bal_zero` — taratorio's PR #22190 + the temp-BAL feature + these per-phase timers).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.