erigontech / erigontech/erigon
chaintip parallel exec: ~55% of the exec+commitment window is "other" (flush-to-mem + orchestration + stalls), not exec or commitment
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
## Summary
In chaintip parallel execution, the biggest slice of the per-block **exec+commitment window** is neither EVM execution nor commitment computation. Measuring each phase directly, **~55% of the window is "other"** — per-block flush of writes into `sd.mem`, apply-loop / committer orchestration handoffs, and cold-read stalls. Commitment compute is only ~44% of the window, and EVM exec (which overlaps commitment) is ~16%.
This is worth a look because "other" is the largest term and isn't an obvious cost.
## Per-block breakdown (warm, SSD, ~200-block mean)
| Phase | time/block | share of window | notes |
|---|---|---|---|
| exec (EVM, `bdur`) | ~42 ms | ~16% | runs concurrently with commitment (overlaps) |
| commitment compute | ~117 ms | ~44% | measured in the committer (trie fold + root + its reads) |
| **"other"** | **~146 ms** | **~55%** | flush-to-`sd.mem` + apply/committer orchestration + cold-read stalls |
| **exec+commitment window** | **~263 ms** | 100% | this is what `gas/s` is normalized by |
| commit (flush + MDBX commit) | ~409 ms | — | **excluded** from the exec+commitment window / `gas/s` |
| **wall/block** | ~677 ms | — | ≈ window + commit + collate |
`"other" = window − max(exec, commitment)` (exec and commitment overlap, so they are not additive).
## Two takeaways
1. **`gas/s` measures the exec+commitment window only** — it excludes the ~409 ms/block flush+commit. That is the intended behavior (a live node defers persistence to FCU), but it means `gas/s` is not the end-to-end block rate.
2. **Within that window, real commitment compute is only ~44%.** The ~55% "other" term is the surprise and the best optimization target — it is currently unattributed and deserves a profile (flush-to-`sd.mem` cost, stage-to-stage handoff latency in the exec→apply→committer pipeline, and cold mmap-fault stalls are the suspects).
## Setup
- `integration stage_exec --sync.mode.chaintip`, mainnet, parallel executor + BAL-driven commitment, all files on NVMe, warm cache.
- Numbers from a per-block committer-level timer (commitment compute) plus per-phase timers around the exec stage and `doms.Commit`.
- Absolute window varies with block gas (~235–263 ms across ranges); the **~44% commitment / ~55% other** split is the stable result. Avg block in the sampled window: ~37M gas, ~283 txs.
- Instrumentation branch: **`bal_zero`** (`origin/bal_zero` — taratorio's PR #22190 + the temp-BAL feature + these per-phase timers).
Contributor guide
Assessment
This issue has not been assessed yet.