erigontech / erigontech/erigon
execmodule: block overlay in SharedDomains + persistent SD across operations
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
## Parent Issue
Part of #19873 — specifically the checkbox:
> Move remaining rawdb RW writes out of execution into SD/memorydb to break the RwTx dependency
## Overview
This issue tracks two incremental steps to decouple block-level metadata writes from `RwTx` and eliminate intermediate DB commits in the Engine API block processing pipeline.
---
## Step 1: Block Overlay in SharedDomains (implemented)
**Branch**: `feat/blockdb-overlay`
Add a `*membatchwithdb.MemoryMutation` block overlay to `SharedDomains`, so `InsertBlocks` can accumulate block metadata writes (headers, bodies, TD) in memory and only hold an `RwTx` during a brief flush.
### Changes
- **`db/state/execctx/domain_shared.go`**: Add `blockOverlay *membatchwithdb.MemoryMutation` field with `InitBlockOverlay(tx, tmpDir)`, `BlockOverlay()` accessor, flush/close/merge lifecycle management.
- **`execution/execmodule/inserters.go`**: `InsertBlocks` now opens a `RoTx`, creates `SharedDomains` + block overlay, writes all block data to the in-memory overlay, then opens a brief `RwTx` only for flush+commit.
- **`db/kv/membatchwithdb/memory_mutation_test.go`**: Moved to external test package (`membatchwithdb_test`) to break an import cycle that previously forced an interface indirection for the block overlay type.
### Result
`InsertBlocks` RwTx hold time drops from "entire block processing loop" to "flush duration only".
---
## Step 2: Persistent SharedDomains across operations (planned)
**Goal**: Eliminate all intermediate DB commits in the standard 1-block Engine API path:
```
NewPayload:
InsertBlocks(block) → writes to overlay, NO commit
ValidateChain(hash) → reads from overlay, NO commit
ForkChoiceUpdate:
UpdateForkChoice(head) → reads from overlay, runs pipeline
→ single commit at the end (can be async)
```
Currently there are 3 DB commits per block (InsertBlocks flush, ValidateChain cleanup, UpdateForkChoice commit). The target is 1 commit per block.
### Design
Hold a persistent `SharedDomains` on `ExecModule` that lives across `InsertBlocks` → `ValidateChain` → `UpdateForkChoice`:
- **InsertBlocks**: Writes to persistent SD's block overlay, returns immediately without flushing
- **ValidateChain**: Reads block data from the persistent overlay instead of from DB
- **UpdateForkChoice**: Adopts the persistent SD, runs pipeline, single flush+commit at the end
Memory pressure safety valves:
- InsertBlocks flushes mid-batch if overlay grows too large (large batch during initial sync)
- UpdateForkChoice's `hasMore` loop continues to flush+commit periodically (existing behavior)
### Key Concerns
- Semaphore already ensures mutual exclusion — only one operation runs at a time
- Error recovery: `resetPersistentSD()` discards accumulated writes on failure
- `blockReader.Header(ctx, tx, ...)` accepts `kv.Tx` — `MemoryMutation` implements this, so reads through the overlay work transparently
Contributor guide
Assessment
This issue has not been assessed yet.