erigontech / erigontech/erigon

execmodule: block overlay in SharedDomains + persistent SD across operations

Open
#19,875 2 comments 0 reactions 1 assignee Claimed by @mh0lt View on GitHub
ErigonDB tech debt reduction
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

## Parent Issue

Part of #19873 — specifically the checkbox:
> Move remaining rawdb RW writes out of execution into SD/memorydb to break the RwTx dependency

## Overview

This issue tracks two incremental steps to decouple block-level metadata writes from `RwTx` and eliminate intermediate DB commits in the Engine API block processing pipeline.

---

## Step 1: Block Overlay in SharedDomains (implemented)

**Branch**: `feat/blockdb-overlay`

Add a `*membatchwithdb.MemoryMutation` block overlay to `SharedDomains`, so `InsertBlocks` can accumulate block metadata writes (headers, bodies, TD) in memory and only hold an `RwTx` during a brief flush.

### Changes

- **`db/state/execctx/domain_shared.go`**: Add `blockOverlay *membatchwithdb.MemoryMutation` field with `InitBlockOverlay(tx, tmpDir)`, `BlockOverlay()` accessor, flush/close/merge lifecycle management.
- **`execution/execmodule/inserters.go`**: `InsertBlocks` now opens a `RoTx`, creates `SharedDomains` + block overlay, writes all block data to the in-memory overlay, then opens a brief `RwTx` only for flush+commit.
- **`db/kv/membatchwithdb/memory_mutation_test.go`**: Moved to external test package (`membatchwithdb_test`) to break an import cycle that previously forced an interface indirection for the block overlay type.

### Result

`InsertBlocks` RwTx hold time drops from "entire block processing loop" to "flush duration only".

---

## Step 2: Persistent SharedDomains across operations (planned)

**Goal**: Eliminate all intermediate DB commits in the standard 1-block Engine API path:

```
NewPayload:
InsertBlocks(block) → writes to overlay, NO commit
ValidateChain(hash) → reads from overlay, NO commit

ForkChoiceUpdate:
UpdateForkChoice(head) → reads from overlay, runs pipeline
→ single commit at the end (can be async)
```

Currently there are 3 DB commits per block (InsertBlocks flush, ValidateChain cleanup, UpdateForkChoice commit). The target is 1 commit per block.

### Design

Hold a persistent `SharedDomains` on `ExecModule` that lives across `InsertBlocks` → `ValidateChain` → `UpdateForkChoice`:

- **InsertBlocks**: Writes to persistent SD's block overlay, returns immediately without flushing
- **ValidateChain**: Reads block data from the persistent overlay instead of from DB
- **UpdateForkChoice**: Adopts the persistent SD, runs pipeline, single flush+commit at the end

Memory pressure safety valves:
- InsertBlocks flushes mid-batch if overlay grows too large (large batch during initial sync)
- UpdateForkChoice's `hasMore` loop continues to flush+commit periodically (existing behavior)

### Key Concerns

- Semaphore already ensures mutual exclusion — only one operation runs at a time
- Error recovery: `resetPersistentSD()` discards accumulated writes on failure
- `blockReader.Header(ctx, tx, ...)` accepts `kv.Tx` — `MemoryMutation` implements this, so reads through the overlay work transparently

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.