erigontech / erigontech/erigon
tracking: execution module consolidation (PipelineExecutor, notifications, Cache)
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
## Summary
Tracking issue for consolidating execution pipeline ownership into `execmodule`. This work makes the execution module more self-contained and aligns with the broader 2-cache rationalization (#19623).
## Work Streams
### 1. PipelineExecutor (feat/pipeline-executor)
Consolidate all staged sync pipeline invocations into a single `PipelineExecutor` struct:
- `ProcessFrozenBlocks` (startup)
- `RunLoop` (FCU catchup)
- `ValidateBlock` (fork validation)
Move `ForkValidator` from `engine_helpers` to `execmodule`, remove the `validatePayloadFunc` closure, decouple `stage_finish` from `ForkValidator`.
**Status**: Branch `feat/pipeline-executor` — complete, waiting for review of dependencies.
### 2. Pre-commit notifications (#19957)
Move `Hook.AfterRun` to fire before DB commit. No consumer reads from DB after receiving a notification — all payloads are self-contained. The pipeline's RwTx/overlay has all data needed for reads.
**Status**: Branch `feat/pre-commit-notifications` — complete, waiting for review of dependencies.
### 3. Eliminate fcuOverlay in updateForkChoice
The `fcuOverlay` (MemoryMutation) in `updateForkChoice` is redundant — the SD already has `blockOverlay` for table-level writes (headers, bodies, canonical hashes, stage progress, forkchoice hashes). Consolidate:
- Remove the separate `fcuOverlay`; use `sd.BlockOverlay()` as the pipeline `tx`
- `sd.Flush()` handles both domain + table writes in one call
- Simplifies CommitCycle (no separate overlay to flush/recreate)
- Simplifies `runForkchoiceCommitOverlay` (drops overlay parameter)
- Lazy revert unchanged: `sd.Close()` discards everything
**Status**: Design complete, implementation pending.
### 4. Remove `tx` parameter from pipeline (final step)
Currently `RunLoop`, `Sync.Run`, `RunUnwind`, `RunPrune`, and every stage func take `(sd, tx)` separately. Now that `tx == sd.BlockOverlay()`, the `tx` parameter is redundant. Remove it:
- `RunLoop(ctx, sd, cfg)` instead of `RunLoop(ctx, sd, tx, cfg)`
- `Sync.Run(sd, initialCycle, firstCycle)` instead of `Sync.Run(sd, tx, ...)`
- Each stage calls `sd.BlockOverlay()` when it needs table access
- Purely mechanical refactor across all stages
**Status**: Deferred to last — big diff, easier when everything else is stable.
### 5. Cache cleanup (future)
The `execmodule.Cache` shim bridges RPC reads to SharedDomains for domain data. It predates SD's current capabilities. Cleanup deferred until #19623 Phase 5 stabilizes the `StateReader`/`CacheView` interfaces.
## Relationship to #19623
These changes prepare the ground for the 2-cache rationalization:
| This work | Enables in #19623 |
|-----------|-------------------|
| PipelineExecutor centralizes pipeline calls | Cleaner integration point for Phase 3 synchronous domain apply |
| Pre-commit notifications | Aligns with SD as authoritative read layer (Phase 3+) |
| Eliminate fcuOverlay | SD owns all in-flight state; one flush path |
| Remove tx from pipeline | SD is the single interface for stages — domain + table access |
| Cache documents SD dependency | Cleanup path clear once interfaces stabilize (Phase 5) |
| `controlServer` interface narrowing (future) | Removes p2p/sentry dependency from validation pipeline |
## Order of Operations
1. ~~PipelineExecutor~~ (done)
2. ~~Pre-commit notifications~~ (done)
3. **Eliminate fcuOverlay** ← next
4. Remove `tx` from pipeline ← last (mechanical, big diff)
5. Cache cleanup (after #19623 Phase 5)
## Related Issues
- #19623 — Rationalize IntraBlockState to a 2-Cache Model
- #19798 — Extract apply loop into event stream with fan-out consumers
- #19855 — TransactionState/BlockState separation, TxTask refactoring
- #19957 — Send notifications before DB commit
Contributor guide
Assessment
This issue has not been assessed yet.