ethereum-optimism / ethereum-optimism/optimism

[Tracking] [Sync Tester] op-sync-tester: service for mock EL sync testing

Open
#16,452 2 comments 6 reactions 1 assignee Assigned to @pcw109550 View on GitHub
A-op-sync-tester go H-interop
Dominant language
Go
Stars
6.5k
Forks
4k
Avg merge
2d 18h
Merged PRs (30d)
134

Description

We need to do more long-range sync testing of the L2 consensus-layer.

But to do so, we are currently stuck creating a full EL deployment: from empty geth DB, or temporary new chain.
The EL chain DB of a real chain is too large to replicate easily for quick tests. And too difficult to roll back efficiently.

Instead, we can mock the execution-layer: by serving the engine API, we can make the CL sync. And we can still serve real blocks; the mock engine API is just a mask over the real chain, to make it appear out of sync.

### spec

This is a new top-level standalone service, with the regular metrics/CLI/RPC setup. See op-faucet for a minimal example.

It serves an RPC to pretend to be an execution engine (we can use JSON-RPC auth, so we can hide the RPC behind a JWT secret, for CI).

RPC methods to serve:
- `eth_getBlockReceipts`: serve if `block num < test_head`
- `eth_getBlockByHash`: serve if `block num < test_head`
- `eth_getBlockByNumber`: serve if `block num < test_head`
- `engine_newPayload{V1,V2,V3,V4}`: check that the payload matches the real one. Error if it does not, and abort the sync test.
- `engine_getPayload{V1,V2,V3,V4}`: return the block that matches the expected block from the block-building job arguments.
- `engine_forkchoiceUpdated{V1,V2,V3}`: check if the forkchoice hashes are canonical. Update the `test_head`, `test_safe`, `test_finalized` in-memory forkchoice values. And check if the block-building attributes (if any) match the inputs to the actual block.
- We should NOT have to support `eth_getStorageAt`, but might if we expand with pre-isthmus fault-proof scope.
- No other RPC methods

And for block-label queries, mock `latest = test_head`, and `safe = test_safe`, `finalized = test_finalized`.

During the sync test, whenever some expectation is broken, abort the test.

Also add the option for an artificial processing delay per RPC method, so we can test what a slow execution engine feels like.

As follow-up in a later iteration of the op-sync-tester we can also mock random errors, like saying the node is syncing, or some temporary error.

### Test Sessions

If we want, we can use the dynamic HTTP routing to make `/synctest?head=100&safe=95&finalized=90` map to a sync-test RPC session.
This way the service never has to restart; different testers can use it in parallel, all with their own test-view of the chain.

Optionally we can support more query params too, e.g. to setup a sync test for the last N blocks, or to select a random range of blocks.

Optionally we can also add an upper-bound, so that the engine API stops responding after reaching a certain block number. This way the test can end more naturally without consuming the full chain.

The RPC utils in op-service already support access to the HTTP router that serves the websocket.

We should add a middle-ware option that parses the HTTP query parameters, and attaches them to the `ctx` that is then passed down to the RPC handlers. And attach some mutable session storage, for the sync progress and latest known block-building random payloadID and attributes.

The RPC handlers can then determine the session from the `ctx` (if we add session middleware, or just use the requester address), as well as the session config / mutable data.
And then just update that session; no global singleton state.

### Usage

- deploy in kurtosis (low priority)
- deploy in devstack sysgo (for testing the test tool)
- deploy stand-alone for CI or local dev: configure in k8s once, on a public endpoint with some auth (maybe even multiple auth tokens), and allow anyone with the auth to sync-test against it.

Both the Rust stack and the Go L2 stack should be able to sync-test against this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.