ethereum-optimism / ethereum-optimism/optimism
[Tracking] [Sync Tester] op-sync-tester: service for mock EL sync testing
- Dominant language
- Go
- Stars
- 6.5k
- Forks
- 4k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 134
Description
We need to do more long-range sync testing of the L2 consensus-layer.
But to do so, we are currently stuck creating a full EL deployment: from empty geth DB, or temporary new chain.
The EL chain DB of a real chain is too large to replicate easily for quick tests. And too difficult to roll back efficiently.
Instead, we can mock the execution-layer: by serving the engine API, we can make the CL sync. And we can still serve real blocks; the mock engine API is just a mask over the real chain, to make it appear out of sync.
### spec
This is a new top-level standalone service, with the regular metrics/CLI/RPC setup. See op-faucet for a minimal example.
It serves an RPC to pretend to be an execution engine (we can use JSON-RPC auth, so we can hide the RPC behind a JWT secret, for CI).
RPC methods to serve:
- `eth_getBlockReceipts`: serve if `block num < test_head`
- `eth_getBlockByHash`: serve if `block num < test_head`
- `eth_getBlockByNumber`: serve if `block num < test_head`
- `engine_newPayload{V1,V2,V3,V4}`: check that the payload matches the real one. Error if it does not, and abort the sync test.
- `engine_getPayload{V1,V2,V3,V4}`: return the block that matches the expected block from the block-building job arguments.
- `engine_forkchoiceUpdated{V1,V2,V3}`: check if the forkchoice hashes are canonical. Update the `test_head`, `test_safe`, `test_finalized` in-memory forkchoice values. And check if the block-building attributes (if any) match the inputs to the actual block.
- We should NOT have to support `eth_getStorageAt`, but might if we expand with pre-isthmus fault-proof scope.
- No other RPC methods
And for block-label queries, mock `latest = test_head`, and `safe = test_safe`, `finalized = test_finalized`.
During the sync test, whenever some expectation is broken, abort the test.
Also add the option for an artificial processing delay per RPC method, so we can test what a slow execution engine feels like.
As follow-up in a later iteration of the op-sync-tester we can also mock random errors, like saying the node is syncing, or some temporary error.
### Test Sessions
If we want, we can use the dynamic HTTP routing to make `/synctest?head=100&safe=95&finalized=90` map to a sync-test RPC session.
This way the service never has to restart; different testers can use it in parallel, all with their own test-view of the chain.
Optionally we can support more query params too, e.g. to setup a sync test for the last N blocks, or to select a random range of blocks.
Optionally we can also add an upper-bound, so that the engine API stops responding after reaching a certain block number. This way the test can end more naturally without consuming the full chain.
The RPC utils in op-service already support access to the HTTP router that serves the websocket.
We should add a middle-ware option that parses the HTTP query parameters, and attaches them to the `ctx` that is then passed down to the RPC handlers. And attach some mutable session storage, for the sync progress and latest known block-building random payloadID and attributes.
The RPC handlers can then determine the session from the `ctx` (if we add session middleware, or just use the requester address), as well as the session config / mutable data.
And then just update that session; no global singleton state.
### Usage
- deploy in kurtosis (low priority)
- deploy in devstack sysgo (for testing the test tool)
- deploy stand-alone for CI or local dev: configure in k8s once, on a public endpoint with some auth (maybe even multiple auth tokens), and allow anyone with the auth to sync-test against it.
Both the Rust stack and the Go L2 stack should be able to sync-test against this.
Contributor guide
Assessment
This issue has not been assessed yet.