ethereum-optimism / ethereum-optimism/optimism
opgeth-decoupling: migrate op-e2e/actions onto op-reth-test-engine (subprocess EL)
- Dominant language
- Go
- Stars
- 6.5k
- Forks
- 4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 145
Description
Part of #20257. The **Go-side counterpart** to #20415 — split out so the Rust engine and the Go migration are tracked independently.
## Goal
Rewrite `op-e2e/actions` so its in-test execution layer is the **`op-reth-test-engine`** subprocess (#20415) instead of an in-process op-geth (`*geth.Ethereum` + `core.BlockChain` + the `L2EngineAPI` from the out-of-scope `op-program`). This is the last in-process op-geth consumer; until it lands, `go.mod` cannot flip to upstream go-ethereum (#20266).
## Wire contract (from the #20415 spike)
The engine binary serves **newline-delimited JSON-RPC over a Unix-domain socket** (`reth-ipc` `StreamCodec`, `Separator::Byte(b'\n')`), across three namespaces:
- `engine_*` — `newPayloadV1..V4`, `forkchoiceUpdatedV1..V3`, `getPayloadV1..V4`
- `eth_*` — read-only: `getBlockByNumber`/`ByHash`, `getHeaderByNumber`/`ByHash`, `getBlockReceipts`, `blockNumber`
- `optest_*` — `optest_includeTx(payloadId, rawTx)`, `optest_remainingBlockGas(payloadId)`
go-ethereum's `rpc` IPC client speaks newline-agnostic JSON streaming over UDS, so it is compatible out of the box.
## Scope
### 1. Spawn the subprocess instead of building an in-process EL
Replace [`newBackend` in `op-e2e/actions/helpers/l2_engine.go#L77`](https://github.com/ethereum-optimism/optimism/blob/6640c2b066f93afd0444b1e6cbabde9c495cf75a/op-e2e/actions/helpers/l2_engine.go#L77) with `exec.Command("op-reth-test-engine", "--socket", path, "--genesis", …, "--rollup-config", …)` + an IPC JSON-RPC client. The [`EngineApi *engineapi.L2EngineAPI` field (#L44)](https://github.com/ethereum-optimism/optimism/blob/6640c2b066f93afd0444b1e6cbabde9c495cf75a/op-e2e/actions/helpers/l2_engine.go#L44) and `*geth.Ethereum` go away; subprocess stderr → `t.Log`. Lifecycle: kill on test cleanup; OS reclaims the socket (and the engine auto-removes a stale socket on start).
### 2. Mechanical chain-query migration (~101 sites, ~34 files)
Replace direct `engine.L2Chain().*` reads with `eth_*` client calls (or the header-only variant from #20265). **Accurate count: 101 `L2Chain()` call sites across ~34 files** in `op-e2e/actions/**` (not the rough "~100 across ~14" in #20415) — concentrated in `proofs/*`, `derivation/*`, `upgrades/*`, plus `helpers/l2_engine.go` and `proofs/helpers/runner.go`. Purely mechanical.
### 3. Test extensions → `optest_*`
Replace [`L2EngineAPI.IncludeTx` (`op-program/.../l2_engine_api.go#L140`)](https://github.com/ethereum-optimism/optimism/blob/6640c2b066f93afd0444b1e6cbabde9c495cf75a/op-program/client/l2/engineapi/l2_engine_api.go#L140) and [`RemainingBlockGas` (#L118)](https://github.com/ethereum-optimism/optimism/blob/6640c2b066f93afd0444b1e6cbabde9c495cf75a/op-program/client/l2/engineapi/l2_engine_api.go#L118) calls with `optest_*` client calls. These are concentrated in the `ActL2IncludeTx` / `ActL2IncludeTxIgnoreForcedEmpty` helpers ([`l2_engine.go#L203-L237`](https://github.com/ethereum-optimism/optimism/blob/6640c2b066f93afd0444b1e6cbabde9c495cf75a/op-e2e/actions/helpers/l2_engine.go#L203-L237)) plus a handful of proofs/sequencer tests.
### 4. Reframe the txpool-coupled helper
The current `ActL2IncludeTx` selects the next pending tx via `EngineApi.PendingIndices` + `Eth.TxPool().ContentFrom`. **The new design has no txpool** — `pendingIndices` / `forcedEmpty` / `setForceEmpty` are gone. Reframe these helpers to **sign-and-pass** the chosen tx directly into `optest_includeTx`. The only L2-side txpool injection that needs reframing is [`dencun_fork_test.go#L216`](https://github.com/ethereum-optimism/optimism/blob/6640c2b066f93afd0444b1e6cbabde9c495cf75a/op-e2e/actions/upgrades/dencun_fork_test.go#L216) (`engine.Eth.TxPool().Add`). The `reorg_test`/`safedb_test` `TxPool().Add` calls submit **L1** batcher txs to the L1 miner and are unaffected.
## Dependencies
- **Blocked by #20415** — the `op-reth-test-engine` binary must exist and be callable. This is the only **hard** blocker.
- **Blocks #20266** — this removes the last in-process op-geth consumer; `go.mod` can't flip until it lands.
- Relates: #21182 (op-acceptance-tests Karst+ op-reth sequencing — defers actions/proofs coverage here), #20275 (delete `op-e2e/opgeth/`), #20265 (the ethclient call-site / `wait.go` migration).
### Dependency analysis & minimal-dependency path
The op-reth test engine (#20415) **only delivers value once the action tests switch to it** — so we want to switch as early as possible. The key question: does that switch depend on the op-core extractions (#20260–#20264)? **It does not have to.**
**The switch is independent of op-core if done while still on the op-geth `go.mod`.** The action tests run under the current `go.mod` (op-geth `replace` still in place) until the final flip (#20266). While op-geth backs `go-ethereum`, the *existing* ethclient already decodes the engine's `eth_*` responses — deposit txs (op-geth's `0x7E` `UnmarshalJSON` arm) and OP receipt fields — with **no op-core code**. So steps 1–4 above (spawn subprocess, `L2Chain()` → `eth_*`, `IncludeTx`/`RemainingBlockGas` → `optest_*`, reframe the txpool helper) can all land **before** any op-core package exists.
**Where the op-core dependency actually bites:** only at the **#20266 go.mod flip**. Once `go-ethereum` resolves to *upstream*, those same `eth_*` call sites must decode deposit txs / OP receipts via the OP-aware `sources.EthClient` (#20264) — which is exactly what #20265 migrates the call sites to. So #20264/#20265 are prerequisites of the **flip**, not of the **switch**.
**Recommended sequencing (maximizes early value, minimizes coupling):**
1. **#20415** — build the Rust engine (independent, in parallel now).
2. **#21196 (this issue)** — switch the action tests onto the engine **while on op-geth `go.mod`**. Removes the in-process op-geth EL, validates the engine end-to-end, captures value immediately. **No op-core dependency.**
3. **#20260–#20264** — op-core extractions, in parallel.
4. **#20265** — migrate the (now op-reth-fed) `eth_*` call sites to the OP-aware `sources.EthClient`.
5. **#20266** — flip `go.mod` to upstream go-ethereum.
Net: the only thing #21196 *must* wait for is #20415. The #20264 relationship is a **flip-time** concern handled via #20265, not a gate on switching the sequencer.
## Acceptance criteria
- `op-e2e/actions` no longer imports `github.com/ethereum/go-ethereum/{core,eth,core/state,core/vm}` or `op-program/client/l2/engineapi`; no `*geth.Ethereum` in-process EL remains.
- Action tests pass with the L2 EL provided by the `op-reth-test-engine` subprocess.
- `IncludeTx`/`RemainingBlockGas` flows go through `optest_*`; no `PendingIndices`/`ForcedEmpty`/`SetForceEmpty` remain.
- `dencun_fork_test` L2 tx injection uses `optest_includeTx`.
- With this + #20415 landed, the action-test path is ready for the #20266 cutover.
## Out of scope
- The Rust engine itself (#20415).
- The ethclient call-site / `wait.go` split (#20265) and the `op-e2e/opgeth/` deletion (#20275).
- Migrating action tests to op-devstack-style — parallel effort, not blocked by this.
🤖 *Generated by Claude Code*
Contributor guide
Assessment
This issue has not been assessed yet.