ethereum-optimism / ethereum-optimism/optimism

Cannon: Coverage of non-forge tests for solidity contracts

Open
#13,103 9 comments 0 reactions 0 assignees View on GitHub
A-pkg-contracts-bedrock T-proofs
Dominant language
Go
Stars
6.5k
Forks
4k
Avg merge
2d 15h
Merged PRs (30d)
145

Description

Some contracts in our repository (e.g., [MIPS64.sol](https://github.com/ethereum-optimism/optimism/blob/4f8e9b519c9d10535d438cfe617f3f014df12d01/packages/contracts-bedrock/src/cannon/MIPS64.sol)) are tested outside of forge in a go-ethereum (geth) execution environment.

For Cannon contracts specifically, an EVM is [instantiated](https://github.com/ethereum-optimism/optimism/blob/4f8e9b519c9d10535d438cfe617f3f014df12d01/cannon/mipsevm/testutil/evm.go#L93C1-L129C2) with pre-deployed contracts using forge-generated artifacts. Cannon uses differential testing to verify that the Solidity and offchain VMs produce identical results.
These differential tests are implemented as standard Go tests using the EVM setup described above. This is done for ease of writing tests as Go is a much faster and more expressive than forge for testing VMs. The downside of using Go is that we don't have a way to easily figure out how much of the Solidity VMs these Go tests cover.

To solve this, we need a Go tool/package that can:

1. Take forge/solc artifacts and an EVM execution trace as input
2. Generate lcov-formatted coverage data as output

Ideally, this package is implemented as an [EVM tracing hook](https://github.com/ethereum/go-ethereum/blob/915248cd6bf14b9d1b29db182375bc456d4ef868/core/tracing/hooks.go#L174) so that it can easily be slotted into the cannon EVM driver.

This package can then be later used to generate coverage data from Cannon's Go tests. However, the follow up is out of scope of this ticket.

Contributor guide

Open the contributing guide

Research direction

Start by reading cannon/mipsevm/testutil/evm.go to understand the Cannon EVM setup, then inspect go-ethereum's core/tracing/hooks.go. Define the package inputs around forge/solc artifacts and EVM execution traces, with lcov-formatted coverage as the output. Done means an EVM tracing hook can produce coverage data for Solidity contracts; integrating it into Cannon's Go tests is out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, solidity
Domain
devtools, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.