ethereum-optimism / ethereum-optimism/asterisc

Improvements to instruction testing

Open
#128 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
109
Forks
28
PR merge metrics
No merged PRs in 30d

Description

Currently, Asterisc has variety of fuzzing tests and instruction tests.
However, they fail to capture all of the instruction encoding which might arise in production environment. While the whole
combination of encoding parameters are too large, we should at least aim to validate the instruction encoding used in Kona.

For example, the following test executes each instruction in Kona against a VM with zero values in all registers and memory:
```
func TestAllInstructions(t *testing.T) {

testSuiteELF, err := elf.Open("../../../../kona/target/riscv64gc-unknown-none-elf/release-client-lto/kona")
require.NoError(t, err)
defer testSuiteELF.Close()

originalVmState, err := fast.LoadELF(testSuiteELF)
require.NoError(t, err, "must load test suite ELF binary")

contracts := testContracts(t)
addrs := testAddrs
env := newEVMEnv(t, contracts, addrs)

for _, prog := range testSuiteELF.Progs {

// ignore non-executable sections
if (prog.Flags & 1) == 0 {
continue
}

for i := uint64(0); i < uint64(prog.Memsz); i+=4 {

// start with fresh VM and copy 1 instruction from program to memory
// this way we avoid instruction being written over by one another
vmState := fast.VMState {
PC: 0,
Memory: fast.NewMemory(),
Registers: [32]uint64{},
ExitCode: 0,
Exited: false,
Heap: 0x7f_00_00_00_00_00,
}
instr := make([]byte, 4)

originalVmState.Memory.GetUnaligned(prog.Vaddr + i, instr)
vmState.Memory.SetUnaligned(0, instr)

instState := fast.NewInstrumentedState(&vmState, nil, nil, nil)

wit, err := instState.Step(true)
require.NoError(t, err)

fastPostState := vmState.EncodeWitness()
fastRoot, err := fastPostState.StateHash()
require.NoError(t, err)

// Now run the same in slow mode
input, err := wit.EncodeStepInput(fast.LocalContext{})
require.NoError(t, err)
slowRoot, err := slow.Step(input, nil)
require.NoErrorf(t, err, "slow VM err at PC %08x: %v", vmState.PC, err)
if slowRoot != fastRoot {
t.Fatalf("slow state %x must match fast state %x", slowRoot, fastRoot)
}

// Now run the same in EVM implementation
_, evmPostHash, _ := stepEVM(t, env, wit, addrs, i, nil)

if evmPostHash != fastRoot {
t.Fatalf("evm state must match fast state \nfast:%x\nevm: %x\n", evmPostHash, fastRoot)
}
}
}
}
```

While this can't guarantee correctness or equivalency for all initial states, it can guarantee there is at least one "happy path" that is equivalent in the three implementations. Which also means at very least all three implementations decode the instruction in the same way, without raising exceptions.

**Describe the solution you'd like**
We should consider adding tests that validate all instruction encoding contained in the Kona binary in differential tests by applying them to each of the three (slow/fast/sol) implementations.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.