bytecodealliance / bytecodealliance/wasmtime

ISLE: fuzz generated code with on-the-fly `Arbitrary` extractor results

Open
#3,574 1 comment 0 reactions 0 assignees View on GitHub
enhancement isle
Dominant language
Rust
Stars
18.6k
Forks
1.8k
Avg merge
1d 18h
Merged PRs (30d)
126

Description

In [this comment](https://github.com/bytecodealliance/wasmtime/pull/3560#discussion_r757955589) on ISLE documentation, @avanhatt mentioned fuzzing as related to the `Context` trait that separates generated code from the Cranelift glue. This got me thinking that there is actually a nice way we could fuzz the ISLE-generated code independently.

Specifically, we could write an alternate implementation of the `Context` that stubs out external constructors, and that implements all external extractors in terms of an `arbitrary::Unstructured` state that it carries. As the pattern-matching makes queries about its input by calling external extractors, the context object could invent answers on the fly with `Arbitrary` implementations.

This would allow us both to test the generated code to look for unexpected panics -- for example, if we have `unreachable!()` anywhere -- and to look for inefficiencies/timeouts, e.g. if there is an infinite recursion.

There is some maintenance effort involved (separate implementation of each external etor/ctor that we add) but it might be worthwhile; thoughts?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.