bytecodealliance / bytecodealliance/sightglass

Benchmark more regex variations

Open
#226 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
WebAssembly
Stars
88
Forks
37
Avg merge
5h 5m
Merged PRs (30d)
1

Description

We have a regex benchmark merged in #224. But there are a lot of variations between regex implementations. Even the `regex` crate alone has several tunable parameters that I'd expect to have a significant impact on this benchmark.

Some thoughts if anyone wants to dig into the behavior of this benchmark or add more regex benchmarks:

It looks to me like the regular expressions in #224 are all intended to match only ASCII text. If so, `RegexBuilder::new(pattern).unicode(false)` should produce much smaller automata for the email and URI patterns. (I don't expect any impact on the IP-address pattern.) That shouldn't affect how long we spend compiling the resulting wasm program, but I'd expect a runtime speedup.

I'm guessing the reason this wasm is so large is mostly due to including all the Unicode character class tables. Disabling default features for the `regex` crate should make it quite a bit smaller. For a fair comparison we'd want to enable the `perf` feature to get all the default performance optimizations back. Also the docs say the `std` feature is currently required in any case.

I would also be interested to know what impact that `perf` feature has on both compile time and runtime for this benchmark. In particular, `perf-dfa` should significantly change the control-flow patterns in the core matching loop, and that might hit better or worse codegen patterns in Cranelift.

I don't think the `perf-literal` feature will actually get used on any of these benchmarks, since the longest literal string is "://", so disabling that should save some compile time without much impact on runtime.

As a more extreme experiment it would be interesting to pre-compile the patterns and see if that hits different codegen paths. Using https://crates.io/crates/proc-macro-regex in its mode where the table size is restricted should produce wildly different input to Cranelift, because that's the only case where the state machine is expressed in control flow rather than table lookups.

An alternative way to pre-compile to control flow is to use [libfsm](https://github.com/katef/libfsm/blob/main/doc/tutorial/phases.md#code-generation) to generate C, and compile that into a small benchmark driver.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.