paritytech / paritytech/revive

Add JIT benchmarks

Open
#321 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement good first issue
Dominant language
Rust
Stars
102
Forks
26
Avg merge
1d 13h
Merged PRs (30d)
6

Description

Setting the REVIVE_USE_COMPILER env var will enable the JIT in PVM in the pallet. However just setting this variable and running the benchmarks does not tell anything because it measures the whole execution from the get go.

Add to that, because JITed contracts run orders of magnitudes faster, the benchmarking parameters used for benchmark the {EVM,PVM} interpreters are too low to be telling.

What should be done here:

  • Add a bench-pvm-jit feature flag to the benchmarks crate which will lead to benchmark the JIT instead the interpreter.
  • Implement functions wall-time measuring the compilation and execution (execution means the sandbox is already up - zero startup or compilation overhead, just the execution time).
  • Find new parameters suitable for measuring the JIT. They are selected if the bench-pvm-jit feature is set. Ideally they are big enough to be telling but still small enough so the EVM interpreter can execute them too (will be very slow but we need the numbers to compare)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the benchmarks crate and trace how REVIVE_USE_COMPILER selects the JIT and how the existing EVM/PVM benchmark parameters are defined. Add the bench-pvm-jit path with separate compilation and execution wall-time measurements, then validate that its parameters provide useful JIT results while remaining executable by the EVM interpreter.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers, performance, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.