paritytech / paritytech/revive
Add JIT benchmarks
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 102
- Forks
- 26
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 6
Description
Setting the REVIVE_USE_COMPILER env var will enable the JIT in PVM in the pallet. However just setting this variable and running the benchmarks does not tell anything because it measures the whole execution from the get go.
Add to that, because JITed contracts run orders of magnitudes faster, the benchmarking parameters used for benchmark the {EVM,PVM} interpreters are too low to be telling.
What should be done here:
- Add a
bench-pvm-jitfeature flag to the benchmarks crate which will lead to benchmark the JIT instead the interpreter. - Implement functions wall-time measuring the compilation and execution (execution means the sandbox is already up - zero startup or compilation overhead, just the execution time).
- Find new parameters suitable for measuring the JIT. They are selected if the
bench-pvm-jitfeature is set. Ideally they are big enough to be telling but still small enough so the EVM interpreter can execute them too (will be very slow but we need the numbers to compare)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the benchmarks crate and trace how REVIVE_USE_COMPILER selects the JIT and how the existing EVM/PVM benchmark parameters are defined. Add the bench-pvm-jit path with separate compilation and execution wall-time measurements, then validate that its parameters provide useful JIT results while remaining executable by the EVM interpreter.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100