IntersectMBO / IntersectMBO/plutus

Validation benchmark times vary with order/number of exports in `PlutusBenchmark.Common

Open
#5,906 1 comment 0 reactions 0 assignees View on GitHub
Benchmarks Internal status: triaged
Dominant language
Haskell
Stars
1.6k
Forks
508
Avg merge
3d 10h
Merged PRs (30d)
22

Description

PR #5905 (and previously #5903, #5892, and #5886) modified the benchmarking code to use a production-like evaluator for the bls12-381 and Marlowe benchmarks. Unexpectedly this appeared to increase the time of the validation benchmarks by about 2%. Experimentation suggests that the validation times are sensitive to the number and order of the exports in [PlutusBenchmark.Common](https://github.com/IntersectMBO/plutus/blob/master/plutus-benchmark/common/PlutusBenchmark/Common.hs). The functions `evaluateCekForBench` and `evaluateCekLikeInProd` were exported from this file but are unused elsewhere; when the first of these was removed the validation benchmarks pretty much went back to where they were (and this is the version that was merged), but if the second is also removed the second then the 2% slowdown reappears. There were [other examples](https://github.com/IntersectMBO/plutus/pull/5903#issuecomment-2053947428) where permuting the exports more or less randomly gave an overall slowdown of 4.6%.

It's not clear what's causing this: one possibility is changes in inlining.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.