IntersectMBO / IntersectMBO/plutus

Improve output of benchmark tests

Open
#6,141 1 comment 0 reactions 0 assignees View on GitHub
Internal status: triaged
Dominant language
Haskell
Stars
1.6k
Forks
508
Avg merge
3d 10h
Merged PRs (30d)
22

Description

At least some of the tests in plutus-benchmark compare golden files contain the “abstract” cpu and mem usage. If these results change, then the developer must carefully compare the results to see if the abstract resource usage has increased. In reality, we should only consider a test as “failed” if the resource usage increases.

Comment from @zliu41 :
I don’t think we can make a test succeed if the cost decreases. Otherwise, in PR1 the cost may go down 10% and it goes unnoticed. Then in PR2 the cost may go up 10% and it goes unnoticed again, because the cost is no bigger than the original value.

But there are ways we can make it easier to tell which tests have increased or decreased costs:

- Use different colors for tests whose cost go up vs. go down

- Show percentage difference

- NumericUnderscores

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.