IntersectMBO / IntersectMBO/ouroboros-consensus
Guarantee usefulness of `db-analyser` ledger ops result.
- Dominant language
- Haskell
- Stars
- 67
- Forks
- 43
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 43
Description
We need to establish that the `benchmark-ledger-ops` analysis of `db-analyser` are useful in spotting potential performance regressions or optimizations. For that, we need its results to be reproducible and meaningful.
We need to run the `benchmark-ledger-ops` analysis with the same build, data, machine, and workload and corroborate if the runs have small enough variance among each other. Additionally, we need to make sure the input data is suitable to clearly reveal any measurable changes wrt. ledger ops.
Requires https://github.com/input-output-hk/ouroboros-consensus/issues/230
# Acceptance criteria
- A set of input data is identified that:
- Reveals known performance problems.
- Exhibits low variance among runs.
# Subtasks
- [ ] Validate that `benchmark-ledger-ops` exhibits the same mutator times as a running node. (eg Issue https://github.com/IntersectMBO/ouroboros-consensus/issues/1272)
- [ ] Ensure we use the same RTS flags as `cardano-node` only for benchmark-specific invocations (which can be done in tools such as `beacon` or invocation scripts).
- [ ] Calibrate and generate a useful set of input data (P&T)
- [ ] Feed this data to the variance analysis of P&T.
Contributor guide
Assessment
This issue has not been assessed yet.