IntersectMBO / IntersectMBO/ouroboros-consensus

Guarantee usefulness of `db-analyser` ledger ops result.

Open
#223 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
67
Forks
43
Avg merge
5d 13h
Merged PRs (30d)
43

Description

We need to establish that the `benchmark-ledger-ops` analysis of `db-analyser` are useful in spotting potential performance regressions or optimizations. For that, we need its results to be reproducible and meaningful.

We need to run the `benchmark-ledger-ops` analysis with the same build, data, machine, and workload and corroborate if the runs have small enough variance among each other. Additionally, we need to make sure the input data is suitable to clearly reveal any measurable changes wrt. ledger ops.

Requires https://github.com/input-output-hk/ouroboros-consensus/issues/230

# Acceptance criteria

- A set of input data is identified that:
- Reveals known performance problems.
- Exhibits low variance among runs.

# Subtasks
- [ ] Validate that `benchmark-ledger-ops` exhibits the same mutator times as a running node. (eg Issue https://github.com/IntersectMBO/ouroboros-consensus/issues/1272)
- [ ] Ensure we use the same RTS flags as `cardano-node` only for benchmark-specific invocations (which can be done in tools such as `beacon` or invocation scripts).
- [ ] Calibrate and generate a useful set of input data (P&T)
- [ ] Feed this data to the variance analysis of P&T.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.