oxidecomputer / oxidecomputer/opte
Benchmarks: want summary boxplots generated during CI
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 77
- Forks
- 11
- Avg merge
- 9d 20h
- Merged PRs (30d)
- 8
Description
Currently, criterion generates very nice plots of the distribution of runtimes for all of our benchmarks. This is the right level of detail for honing in on a specific benchmark, e.g., the distribution of times received from FBT probes on rx.
A glance view of processing times in and between families of packets would be useful. Each set of results processed by criterion outputs a list of box-whisker values (tukey.json) and other summary statistics (estimates.json), which makes this data easily available. We're already pulling in plotters as a library via criterion, which may be a useful starting point (as handy as Gnuplot is under other circumstances).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the CI benchmark jobs and the Criterion output consumed for each result set. Read the generated tukey.json and estimates.json data and inspect the existing plotters integration. Done means CI produces summary boxplots showing processing times within and between packet families.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ci-cd, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100