apache / apache/lucene

Generalize jmh-table.py into a reusable visualization for any JMH benchmark

Open
#15,968 1 comment 1 reaction 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Spinoff from #15950 as @mikemccand suggested. The `jmh-table.py` visualization introduced in this PR for ScoreDoc sort benchmark is really cool, it's already a near-general solution.

One enhancement is: the script currently hardcodes `size` and `distribution` which ties specifically to `ScoreDocSortBenchmark`. It would be better to generalize this into a reusable tool that works with any of lucene's JMH benchmarks by auto-discovering `@Param` names.

As a motivating example, @gsmiller's PartitionByLeafBenchmark in #15938 has different params and he had to manually feed results to AI to build a better visual summary table. A generalized `jmh-table.py` as a *Skill*-ish tool would have worked out of the box.

Contributor guide

Open the contributing guide

Research direction

Start with the existing jmh-table.py visualization introduced for the ScoreDoc sort benchmark, then compare its hardcoded size and distribution handling with the parameters in PartitionByLeafBenchmark. Generalize parameter handling by discovering JMH @Param names so the tool works across Lucene JMH benchmarks, and verify that it produces useful summary tables for benchmarks with different parameters.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
data-visualization, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.