adoptium / adoptium/aqa-test-tools

Tabular View Doesn't Show Some Metrics

Open
#265 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Jupyter Notebook
Stars
33
Forks
97
Avg merge
7h 9m
Merged PRs (30d)
5

Description

By default (no benchmark filter), Tabular View should show all metrics. As shown in the screenshots below, it doesn't shows only one metric (i.e. footprint) for `pingperf-quarkus-startup` benchmark and misses the other one (i.e. startup time). When we use the benchmark filter to explicitly selected Quarkus benchmarks or when we use Perf Compare, then we can see both the metrics for pingperf as shown in the screenshots below.

We noticed this issue after #230 was merged and the server was updated with the latest changes.

Observation: Looking at the raw data, the baseline build might have been aborted in its first iteration, while the test build would have been aborted after finishing 3 iterations. This is because builds had to be aborted because of Jenkins overloading. Not sure whether that has anything to do with this issue. Each iteration results in 5 values.

### Without Benchmark Filter (Can see only one metric)
![image](https://user-images.githubusercontent.com/10422081/84185403-f3125700-aa5c-11ea-98a0-079c7a43c068.png)

### With Benchmark Filter (Can see both metrics)
![image](https://user-images.githubusercontent.com/10422081/84185330-d8d87900-aa5c-11ea-8133-938b8f0f87b7.png)

### Perf Compare (Can see both metrics)
![image](https://user-images.githubusercontent.com/10422081/84186298-5650b900-aa5e-11ea-8e82-b1becece9e30.png)

Contributor guide

Open the contributing guide

Research direction

Reproduce the default, benchmark-filtered, and Perf Compare Tabular View cases for the pingperf-quarkus-startup benchmark using the raw data described in the issue. Compare how metrics are selected when no filter is applied, including builds with different numbers of completed iterations; done means the unfiltered view shows both footprint and startup time.

Written by the indexing model from the issue text.

Assessment

Domain
data-visualization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.