jonhoo / jonhoo/inferno

Differential output tooltips are confusing

Open
#293 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2.2k
Forks
160
PR merge metrics
No merged PRs in 30d

Description

Consider these two compressed line profiling reports.

Baseline:

```
base_only 50
both1 25
both2 25
```

Modified:

```
both1 25
both2 50
changed_only 125
```

The backwards looking difference flamegraph has explanatory tooltips as follows:

* `both1 (25 units, 12.50%; 0.00%)`
* `both2 (50 units, 25.00%; +12.50%)`
* `changed_only (125 units, 62.50%; +62.50%)`

The forwards looking difference flamegraph:

* `base_only (50 units, 50.00%; -50.00%)`
* `both1 (25 units, 25.00%; 0.00%)`
* `both2 (25 units, 25.00%; +25.00%)`

The main issue is the difference (+/-) percentages. They are really bad at explaining what happened.

Consider `both2`: it went from 25 to 50. In the backwards looking diff that's described as `+12.50%` even though it doubled. In the forwards looking diff that's described as `both2 (25 other counts, 25.00%; +25.00%)`.

1. This is inconsistent: `+%12.50` vs `+25.00%`.
2. It's talking about % of overall time, basically, but that's not what most people care about, they care about _absolute_ change. "This function got faster" or "this function uses more memory", not "this function is higher percentage of the whole". If everything in your program somehow ran twice as slow, it'd be `+0.00%` even though everything is twice as slow!
3. If % of total is _really_ what you care about that's why there's a normalization option when creating the diff.

Consider `changed_only`: it didn't exist in the base profile, only in the changed profile. `+62.50%` does _not_ convey that important piece of information, it just makes it seem like it grew 62% from the base.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the differential flamegraph tooltip generation and the code paths for backwards- and forwards-looking diffs in the repository. Reproduce the reports in the issue and compare how both percentage values are calculated. Done means the tooltips communicate the absolute change consistently, including functions present only in one profile, with tests covering these examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-visualization
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.