[Custom Issue]: Scaling scores - comparing two sets of Vernier output.
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 6
- Forks
- 11
- Avg merge
- 18h 7m
- Merged PRs (30d)
- 1
Description
### Contact Details
_No response_
### Are there any linked Issues or Pull Requests?
Similar in nature to #246, but ranking regions according to a different parameter.
### Brief description
It'd be useful to support a command-line tool which takes two sets of Vernier output and orders profiled regions according to how well they are scaling. The two runs would use different resources, such as 1 versus 2 MPI ranks in the sample below.
e.g. `vernier-scaling /path/to/runA /path/to/runB`.
```
Baseline file: baseline-file.out, procs: 0001, threads: 01
Test file: test-file.out, procs: 0002, threads: 01
Rank Routine name Time difference (s)
------- ------------------------------------------------------- -------------------
1 anti_scaling_routine -0.500
2 no_scaling_routine 0.000
3 sub_linear_scaling_routine 0.500
4 perfect_scaling_routine 1.000
5 super_linear_scaling_routine 1.500
```
The "scaling score" is calculated as:
```math
S =
\left( \frac{p_1 t_1 T_1}{p_2 t_2 T_2} - 1 \right) /
\left( \frac{p_1 t_1}{p_2 t_2} - 1 \right)
+ 1
```
where $p$ is the number of MPI tasks, $t$ is the number of threads per MPI task and $T$ is the elapsed time. The index 1 refers to the reference
(baseline) time and resources; index 2 is the test time and resources.
| Score value | Meaning |
|--|--|
| $S < 0$ | Region is anti-scaling. |
| $S = 0$ | Region shows no scaling. |
| $0 < S < 1$ | Region shows sub-linear scaling. |
| $S = 1$ | Region shows ideal scaling. |
| $S > 1$ | Region shows super-linear scaling. |
There would need to be a **cut-off** mechanism in terms of absolute time taken: a region that takes $2 \mu s$, up from $1 \mu s$ would be anti-scaling but might clutter the output with information that isn't useful.
### Further details of the issue.
_No response_
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The payload names no files, tests, or CLI entry point; start by reviewing the implementation context of similar issue #246 and how Vernier output records resources and elapsed times. Done means a command-line comparison of two runs that ranks regions by the specified scaling score and applies an agreed absolute-time cutoff.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- cli, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100