Generic logic to get get and store benchmark metric
- Dominant language
- HTML
- Stars
- 155
- Forks
- 346
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 36
Description
Extend based on https://github.com/adoptium/aqa-tests/pull/6339/, make checkDacapoH2Metric() and archiveDacapoH2Metric() generic.
On second thought, it may be better to have a separate script (i.e., Python - `benchmarkMetric.py`) to handle this.
- In `benchmarkMetric.py`, it contains a generic function `checkMetric()`. `checkMetric()` takes job console output and reads `benchmarkMetric.json` and runs each regex, and stores into one JSON output.
`benchmarkMetric.json` example:
```
[
{
"Benchmark": "dacapo-eclipse",
"metrics":
{
"dacapo-eclipse" : [
{"regex": "DaCapo.*eclipse\\sPASSED\\sin\\s(\\d*\\.?\\d*)\\smsec"},
{"higherbetter": true},
{"units": "msec"}
]
}
},
{
"Benchmark": "dacapo-h2",
"metrics":
{
"dacapo-h2" : [
{"regex": "DaCapo.*h2\\sPASSED\\sin\\s(\\d*\\.?\\d*)\\smsec"},
{"higherbetter": true},
{"units": "msec"}
]
}
}
]
```
Note: Backslashes must be double-escaped (`\\`) in JSON strings.
Output: JSON file as specified in https://github.com/adoptium/aqa-tests/issues/6327. For example:
```
[
{
"Benchmark": "dacapo-eclipse",
"metrics":
{
"dacapo-eclipse" : [
{"value": "123456"},
{"higherbetter": true},
{"units": "msec"}
]
}
},
{
"Benchmark": "dacapo-h2",
"metrics":
{
"dacapo-h2" : [
{"value": "4689"},
{"higherbetter": true},
{"units": "msec"}
]
}
}
]
```
related: https://github.com/adoptium/aqa-tests/issues/6327
top level issue: https://github.com/adoptium/aqa-tests/issues/6268
@smlambert @LongyuZhang if I missed anything, please feel free to update.
Contributor guide
Assessment
This issue has not been assessed yet.