buglets with documentation of units in results.json
- Dominant language
- C#
- Stars
- 1.1k
- Forks
- 111
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 3
Description
The results.json file contains, after a fashion, documentation on all of the measured quantities. Phew!
The documentation is in places inconsistent or incomplete.
Printing of benchmarks/cpu/raw prints far more digits of precision than the underlying system is capable of measuring.
Some times are reported in ms, and other times in us, and rates in per-second. I'd pick either ms or us, and stick with per-second.
Pedantic: if you report in us, perhaps use UTF8 mu symbol rather than Latin u.
Many, but not all, longDescriptions have units. All should have units, replicating what's already in shortDescription. See benchmarks/build-time, benchmarks/start-time, benchmarks/published-size where the units are omitted.
Some shortDescriptions use '(ms)' but the longDescription says 'in ms'. Use the style '(...)' uniformly.
Are sizes (such as benchmarks/swap) reported in SI Mega (1e6) or Mibi (1024*1024) units. This may be tricky.
The "format" is always given as "n0". I had to look n0 format up. That's irrelevant, since this is json, and json doesn't support using ',' to group digits. I don't see the point in reporting "format".
What happens if the number being reported is mis-computed upstream as Inf or NaN. What happens when it is printed in the json file? (JSON doesn't support Inf or NaN...)
Contributor guide
Assessment
This issue has not been assessed yet.