dotnet / dotnet/performance

Improvement to Output of ResultsComparer for Perf Report Analysis

Open
#2,476 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
F#
Stars
773
Forks
301
Avg merge
6d 9h
Merged PRs (30d)
15

Description

@mrsharm and I were discussing ways in which we could speed up the perf report process. We think generating the following links at the top of each entry to the table could speed up the process of investigating a specific benchmark. This extra data could be generated with a flag, so that the final report can exclude it. This would only be for analysis purposes.

1. Link to a search of all `dotnet/runtime` issues with that specific test as a query. For example, https://github.com/dotnet/runtime/issues?q=is%3Aissue+System.Memory.Span%3CByte%3E.Fill%28Size%3A+512%29
2. Link to a search of all `dotnet/perf-autofiling-issues` issues with that specific test as a query. For example, https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aissue+System.Memory.Span%3CByte%3E.Fill%28Size%3A+512%29
3. Link(s) to the historical perf lab data for this test. This requirement is a little bit trickier, as providing an individual link for every config would add too much text to the perf report and make it unreadable. We brainstormed a solution in which only two or three common architectures are linked, to give the perf report writer a starting point. For example, we could provide links to Windows x64, Windows Arm64, and Ubuntu x64:
- https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows%2010.0.18362%2fSystem.Memory.Span(Byte).Fill(Size%3a%20512).html
- https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_arm64_Windows%2010.0.19041%2fSystem.Memory.Span(Byte).Fill(Size%3a%20512).html
- https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_ubuntu%2018.04%2fSystem.Memory.Span(Byte).Fill(Size%3a%20512).html

What would be ideal is if we could provide one link, *for a specific benchmark*, that has links to each configuration. Basically a reversal of how the data is currently indexed https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/TestHistoryIndexIndex.html. @DrewScoggins, do you know how difficult it would be to be able to query this data in reverse (`benchmark -> architecture` instead of `architecture -> benchmark`)?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.