Improvement to Output of ResultsComparer for Perf Report Analysis
- Dominant language
- F#
- Stars
- 773
- Forks
- 301
- Avg merge
- 6d 9h
- Merged PRs (30d)
- 15
Description
@mrsharm and I were discussing ways in which we could speed up the perf report process. We think generating the following links at the top of each entry to the table could speed up the process of investigating a specific benchmark. This extra data could be generated with a flag, so that the final report can exclude it. This would only be for analysis purposes.
1. Link to a search of all `dotnet/runtime` issues with that specific test as a query. For example, https://github.com/dotnet/runtime/issues?q=is%3Aissue+System.Memory.Span%3CByte%3E.Fill%28Size%3A+512%29
2. Link to a search of all `dotnet/perf-autofiling-issues` issues with that specific test as a query. For example, https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aissue+System.Memory.Span%3CByte%3E.Fill%28Size%3A+512%29
3. Link(s) to the historical perf lab data for this test. This requirement is a little bit trickier, as providing an individual link for every config would add too much text to the perf report and make it unreadable. We brainstormed a solution in which only two or three common architectures are linked, to give the perf report writer a starting point. For example, we could provide links to Windows x64, Windows Arm64, and Ubuntu x64:
- https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows%2010.0.18362%2fSystem.Memory.Span(Byte).Fill(Size%3a%20512).html
- https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_arm64_Windows%2010.0.19041%2fSystem.Memory.Span(Byte).Fill(Size%3a%20512).html
- https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_ubuntu%2018.04%2fSystem.Memory.Span(Byte).Fill(Size%3a%20512).html
What would be ideal is if we could provide one link, *for a specific benchmark*, that has links to each configuration. Basically a reversal of how the data is currently indexed https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/TestHistoryIndexIndex.html. @DrewScoggins, do you know how difficult it would be to be able to query this data in reverse (`benchmark -> architecture` instead of `architecture -> benchmark`)?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.