dotnet / dotnet/performance

[android] Add Test History visualizations for new Android scenarios

Open
#4,832 1 comment 1 reaction 2 assignees Claimed by @vitek-karas View on GitHub
enhancement Scenario
Dominant language
F#
Stars
773
Forks
301
Avg merge
6d 9h
Merged PRs (30d)
15

Description

With the recent additions to Android scenario testing (https://github.com/dotnet/performance/pull/4808, https://github.com/dotnet/performance/pull/4770, ...), we need to have convenient access to the measured data.

## Proposal
Add new section to [Test History Index](https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/TestHistoryIndexIndex.html) with **Mobile Scenarios** name. Note, we already have **Mobile Tests** section, but it is mostly non-functional at the moment and until we unify the flow for both iOS and Android scenarios it would be better to keep them separate to avoid confusion.

1. To the **Mobile Scenarios** section add the following links:
- dotnet/runtime Android Sample App
- .NET Android Default Template
- MAUI Android Default Template
- MAUI Blazor Android Default Template
- (... extend with more scenarios as we need ...)

2. The individual subpages should contain the following links:
- dotnet/runtime Android Sample App
- dotnet/runtime Android Sample App - Mono JIT
- dotnet/runtime Android sample app - Mono AOT
- dotnet/runtime Android sample app - CoreCLR JIT
- dotnet/runtime Android sample app - CoreCLR R2R
- .NET Android Default Template
- .NET Android Default Template - Mono ProfiledAOT
- .NET Android Default Template - CoreCLR JIT
- MAUI Android Default Template
- MAUI Android Default Template - Mono ProfiledAOT
- MAUI Android Default Template - CoreCLR JIT
- MAUI Blazor Android Default Template
- MAUI Blazor Android Default Template - Mono ProfiledAOT
- MAUI Blazor Android Default Template - CoreCLR JIT

3. The individual *scenario - configuration* links should lead to a page with measurement results (links)
- Startup
- Startup NoAnimation (when available)
- SOD - APK size
- SOD - Extracted size
- Memory Consumption (when available)

4. Each measurement result page should has the following UX
- View of the full history trend

Image

- Upon clicking on the history trend, it should be possible to select a time range, representing a commit history of the underlying repositories (currently, only dotnet/runtime and dotnet/performance are shown).

Image

Image

- After selecting the end point of the range, generated links to GitHub compare (e.g., [compare](https://github.com/dotnet/runtime/compare/a40f92ea4ed936117f7d490150b0195532a97493...109c9469a7f76bdc98300bd9c7bb4eaf014be971)) should be provided for ALL underlying repositories (currently, only link to dotnet/runtime is provided). The underlying repositories for the Android scenarios are:
- dotnet/runtime Android Sample App: dotnet/runtime and dotnet/performance
- .NET Android Default Template: dotnet/runtime, dotnet/android, and dotnet/performance
- MAUI Android Default Template: dotnet/runtime, dotnet/android, dotnet/maui, and dotnet/performance
- MAUI Blazor Android Default Template: dotnet/runtime, dotnet/android, dotnet/maui, and dotnet/performance

The commit and version history data is stored in `BuildAdditionalData` column

Image



### Things to consider
- Instead of step (3) where we list links to individual measurements results, to avoid extensive nesting, we could directly redirect to a "Full History Trend" page (4) where instead of having a single measurement, we could list all relevant measurements (startup, SOD, etc.). There seems to be a capability of doing this as we already show multiple measurements for auto-filling issues:
Image

- ~Currently, `Measurements` kusto table doesn't provide reliably datetime information for .NET Android, MAUI, and MAUI-Blazor measurements. To provide the information, cross-cluster join with `TimelineBuilds` table is required:~
```
| join kind=inner (
cluster().database().TimelineBuilds
| where Definition contains "dotnet-performance"
| where SourceBranch == "main"
| project BuildNumber, QueueTime
) on $left.BuildName == $right.BuildNumber
```
~This is not ideal, as it requires access to the `` + VPN.~
After https://github.com/dotnet/performance/pull/4838, the `BuildAdditionalInformation['SDK_INSTALL_DATETIME']` contains the datetime information.

### Additional work
- remove the old Android links from **Mobile Tests** section

### Future work
- when iOS measurements are migrated to the *new* flow, update their respective links as well

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.