dotnet / dotnet/sdk

Investigate performance of .NET watch tests

Open
#50,228 2 comments 1 reaction 1 assignee Claimed by @tmat View on GitHub
untriaged
Dominant language
C#
Stars
3.2k
Forks
1.3k
PR merge metrics
PR metrics pending

Description

In experimenting with a dnceng MCP that has access to the PR performance data, I asked it to identify any tests we should investigate for performance improvements. It flagged the watch tests as a potential place to start at a median runtime of 20 minutes (next closest test had a median of 10 minutes).

Looks like a couple of the legs in particular are problematic. Note that this doesn't account for the branch as I'm not sure yet how to filter for that but note that svc queues should be the servicing branches and the other queues should be main so it looks like the slowness is in main. I'll keep poking and see if it has that data.

Corrected Analysis: dotnet-watch.Tests.dll.1 WorkItem Performance by Queue
SLOWEST WORKITEM EXECUTION TIMES:

ubuntu.2204.amd64.open 🔴

Median: 31.8 minutes (1,906 seconds)
75th percentile: 33.5 minutes
90th percentile: 37.3 minutes
95th percentile: 76.6 minutes
Max: 107.5 minutes
Queue wait: 12.9 min median, up to 89.7 min
osx.15.amd64.open 🟡

Median: 26.7 minutes (1,601 seconds)
75th percentile: 27.2 minutes
90th percentile: 28.2 minutes
Max: 75.4 minutes
Queue wait: Minimal (1-2 seconds median)
osx.13.arm64.open 🟡

Median: 23.4 minutes (1,402 seconds)
75th percentile: 23.9 minutes
90th percentile: 24.3 minutes
Max: 66.8 minutes
Queue wait: Minimal (0-1 seconds median)

FASTEST WORKITEM EXECUTION TIMES:

windows.amd64.vs2022.pre.open ✅

Median: 20.4 minutes (1,221 seconds)
75th percentile: 21.2 minutes
90th percentile: 22.3 minutes
Queue wait: 10.9 min median
windows.amd64.vs2022.pre.open.svc ✅

Median: 11.1 minutes (664 seconds)
75th percentile: 14.0 minutes
90th percentile: 16.8 minutes
Queue wait: 16.1 min median
osx.13.amd64.open ✅

Median: 9.7 minutes (580 seconds)
75th percentile: 11.4 minutes
90th percentile: 12.2 minutes
Queue wait: Variable (1 sec to 56.8 min)
ubuntu.2204.amd64.open.svc ✅

Median: 3.6 minutes (218 seconds)
75th percentile: 6.5 minutes
90th percentile: 10.0 minutes
Queue wait: 14.2 min median

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.