dotnet / dotnet/SqlClient

Support net462 in the perf test pipelines

Open
#4,618 2 comments 1 reaction 0 assignees View on GitHub
:new: Triage Needed Area\Engineering Performance :chart_with_upwards_trend:
Dominant language
C#
Stars
989
Forks
340
Avg merge
4d 18h
Merged PRs (30d)
69

Description

### Summary

The perf test pipelines cannot run benchmarks against .NET Framework. Add `net462` as a selectable framework so we can measure the netfx code paths.

### Motivation

The driver still ships `net462` on Windows, but we have no way to perf-test it. This came up in review on #4543 ([thread](https://github.com/dotnet/SqlClient/pull/4543#discussion_r3877557271)): `ChannelDbConnectionPool` contains zero framework guards (`#if NETFRAMEWORK` / `#if NET`), and its saturated synchronous checkout path relies on cooperative thread-pool blocking that .NET Framework does not implement the same way.

That PR added a net462 unit test covering the path for *liveness*, which is the right guard against a hang or deadlock. But a liveness test cannot detect a throughput or latency regression, and the perf harness is currently structurally unable to measure it. So the netfx behavior of that path is unmeasured today.

This generalizes beyond the pool: any perf-sensitive change to a shared code path is currently validated only on .NET.

### Current state

`Microsoft.Data.SqlClient.PerformanceTests.csproj` targets:

```xml
net8.0;net9.0;net10.0
```

All three perf pipelines expose a `dotnetFramework` parameter with a hardcoded allowlist matching those TFMs:

- `eng/pipelines/perf/sqlclient-perf-pipeline.yml`
- `eng/pipelines/perf/sqlclient-perf-pr-pipeline.yml`
- `eng/pipelines/perf/sqlclient-perf-experiment.yml`

The value flows through to `--framework`, which the run scripts pass to `dotnet build -f` and `dotnet run -f`.

### Proposed work

1. **Perf project** — add `net462` to `TargetFrameworks`, conditioned on a Windows host, mirroring how the driver and unit test projects do it. The project already references the driver by both `ProjectReference` and `PackageReference` (for baseline-vs-current package mode); both legs need to resolve on net462.

2. **Pipelines** — add `net462` to the `dotnetFramework` allowlist in all three YAML files, and update the accompanying comments that currently enumerate `net8.0/net9.0/net10.0`.

3. **Platform validation** — `net462` is only valid with `platform=windows`. This is the same cross-parameter validation problem as `UseManagedSniOnWindows`: ADO cannot cross-validate two queue-time parameters in a pipeline that `extends` a template, so the check belongs in the run scripts. `run-perf-tests.sh` (the Linux entry point) should reject `net462` outright; `run-perf-tests.ps1` already has the host-detection pattern to copy.

4. **Runtime provisioning** — the scripts currently install shared .NET runtimes for the selected framework. `net462` needs no runtime install, but does need the targeting pack available on the build agent. Worth confirming the Windows perf agent image has it before wiring this up.

5. **BenchmarkDotNet** — confirm the netfx toolchain works under the harness, particularly the interleaved A/B runner (`interleave_perf.py`) and whether `BenchmarkDotNet.Diagnostics.Windows` needs different handling on this leg.

6. **Verify a real measurement** — once wired, run the connection pool benchmarks on `net462` and confirm the numbers are plausible rather than merely that the pipeline goes green.

### Notes

- Windows-only by nature, so this adds a leg rather than replacing anything. The existing `net8.0`/`net9.0`/`net10.0` legs are unaffected.
- Worth deciding whether `net462` should be part of the default PR perf run or opt-in at queue time. Opt-in seems right initially given the added agent time.
- Related: #4543 (where this was deferred).

Contributor guide

Open the contributing guide

Research direction

Start with Microsoft.Data.SqlClient.PerformanceTests.csproj and the three listed perf pipeline YAML files, then trace --framework through run-perf-tests.sh and run-perf-tests.ps1. Check the Windows targeting pack and BenchmarkDotNet handling, including interleave_perf.py. Done means net462 is available only on Windows, invalid Linux selections are rejected, and a connection-pool benchmark produces a plausible measurement.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, powershell, python, shell, yaml
Domain
ci-cd, devops, performance, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.