microsoft / microsoft/testfx

[perf-improver] Add benchmark coverage for ObjectModelConverters.FixUpTestCase

Open
#11,361 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

agentic-workflows area/performance type/automation
Dominant language
C#
Stars
1k
Forks
312
Avg merge
8h 30m
Merged PRs (30d)
469

Description

> [!TIP]
> **Your pull request is ready to create! 🎉 ✅**
>
> Everything is OK—the changes have been pushed to branch `perf-assist/objectmodelconverters-fixuptestcase-benchmark-1c1ff76b0a57909e`. Please review the changes, including any protected files, before creating the pull request.
>
> **[Create the pull request](https://github.com/microsoft/testfx/compare/main...perf-assist/objectmodelconverters-fixuptestcase-benchmark-1c1ff76b0a57909e?expand=1&title=%5Bperf-improver%5D%20Add%20benchmark%20coverage%20for%20ObjectModelConverters.FixUpTestCase)**
>
> The original pull request description is below.

---

## Goal and rationale

`ObjectModelConverters.FixUpTestCase` (VSTestBridge) runs once per VSTest `TestCase` during discovery/execution when a test framework goes through the VSTest bridge. A perf-improver fix on 2026-08-13 (PR merged) replaced a LINQ `Any(lambda)` with a manual loop on this hot path, but no dedicated benchmark existed to guard against future regressions or measure the impact of further optimizations here.

## Approach

- Added `InternalsVisibleTo` for `MSTest.Performance.Benchmarks` to `Microsoft.Testing.Extensions.VSTestBridge.csproj`, matching the existing IVT pattern already used for `TestFramework`, `MSTest.TestAdapter`, and `MSTestAdapter.PlatformServices`.
- Added a `ProjectReference` to VSTestBridge from the benchmarks project.
- Added `ObjectModelConvertersBenchmarks.cs` measuring `FixUpTestCase` on a freshly-constructed `TestCase` per iteration (via `[IterationSetup]`, since the method mutates the `TestCase`'s executor URI/properties).

## Performance evidence

Ran with `--job short` (`dotnet run -c Release --no-build -f net10.0 -- --filter "*ObjectModelConvertersBenchmarks*" --job short`):

| Method | Mean | Allocated |
|---|---|---|
| FixUpTestCase | 3.345 us | 112 B |

This establishes a baseline: the small residual allocation comes from `TestCase.SetPropertyValue`'s dictionary-backed property store, useful reference for any future optimization attempts in this area.

## Trade-offs

None — this is additive test/benchmark infrastructure only, no production code changes.

## Reproducibility

```sh
./build.sh -c Release
cd test/Performance/MSTest.Performance.Benchmarks
dotnet run -c Release --no-build -f net10.0 -- --filter "*ObjectModelConvertersBenchmarks*" --job short
```

## Test Status

- `./build.sh -c Release`: succeeded, 0 warnings/0 errors.
- `Microsoft.Testing.Extensions.VSTestBridge.UnitTests` filtered to `ObjectModelConvertersTests` (`--filter "FullyQualifiedName~ObjectModelConvertersTests"`): 25/25 passed.

---

> [!NOTE]
> GitHub Actions is not permitted to create or approve pull requests in this repository.

> The changes have been pushed to branch `perf-assist/objectmodelconverters-fixuptestcase-benchmark-1c1ff76b0a57909e` and are ready to review.

To fix the permissions issue, go to **Settings** → **Actions** → **General** and enable **Allow GitHub Actions to create and approve pull requests**. See also: [gh-aw FAQ](https://github.github.com/gh-aw/reference/faq/#why-is-my-create-pull-request-workflow-failing-with-github-actions-is-not-permitted-to-create-or-approve-pull-requests)

Show patch preview (34 of 93 lines)

```diff
From 0064f2377b289bd00b3272ab40675bfcf86d61df Mon Sep 17 00:00:00 2001
X-GH-AW-Base-Commit: 62e84e66d8d54cffa902337b909b27208e2ae7c8
From: "github-actions[bot]"
Date: Thu, 17 Sep 2026 13:47:20 +0000
Subject: [PATCH] Add benchmark coverage for
ObjectModelConverters.FixUpTestCase

FixUpTestCase runs once per VSTest TestCase during discovery/execution
through the VSTest bridge (a fix landed 2026-08-13 replaced a LINQ
Any(lambda) with a manual loop, but no dedicated benchmark existed).
Adds ObjectModelConvertersBenchmarks to MSTest.Performance.Benchmarks
and wires up the required InternalsVisibleTo/ProjectReference.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
...oft.Testing.Extensions.VSTestBridge.csproj | 1 +
.../MSTest.Performance.Benchmarks.csproj | 1 +
.../ObjectModelConvertersBenchmarks.cs | 38 +++++++++++++++++++
3 files changed, 40 insertions(+)
create mode 100644 test/Performance/MSTest.Performance.Benchmarks/ObjectModelConvertersBenchmarks.cs

diff --git a/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/Microsoft.Testing.Extensions.VSTestBridge.csproj b/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/Microsoft.Testing.Extensions.VSTestBridge.csproj
index 2cb7bf5..5239f88 100644
--- a/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/Microsoft.Testing.Extensions.VSTestBridge.csproj
+++ b/src/Platform/Microsoft.Testing.Extensions.VSTestBridge/Microsoft.Testing.Extensions.VSTestBridge.csproj
@@ -22,6 +22,7 @@



+



diff --git a/test/Performance/MSTest.Performance.Benchmarks/MSTest.Performance.Benchmarks.csproj b/test/Performance/MSTest.Performance.B
... (truncated)
```

> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `southcentralus0.in.applicationinsights.azure.com`
>
> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "southcentralus0.in.applicationinsights.azure.com"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>

> 🤖 **Automated content by GitHub Copilot.** Generated by the [Perf Improver](https://github.com/microsoft/testfx/actions/runs/35227706065/agentic_workflow) workflow. · copilot · auto · 261.3 AIC · ⌖ 11.8 AIC · ⊞ 16.5K · [◷]( · [◷](https://github.com/search?q=repo%3Amicrosoft%2Ftestfx+%22gh-aw-workflow-id%3A+perf-improver%22&type=pullrequests))
> Comment /perf-assist to run again
>

Add this agentic workflow to your repo

To install this agentic workflow, run

```
gh aw add githubnext/agentics/workflows/perf-improver.md@main
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ObjectModelConverters.FixUpTestCase in the VSTestBridge source and review existing benchmarks under test/Performance/MSTest.Performance.Benchmarks. Run the documented filtered dotnet benchmark command after wiring the project reference and access settings; done means the benchmark builds, runs, and reports FixUpTestCase coverage without breaking the existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
performance, testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.