MSTest [DoNotParallelize] docs omit defer-to-end ordering and cancellation behaviour
- Dominant language
- No language data
- Stars
- 4.8k
- Forks
- 6.1k
- Avg merge
- 19h 10m
- Merged PRs (30d)
- 268
Description
## Description
The conceptual docs for MSTest's `[DoNotParallelize]` describe it only as preventing parallel/concurrent execution. In practice the attribute provides **two** guarantees, and the second (deferral) — plus a related cancellation interaction — is undocumented and surprising.
Affected page: [Test execution and control in MSTest → `DoNotParallelizeAttribute`](https://learn.microsoft.com/dotnet/core/testing/unit-testing-mstest-writing-tests-controlling-execution#donotparallelizeattribute) (source: `docs/core/testing/unit-testing-mstest-writing-tests-controlling-execution.md`).
The API reference page ([`DoNotParallelizeAttribute` Class](https://learn.microsoft.com/dotnet/api/microsoft.visualstudio.testtools.unittesting.donotparallelizeattribute)) is generated from the framework's XML doc, which has just been expanded in [microsoft/testfx#10245](https://github.com/microsoft/testfx/pull/10245); once that ships the API page will pick up the fuller text automatically, but the **conceptual** page above is authored here in `dotnet/docs` and needs a manual update.
## What the page currently says
> The `DoNotParallelizeAttribute` prevents parallel execution for specific assemblies, classes, or methods. Use this attribute when tests share state or resources that can't be safely accessed concurrently.
This covers mutual exclusion but omits the two points below.
## What is missing
**1. Deferral (runs last within its source).** When in-assembly parallelization is enabled, the scheduler partitions each test source's tests into a parallelizable set and a non-parallelizable set. It runs the **entire** parallelizable set to completion first, and only then runs the non-parallelizable set — one test at a time — at the very end of that source's run. So `[DoNotParallelize]` does not merely mean "runs serially/isolated"; it means "runs **after** every parallelizable test in the same source has finished."
Because partitioning is **per test source (assembly)**, "runs last" is relative to the other tests in that same source; a run spanning several sources has a separate deferred tail per source.
This has an actionable cost implication worth stating: a deferred test cannot overlap with anything, so its duration is *added to* the run rather than absorbed by parallel work happening alongside it. A single slow `[DoNotParallelize]` test is therefore disproportionately expensive — it lengthens the whole run by roughly its own duration, on top of never running in parallel.
**2. Cancellation interaction.** Because the deferred tests run only after the parallelizable phase completes, a run that is **canceled or aborted during the parallelizable phase can finish without executing any of them**. "Runs last" and "may not run at all when the run is canceled early" are both true, and the combination is surprising.
**3. (Minor) No-op when parallelization is off.** The existing Note ("You only need `DoNotParallelize` when you've enabled parallel execution") already implies this; it could optionally be made explicit that the attribute is entirely inert when parallelization is not enabled, since MSTest does not parallelize by default.
## Suggested change
Expand the `DoNotParallelizeAttribute` section to state the deferral and the cancellation behaviour, keeping the "per source / per assembly" scoping explicit so "runs last" is not misread in multi-assembly runs. The wording in [microsoft/testfx#10245](https://github.com/microsoft/testfx/pull/10245) (the rewritten framework XML doc) can be reused as a basis.
## Source of truth
Verified against the scheduler in `microsoft/testfx`: `src/Adapter/MSTestAdapter.PlatformServices/Execution/TestExecutionManager.Parallelization.cs` (partition of parallelizable vs. non-parallelizable sets, `Task.WhenAll` on the parallel set, then the deferred set) and `TestExecutionManager.Runner.cs` (per-test `ThrowIfCancellationRequested()`, which is why the deferred set is skipped on early cancellation). The original design RFC ([`docs/RFCs/004-In-Assembly-Parallel-Execution.md`](https://github.com/microsoft/testfx/blob/main/docs/RFCs/004-In-Assembly-Parallel-Execution.md), lines 66-68) also documents the deferral.
---
[Associated WorkItem - 632180](https://dev.azure.com/msft-skilling/Content/_workitems/edit/632180)
Contributor guide
Assessment
This issue has not been assessed yet.