[CI] SOS.UnitTests desktop.cli tests fail with 'Failed to load data access module' (mscordacwks.dll mismatch)
- Dominant language
- C++
- Stars
- 1.3k
- Forks
- 404
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 35
Description
## Summary
15 SOS.UnitTests with `config: desktop.cli` (and 2 `DualRuntimes` tests) are consistently failing on all Windows rolling CI legs (`diagnostics-public-ci`, definition 25) since approximately Mar 10, 2026. The pipeline is currently red — the last 2 consecutive rolling builds both failed.
## Root Cause
All failing tests hit the same error from CDB/SOS:
```
Failed to load data access module, 0x80004002
Verify that 1) you have a recent build of the debugger (10.0.18317.1001 or newer)
2) the file mscordacwks.dll that matches your version of clr.dll is
in the version directory or on the symbol path
```
The SOS commands (`!PrintException`, `!ClrStack`, `!DumpObj`, etc.) produce no diagnostic output because the DAC cannot be loaded for the .NET Framework CLR in the debuggee process. The test regex verification then fails because there is no output to match.
This occurs 17 times across the test suite in a single run. The failure is **deterministic** — identical tests fail on both x64 and x86 with the same error.
## Likely Cause
A .NET Framework servicing update on the CI agent machines introduced a new `clr.dll` version, but the matching `mscordacwks.dll` is not available on the configured symbol path. See closed issue #4625 for a prior occurrence of missing mscordacwks symbols.
## Affected Builds
| Build | Date | Failed Legs |
|-------|------|-------------|
| [1332940](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1332940) | Mar 12 | Win x64 Debug, Win x64 Release, Win x86 Release |
| [1332642](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1332642) | Mar 12 | Win x64 Debug, Win x64 Release, Win x86 Release |
| [1327950](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1327950) | Mar 10 | Win x86 Release |
Last passing rolling build: [1330114](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1330114) (Mar 11).
## Failing Tests (15 on x64, 14 on x86)
All `desktop.cli` config:
- SOS.TaskNestedException, SOS.LineNums, SOS.DivZero, SOS.SimpleThrow, SOS.Reflection
- SOS.StackTraceFaultingExceptionFrame, SOS.GCTests, SOS.OtherCommands
- SOS.NestedExceptionTest, SOS.Overflow, SOS.DumpGen, SOS.ConcurrentDictionaries, SOS.AsyncMain
Plus `projectk.sdk.prebuilt` config:
- SOS.DualRuntimes (9.0.12), SOS.DualRuntimes (8.0.23)
## Reproduction
Artifacts with full test logs are available at:
- `Logs_Windows_x64_Release_1` → `log/Release/SOS.UnitTests_net8.0_x64.log`
- `Logs_Windows_x86_Release_1` → `log/Release/SOS.UnitTests_net8.0_x86.log`
## Suggested Fix
1. Verify which .NET Framework version is currently on the CI agents
2. Ensure the symbol server path is configured so CDB can download the matching `mscordacwks.dll`, or bundle the correct DAC version
3. Alternatively, investigate whether the `cdb-sos` NuGet package (currently `10.0.26100.1`) needs updating to match the new agent environment
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.