Test failure: DnsResolverTest.ResolveAddresses_NonExistent_ReturnsNxDomain returns ServerFailure on Windows Server 2025 CI
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
## Test
`System.Net.NameResolution.Tests.DnsResolverTest.ResolveAddresses_NonExistent_ReturnsNxDomain`
Added recently in #129845 (Implement DnsResolver for Windows).
## Symptom
Failing in CI **exclusively on the `Windows.Server2025.Amd64.Open` Helix queue** — 28 builds between 2026-07-09 and 2026-07-21 (outerloop). Not observed on Windows 10/11 or Server 2022.
```
Assert.Equal() Failure: Values differ
Expected: NxDomain
Actual: ServerFailure
```
Example console log:
https://helixr1107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-heads-main-eee8b308e14c4c03be/System.Net.NameResolution.Functional.Tests/3/console.d3a0b789.log?helixlogtype=result
## Test code
The test resolves a synthetic name in the reserved `.invalid` TLD (RFC 6761) and asserts the response code is exactly `NxDomain`:
```csharp
private const string NonExistentHost = "this-name-definitely-does-not-exist.dotnet-test.invalid";
...
DnsResult result = await ResolveAddresses(async, r, NonExistentHost);
Assert.Equal(DnsResponseCode.NxDomain, result.ResponseCode);
Assert.Empty(result.Records);
```
(`src/libraries/System.Net.NameResolution/tests/FunctionalTests/DnsResolverTest.cs:228`)
## Root-cause investigation
**This is not a product bug.** The Windows PAL faithfully maps whatever RCODE the OS resolver returns:
```csharp
// src/libraries/System.Net.NameResolution/src/System/Net/DnsResolverPal.Windows.cs:787
DNS_ERROR_RCODE_NAME_ERROR => DnsResponseCode.NxDomain,
DNS_ERROR_RCODE_SERVER_FAILURE => DnsResponseCode.ServerFailure,
...
_ => DnsResponseCode.ServerFailure, // catch-all
```
The query is issued with plain `DNS_QUERY_STANDARD` (no DNSSEC/EDNS flags requested by .NET), so the `ServerFailure` originates in the OS resolver / CI network, not in .NET.
To confirm, I provisioned a fresh **Windows Server 2025 Datacenter Azure Edition** VM (build 10.0.26100) and queried the same name via the identical Win32 `DnsQuery` path:
| Method | Result |
| --- | --- |
| `Resolve-DnsName` | NXDOMAIN |
| `nslookup` | NXDOMAIN |
| Win32 `DnsQuery` w/ `DNS_QUERY_STANDARD` (exact product path) | `status=9003` = `RCODE_NAME_ERROR` (**NXDOMAIN**) |
| Control (`microsoft.com`) | SUCCESS |
The clean VM uses Azure DNS `168.63.129.16`, which correctly returns NXDOMAIN for `.invalid`.
**Conclusion:** A clean Server 2025 VM returns NXDOMAIN through the same API. The CI SERVFAIL comes from the **Helix Server 2025 environment's DNS resolver** returning SERVFAIL for the reserved `.invalid` TLD (differs from Azure DNS), which is outside our control. Note the catch-all `_ => ServerFailure` mapping also means a local error/timeout on the CI resolver would surface as `ServerFailure`.
## Proposed fix
Relax the assertion to accept either negative response code (both mean "no such host"), matching the existing tolerant pattern used by `ResolvePtr_IPv6Address_DoesNotThrow` at `DnsResolverTest.cs:389`:
```csharp
Assert.True(result.ResponseCode is DnsResponseCode.NxDomain or DnsResponseCode.ServerFailure);
Assert.Empty(result.Records);
```
This preserves the test intent (non-existent name -> negative response, no records) while tolerating resolver-dependent behavior in CI.
## Known issue core information
Fill out the known issue JSON section by following the [step by step documentation on how to create a known issue](https://github.com/dotnet/arcade/blob/main/Documentation/Projects/Build%20Analysis/KnownIssues.md#how-to-create-a-known-issue).
```json
{
"ErrorMessage": "",
"ErrorPattern": "Expected: NxDomain[\\s\\S]*?Actual:\\s*ServerFailure",
"BuildRetry": false,
"ExcludeConsoleLog": false
}
```
### Additional information about the issue reported
Matches the xUnit assertion emitted by `DnsResolverTest.ResolveAddresses_NonExistent_ReturnsNxDomain` when the CI resolver returns SERVFAIL instead of NXDOMAIN for the reserved `.invalid` name (currently only on `Windows.Server2025.Amd64.Open`). `ErrorPattern` is a .NET regex; the `[\s\S]*?` tolerates the newline/whitespace between the Expected and Actual lines.
### Known issue validation
**Build: :mag_right:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=1519494
**Error message validated:** `[Expected: NxDomain[\s\S]*?Actual:\s*ServerFailure`]
**Result validation:** :white_check_mark: Known issue matched with the provided build.
**Validation performed at:** 7/22/2026 7:40:16 AM UTC
### Report
#### Summary
|24-Hour Hit Count|7-Day Hit Count|1-Month Count|
|---|---|---|
|0|0|0|
Contributor guide
Research direction
Start in src/libraries/System.Net.NameResolution/tests/FunctionalTests/DnsResolverTest.cs at ResolveAddresses_NonExistent_ReturnsNxDomain, then compare the tolerant pattern at line 389. Run this test on the affected Windows Server 2025 CI queue; done means the non-existent name produces no records while accepting either NxDomain or ServerFailure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- networking, testing-qa
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100