dotnet / dotnet/runtime

[ci-scan] Test failure: Android X509 DynamicChainTests name-constraint status mismatch

Open
#128,890 3 comments 1 reaction 2 assignees Claimed by @simonrozsival View on GitHub
area-System.Security blocking-clean-ci disabled-test Known Build Error os-android
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

> [!NOTE]
> This issue draft was prepared with GitHub Copilot assistance and should be reviewed before posting.

## Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966

Additional observed builds:

- https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442805
- https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442382

Build error leg or test failing: `android-arm Release AllSubsets_CoreCLR` - `System.Security.Cryptography.Tests` - `System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_PermittedTree_Dns`

## Error Details

Several X509 name-constraint tests fail consistently on Android Helix hardware. The failures look like provider/status behavior mismatches: UPN cases fail `chain.Build`, while DNS cases report `InvalidNameConstraints` where the test expects `PartialChain`.

Observed repro details from a representative official runtime-extra-platforms job:

- Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966
- Queue/job: `android-arm Release AllSubsets_CoreCLR`
- Helix work item: https://helix.dot.net/api/jobs/b1f9d1bd-1d1b-438b-ab80-34b1f16ab24e/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
- Work item: `System.Security.Cryptography.Tests`
- Machine: `DNCENGWIN-142`
- XHarness: `TESTS_FAILED`, instrumentation exit 1
- Test summary: total 11739, passed 10748, failed 5, skipped 986

Observed failure summaries:

```text
System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_PermittedTree_Upn
chain.Build

System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_PermittedTree_Dns
Assert.Equal() Failure: Values differ
Expected: PartialChain
Actual: InvalidNameConstraints

System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_ExcludedTree_Upn
chain.Build

System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_ExcludedTree_Dns
Assert.Equal() Failure: Values differ
Expected: PartialChain
Actual: InvalidNameConstraints
```

Other official runtime-extra-platforms hits found in Helix `testResults.xml`:

- Build [1444966 / 20260601.4](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966), `android-arm Release AllSubsets_Mono`, Helix: https://helix.dot.net/api/jobs/59f4de60-e3e0-4d92-bba4-4ffa93ab9445/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
- Build [1444966 / 20260601.4](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966), `android-arm Release AllSubsets_CoreCLR`, Helix: https://helix.dot.net/api/jobs/b1f9d1bd-1d1b-438b-ab80-34b1f16ab24e/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
- Build [1442805 / 20260531.2](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442805), `android-arm Release AllSubsets_CoreCLR`, Helix: https://helix.dot.net/api/jobs/6d2e01a6-4741-4cf6-857d-92a859bf8602/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
- Build [1442805 / 20260531.2](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442805), `android-arm64 Release AllSubsets_CoreCLR`, Helix: https://helix.dot.net/api/jobs/2957a5ca-18ce-4df1-9c23-2f332f8de6e7/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
- Build [1442382 / 20260530.1](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442382), `android-arm Release AllSubsets_CoreCLR`, Helix: https://helix.dot.net/api/jobs/a4492657-e917-4b7d-ae62-633f0eca9b9a/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
- Build [1442382 / 20260530.1](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442382), `android-arm64 Release AllSubsets_CoreCLR`, Helix: https://helix.dot.net/api/jobs/5cc7dc11-993e-4b51-a85a-24a5c7143ae6/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17

## Error Message

```json
{
"ErrorMessage": "",
"ErrorPattern": [
"\\[FAIL\\][^\\n]*System\\.Security\\.Cryptography\\.X509Certificates\\.Tests\\.DynamicChainTests\\.NameConstraintViolation_(PermittedTree|ExcludedTree)_(Dns|Upn)",
"Exception messages: (chain\\.Build|Assert\\.Equal\\(\\) Failure: Values differ)"
],
"BuildRetry": false,
"ExcludeConsoleLog": false
}
```

The ordered `ErrorPattern` array intentionally matches the DOTNET/logcat summary for all four related failures:

- `NameConstraintViolation_PermittedTree_Upn` with `chain.Build`
- `NameConstraintViolation_PermittedTree_Dns` with `InvalidNameConstraints`
- `NameConstraintViolation_ExcludedTree_Upn` with `chain.Build`
- `NameConstraintViolation_ExcludedTree_Dns` with `InvalidNameConstraints`

## Duplicate search

No open issue was found that directly tracks this Android-specific X509 name-constraint mismatch.

Related but not duplicates:

- dotnet/runtime#87413 - Linux UPN name constraints fail to build
- Closed.
- Related name-constraints area, but Linux/OpenSSL-specific and not this Android status mismatch.
- dotnet/runtime#41748 - macOS + Linux: Name constraint violation crashes chain building
- Closed.
- Related historical name-constraint behavior, not Android-specific.
- dotnet/runtime#36897 - System.Security.Cryptography.X509Certificates.Tests fail on iOS/tvOS
- Closed.
- Search hit for the same test names, but iOS/tvOS-specific and not Android.
- dotnet/runtime#83226 - release/7.0 Networking and Security tests fail on spanish servers
- Closed KBE.
- Search hit for a test name, but unrelated environment/localization issue.

## AzDO / Helix history search

Recent public `runtime-extra-platforms` builds (definition 154) on `main` were searched for this signature. The important detail is that the xUnit failure text is not present in the AzDO job console logs; it is present in the Helix `testResults.xml` artifacts linked from those jobs.

- Query window: recent builds from [1444966](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966) (`20260601.4`) back through [1406427](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1406427) (`20260502.2`).
- Scope: failed Android job console logs scanned, then `System.Security.Cryptography.Tests` Helix work items inspected via `testResults.xml` where available.
- Search strings: `NameConstraintViolation_PermittedTree_Upn`, `NameConstraintViolation_PermittedTree_Dns`, `NameConstraintViolation_ExcludedTree_Upn`, `NameConstraintViolation_ExcludedTree_Dns`, `InvalidNameConstraints`, and `PartialChain`.
- Result: 6 official runtime-extra-platforms Helix work items contained all four failures.

Architecture coverage:

- `android-arm`: 4 hits (`AllSubsets_CoreCLR` and one `AllSubsets_Mono`)
- `android-arm64`: 2 hits (`AllSubsets_CoreCLR`)
- `android-x86`: 0 hits in the inspected window
- `android-x64`: 0 hits in the inspected window

### Known issue validation
**Build: :mag_right:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966
**Error message validated:** `[\[FAIL\][^\n]*System\.Security\.Cryptography\.X509Certificates\.Tests\.DynamicChainTests\.NameConstraintViolation_(PermittedTree|ExcludedTree)_(Dns|Upn) Exception messages: (chain\.Build|Assert\.Equal\(\) Failure: Values differ)`]
**Result validation:** :white_check_mark: Known issue matched with the provided build.
**Validation performed at:** 6/2/2026 10:42:33 AM UTC

### Report

|Build|Repository|Test|Pull Request|
|---|---|---|---|
|[1444966](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966)|dotnet/runtime|[System.Security.Cryptography.Tests.WorkItemExecution](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966&view=ms.vss-test-web.build-test-results-tab&runId=40046066&resultId=197950)||
#### Summary
|24-Hour Hit Count|7-Day Hit Count|1-Month Count|
|---|---|---|
|1|1|1|

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.