dotnet / dotnet/runtime

Tracking issue for test failures in .NET Apple mobile apps on NativeAOT

Open
#126,697 2 comments 0 reactions 0 assignees View on GitHub
area-NativeAOT-coreclr os-ios
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

## Description

Apple-mobile NativeAOT extra-platforms now runs the full library set (179 work items on `maccatalyst-arm64`). **165 pass, 14 fail.**

### Test data files not bundled

The tests load data with relative paths like `TestData/foo.bin`, which resolve against `/` on the Apple-mobile bundle, so the open turns into `/TestData/foo.bin` → `DirectoryNotFoundException`/`FileNotFoundException`. The shared `tests.android.targets` / `tests.ioslike.targets` only copy `ContentWithTargetPath` / `xunit-excludes.txt`; arbitrary `` content for these projects never ships into the .app.

| Test | First failing case | Path it tried |
|---|---|---|
| `Microsoft.Bcl.Cryptography.Tests` | `X509CertificateLoaderPkcs12Tests_FromFile.LoadPkcs7_PEM_Fails` | `/TestData/certchain.p7c` |
| `Microsoft.Extensions.Hosting.Unit.Tests` | `HostBuilderTests.CanConfigureAppConfigurationFromFile` | `/private/.../Contents/MacOS/appSettings.json` |
| `System.Data.DataSetExtensions.Tests` | `EnumerableRowCollectionTest.ThenBy` | `/Mono/testdataset1.xml` |
| `System.Diagnostics.FileVersionInfo.Tests` | `FileVersionInfoTest.FileVersionInfo_EmptyFVI` | `/Assembly1.cs` |
| `System.IO.Compression.Brotli.Tests` | `BrotliStreamUnitTests.GetMaxCompressedSize` | `/UncompressedTestFiles/TestDocument.pdf` |
| `System.IO.Compression.Tests` | `ZLibStreamUnitTests.FlushAsync_DuringFlushAsync` | `/ZLibTestData/TestDocument.pdf.z` |
| `System.IO.Packaging.Tests` | `Tests.T028_ModifyPackagePropertiesAndFlush` | `/plain.docx` |
| `System.Net.HttpListener.Tests` | suite also depends on bundled certs | — |
| `System.Net.NetworkInformation.Functional.Tests` | `DnsParsingTests.DnsAddressesParsing` | `/NetworkFiles/resolv_nonewline.conf` |
| `System.Resources.ResourceManager.Tests` | `ResourceManagerTests.File_GetResourceSet_NonStrings` | `MissingManifestResourceException` (file-based resource set not on disk) |

Fix direction: either teach the Apple test runner to `chdir` to `AppContext.BaseDirectory` before running tests, or ensure each affected csproj uses `ContentWithTargetPath` so the data ships into the .app and the relative paths resolve.

### EventSource tests require `EventSourceSupport=true`

Apple-mobile NativeAOT publishes with `EventSourceSupport` disabled (the static-lib + Apple-mobile combination requires `_SuppressNativeLibEventSourceWarning` to be set in `tests.ioslike.targets`). `TelemetryTest.EventSource_ExistsWithCorrectId` and `LoggingTest.EventSource_ExistsWithCorrectId` then fail because the expected event source isn't registered.

| Test | Failing cases |
|---|---|
| `System.Net.NameResolution.Functional.Tests` | `TelemetryTest.EventSource_ExistsWithCorrectId`, `LoggingTest.EventSource_ExistsWithCorrectId` |
| `System.Net.Sockets.Tests` | `TelemetryTest.EventSource_ExistsWithCorrectId`, `LoggingTest.EventSource_ExistsWithCorrectId` |

Fix direction: gate these tests with `[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsEventSourceSupported))]` or `[SkipOnPlatform(TestPlatforms.iOS|tvOS|MacCatalyst, "...")]` when `EventSourceSupport=false`.

### Apple-mobile behavioural differences

| Test | Failing case | Signal |
|---|---|---|
| `System.Net.HttpListener.Tests` | `HttpResponseStreamTests.Write_ContentToClosedConnectionAsynchronously_ThrowsHttpListenerException(ignoreWriteExceptions: False)` | expected `HttpListenerException`, no exception thrown — write-on-closed-socket semantics differ |
| `System.Net.Mail.Functional.Tests` | 1 failure (sandbox/network related) | needs deeper triage |
| `System.Security.Cryptography.Tests` | `X509CertificateLoaderPkcs12CollectionTests_FromByteSpan.LoadSignedFile_Fails` | expected `CryptographicException`, got different exception (SecKey* vs OpenSSL Pkcs12 differences) |

### Startup crash and hang, excluded in tests.proj

These two assemblies fail on every full `tvos-arm64` NativeAOT run and cannot be suppressed with per-test `[ActiveIssue]` because both fail before or outside test execution. They are excluded for Apple-mobile NativeAOT in `src/libraries/tests.proj`.

| Test | Failure mode |
|---|---|
| `System.Net.HttpListener.Tests` | After test discovery the app hangs and is killed at the 4500 second work-item timeout. XHarness reports `APP_CRASH`. |
| `System.Runtime.Caching.Tests` | The app aborts at startup with signal 10 (SIGBUS) before any managed output, so no results file is produced. |

Fix direction: root-cause the `HttpListener` hang and the `Runtime.Caching` startup crash on Apple-mobile NativeAOT, then re-enable each assembly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.