Tracking issue for test failures in .NET Apple mobile apps on NativeAOT
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
## Description
Apple-mobile NativeAOT extra-platforms now runs the full library set (179 work items on `maccatalyst-arm64`). **165 pass, 14 fail.**
### Test data files not bundled
The tests load data with relative paths like `TestData/foo.bin`, which resolve against `/` on the Apple-mobile bundle, so the open turns into `/TestData/foo.bin` → `DirectoryNotFoundException`/`FileNotFoundException`. The shared `tests.android.targets` / `tests.ioslike.targets` only copy `ContentWithTargetPath` / `xunit-excludes.txt`; arbitrary `` content for these projects never ships into the .app.
| Test | First failing case | Path it tried |
|---|---|---|
| `Microsoft.Bcl.Cryptography.Tests` | `X509CertificateLoaderPkcs12Tests_FromFile.LoadPkcs7_PEM_Fails` | `/TestData/certchain.p7c` |
| `Microsoft.Extensions.Hosting.Unit.Tests` | `HostBuilderTests.CanConfigureAppConfigurationFromFile` | `/private/.../Contents/MacOS/appSettings.json` |
| `System.Data.DataSetExtensions.Tests` | `EnumerableRowCollectionTest.ThenBy` | `/Mono/testdataset1.xml` |
| `System.Diagnostics.FileVersionInfo.Tests` | `FileVersionInfoTest.FileVersionInfo_EmptyFVI` | `/Assembly1.cs` |
| `System.IO.Compression.Brotli.Tests` | `BrotliStreamUnitTests.GetMaxCompressedSize` | `/UncompressedTestFiles/TestDocument.pdf` |
| `System.IO.Compression.Tests` | `ZLibStreamUnitTests.FlushAsync_DuringFlushAsync` | `/ZLibTestData/TestDocument.pdf.z` |
| `System.IO.Packaging.Tests` | `Tests.T028_ModifyPackagePropertiesAndFlush` | `/plain.docx` |
| `System.Net.HttpListener.Tests` | suite also depends on bundled certs | — |
| `System.Net.NetworkInformation.Functional.Tests` | `DnsParsingTests.DnsAddressesParsing` | `/NetworkFiles/resolv_nonewline.conf` |
| `System.Resources.ResourceManager.Tests` | `ResourceManagerTests.File_GetResourceSet_NonStrings` | `MissingManifestResourceException` (file-based resource set not on disk) |
Fix direction: either teach the Apple test runner to `chdir` to `AppContext.BaseDirectory` before running tests, or ensure each affected csproj uses `ContentWithTargetPath` so the data ships into the .app and the relative paths resolve.
### EventSource tests require `EventSourceSupport=true`
Apple-mobile NativeAOT publishes with `EventSourceSupport` disabled (the static-lib + Apple-mobile combination requires `_SuppressNativeLibEventSourceWarning` to be set in `tests.ioslike.targets`). `TelemetryTest.EventSource_ExistsWithCorrectId` and `LoggingTest.EventSource_ExistsWithCorrectId` then fail because the expected event source isn't registered.
| Test | Failing cases |
|---|---|
| `System.Net.NameResolution.Functional.Tests` | `TelemetryTest.EventSource_ExistsWithCorrectId`, `LoggingTest.EventSource_ExistsWithCorrectId` |
| `System.Net.Sockets.Tests` | `TelemetryTest.EventSource_ExistsWithCorrectId`, `LoggingTest.EventSource_ExistsWithCorrectId` |
Fix direction: gate these tests with `[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsEventSourceSupported))]` or `[SkipOnPlatform(TestPlatforms.iOS|tvOS|MacCatalyst, "...")]` when `EventSourceSupport=false`.
### Apple-mobile behavioural differences
| Test | Failing case | Signal |
|---|---|---|
| `System.Net.HttpListener.Tests` | `HttpResponseStreamTests.Write_ContentToClosedConnectionAsynchronously_ThrowsHttpListenerException(ignoreWriteExceptions: False)` | expected `HttpListenerException`, no exception thrown — write-on-closed-socket semantics differ |
| `System.Net.Mail.Functional.Tests` | 1 failure (sandbox/network related) | needs deeper triage |
| `System.Security.Cryptography.Tests` | `X509CertificateLoaderPkcs12CollectionTests_FromByteSpan.LoadSignedFile_Fails` | expected `CryptographicException`, got different exception (SecKey* vs OpenSSL Pkcs12 differences) |
### Startup crash and hang, excluded in tests.proj
These two assemblies fail on every full `tvos-arm64` NativeAOT run and cannot be suppressed with per-test `[ActiveIssue]` because both fail before or outside test execution. They are excluded for Apple-mobile NativeAOT in `src/libraries/tests.proj`.
| Test | Failure mode |
|---|---|
| `System.Net.HttpListener.Tests` | After test discovery the app hangs and is killed at the 4500 second work-item timeout. XHarness reports `APP_CRASH`. |
| `System.Runtime.Caching.Tests` | The app aborts at startup with signal 10 (SIGBUS) before any managed output, so no results file is produced. |
Fix direction: root-cause the `HttpListener` hang and the `Runtime.Caching` startup crash on Apple-mobile NativeAOT, then re-enable each assembly.
Contributor guide
Assessment
This issue has not been assessed yet.