Migrate on-device APK tests from NUnit to MSTest
- Dominant language
- C#
- Stars
- 2.1k
- Forks
- 579
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 257
Description
### Android framework version
net11.0-android (Preview)
### Affected platform version
.NET 11 / MSTest 4.4.0
### Description
MSTest 4.4.0 supports NativeAOT: https://www.nuget.org/packages/MSTest/4.4.0
Migrate all on-device APK tests from NUnit to MSTest. This scope includes:
- `tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj`, including tests loaded from `Java.Interop-Tests.NET.csproj`
- `tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests/Xamarin.Android.JcwGen-Tests.csproj`
- The shared Android instrumentation runner in `tests/TestRunner.Core/`
- Related CI configuration, test documentation, and package/version configuration
The current shared runner directly uses NUnit APIs for discovery, filtering, execution, result counting, and TRX generation. The migration must preserve:
- Android instrumentation startup and the `Microsoft.Android.Run` result protocol
- Multiple test assembly support
- Include/exclude category behavior from instrumentation arguments and `RuntimeHostConfigurationOption`
- Runtime-specific exclusions for Mono, CoreCLR, trimmable type maps, LLVM, and NativeAOT
- Fully-qualified test-name exclusions for external test assemblies
- Execution on the instrumentation/JNI-attached thread where required
- `dotnet test`, MTP, and TRX reporting behavior
The NativeAOT configuration currently contains NUnit-specific reflection/rooting workarounds (`IlcGenerateCompleteTypeMetadata` and `NativeAOT.rd.xml`, including roots for NUnit async adapters). Remove any workarounds that become unnecessary after the migration.
This issue does **not** include host-side NUnit test projects such as `MSBuildDeviceIntegration` or the Android SDK/build-task unit tests.
#### Acceptance criteria
- The two on-device APK test applications and `TestRunner.Core` no longer reference or use NUnit.
- Existing on-device tests are migrated to MSTest without losing coverage; any intentional semantic differences are documented.
- Existing include/exclude filters and runtime-specific skip behavior continue to work.
- The following CI lanes continue to build, install, run, and publish results:
- `Mono.Android.NET_Tests-Debug`
- `Mono.Android.NET_Tests-Release`
- `Mono.Android.NET_Tests-NoAab`
- `Mono.Android.NET_Tests-Mono`
- `Mono.Android.NET_Tests-CoreCLRTrimmable`
- `Mono.Android.NET_Tests-NativeAOT`
- `Xamarin.Android.JcwGen_Tests`
- Local documented `dotnet-local` build/test commands continue to work and produce TRX results.
- NUnit-specific NativeAOT metadata/rooting workarounds are removed when they are no longer required.
### Steps to Reproduce
1. Inspect `tests/TestRunner.Core/TestInstrumentation.cs`; it uses `NUnitTestAssemblyRunner` and NUnit filters/results directly.
2. Inspect the two APK test projects; both reference NUnit and use NUnit test attributes/assertions.
3. Build the NativeAOT lane with `-p:TestsFlavor=NativeAOT -p:PublishAot=true`; the project currently requires NUnit-specific runtime directives.
/cc @Evangelink
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with tests/TestRunner.Core/TestInstrumentation.cs, then inspect the two listed APK project files and the NativeAOT configuration and run the documented NativeAOT command. Compare the existing NUnit discovery, filtering, execution, and TRX paths with MSTest while preserving the listed instrumentation and CI lanes; done means both APK projects and TestRunner.Core no longer use NUnit and all acceptance checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, csharp
- Domain
- build-system, ci-cd, mobile-dev, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100