mono-aot-cross crashes AOT-compiling OpenCvSharp.dll (sgen-alloc.c assertion) - worked around in #9

Summary

dotnet publish -c Release -p:RunAOTCompilation=true crashes the AOT compiler while precompiling OpenCvSharp.dll:

error : Precompiling failed for .../OpenCvSharp.dll with exit code 134
error : Mono Ahead of Time compiler - compiling assembly .../OpenCvSharp.dll
error : * Assertion at .../src/mono/mono/sgen/sgen-alloc.c:409, condition `*p == NULL' not met

Reproduced identically on two independently-versioned wasm-tools workload packs:

• net10.0, Microsoft.NET.Runtime.WebAssembly.Sdk 10.0.9 (mono-aot-cross from Microsoft.NETCore.App.Runtime.AOT.*.Cross.browser-wasm 10.0.9)
• net8.0, Microsoft.NET.Runtime.WebAssembly.Sdk 8.0.28 (same pack family, 8.0.28)

Both on Windows and Linux (GitHub Actions ubuntu-latest) runners.

Reproduction

1. Reference a native .a via <NativeFileReference> in a Blazor WASM project (OpenCvSharp5.runtime.wasm in this repo's BlazorApp.csproj)
2. <RunAOTCompilation>true</RunAOTCompilation>
3. dotnet publish -c Release

A plain dotnet new blazorwasm template (no native code, no AOT) is unaffected.

Isolated the actual trigger: OpenCvSharp.dll's P/Invoke surface, not the native .a size

Tested whether it's the native .a size or the managed P/Invoke declaration count that crashes mono-aot-cross. Built a minimal Blazor WASM app that:

• References OpenCvSharp5.runtime.wasm's real, large libOpenCvSharpExtern.a via NativeFileReference (same rename workaround as this repo)
• Declares only 3 made-up [LibraryImport("OpenCvSharpExtern")] methods instead of referencing the real OpenCvSharp5 package (which has ~3100)
• RunAOTCompilation=true, WasmAllowUndefinedSymbols=true (AOT-compiling a P/Invoke thunk doesn't need the native symbol to actually resolve - that's a link-time concern)

Result: all 36 assemblies AOT-compiled successfully, including the tiny test app's own dll - no sgen-alloc.c crash. Combined with the wasm-opt workaround (binaryen swap + WasmOptConfigurationFlags), the full dotnet publish -c Release succeeded end to end: https://github.com/shimat/opencvsharp_blazor_sample/actions/runs/29154804554

So the large native library is not the trigger. The crash is specific to AOT-compiling an assembly with OpenCvSharp.dll's shape - most likely the sheer volume of LibraryImport-generated marshalling stubs (~3100), which apparently overwhelms/trips a bug in mono-aot-cross's GC bookkeeping (sgen-alloc.c:409, *p == NULL) that a small P/Invoke surface never exercises.

None of the single-property mitigations tried (DisableParallelAot, WasmNativeDebugSymbols=false, WasmEnableExceptionHandling=false, WasmEnableSIMD=false) avoided the crash on the real OpenCvSharp.dll, so this isn't a simple flag toggle - it likely needs a dotnet/runtime fix in mono-aot-cross itself to handle assemblies with a very large P/Invoke surface. No further workaround attempted.

Status in this repo

Worked around: since the crash is isolated to this specific assembly's AOT compilation, not to native linking or AOT in general, #9 shadows the SDK's internal _WasmAotCompileApp target to exclude only OpenCvSharp.dll from the AOT set (it still ships as plain IL, executed by the interpreter) while the rest of the app gets full AOT. Verified end-to-end: "AOT'ing 38 assemblies" completes with no crash, OpenCvSharp.dll still ships in the published output, and the full deploy-pages pipeline succeeds - https://github.com/shimat/opencvsharp_blazor_sample/actions/runs/29157608898

This is a workaround, not a fix: it shadows an undocumented internal SDK target that could change shape in a future wasm-tools workload update, and the underlying mono-aot-cross bug (crashing on assemblies with a very large P/Invoke surface) is still unfixed upstream. Leaving this issue open to track that.

Related

• #6 (net10.0/OpenCvSharp5 upgrade)
• #7 (GitHub Pages deploy workflow, AOT-off publish workaround)
• Separate, already-tracked-upstream wasm-opt issue that also blocked Release publish: dotnet/runtime#114723

Start with the reproduced dotnet publish -c Release -p:RunAOTCompilation=true command and the BlazorApp/BlazorApp.csproj NativeFileReference setup. Inspect the mono-aot-cross failure at src/mono/mono/sgen/sgen-alloc.c:409 and compare the full OpenCvSharp.dll P/Invoke surface with the three-method test. Done means the AOT compiler no longer crashes while OpenCvSharp.dll is included in the AOT set.