dotnet / dotnet/runtime

Re-enable library tests skipped on Browser (wasm/CoreCLR) for being too slow in the outerloop lane

Open
#129,973 4 comments 1 reaction 1 assignee Claimed by @radekdoulik View on GitHub
arch-wasm area-Infrastructure-libraries disabled-test
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

### Background

[PR #129436](https://github.com/dotnet/runtime/pull/129436) adds a WebAssembly + CoreCLR libraries-test leg to the scheduled **outerloop** pipeline, enabling `[OuterLoop]` and long-running suites (re-enabled from #125495) with extended Helix/xharness timeouts.

A handful of individual tests are still too slow or too resource-heavy to finish within the per-suite xharness timeout (4h) on the **single-threaded wasm interpreter**, so they were skipped on Browser via `[SkipOnPlatform(TestPlatforms.Browser, …)]`. This issue tracks those skips so they can be revisited when wasm test execution gets faster (e.g. JIT/AOT for test runs, threading) or the tests are made cheaper.

These skips only affect the new wasm/CoreCLR outerloop lane — desktop and other platforms are unchanged, and these tests never ran on any wasm *innerloop* lane.

### Tests skipped because they are too slow (CPU time) on the wasm interpreter

| Suite | Test | Why slow on wasm |
|---|---|---|
| System.Linq.Expressions | `CompilerTests.CompileDeepTree_NoStackOverflow` | compiles a 10,000-deep expression tree |
| System.Memory | `EncodingExtensionsTests.GetBytes_Encoding_ReadOnlySequence_IBufferWriter_LargeMultiSegment` | iterates ~3.2bn chars of UTF-16 |
| System.Memory | `EncodingExtensionsTests.GetChars_Encoding_ReadOnlySequence_IBufferWriter_LargeMultiSegment` | iterates ~6.5bn bytes of UTF-8 |
| System.Text.Json | `BitStackTests.BitStackPushPopLarge` | pushes/pops up to 67M bits |
| System.Text.Json | `Stream.ReadTests.ReadTypeFromJsonWithLargeIgnoredProperties_OuterLoop` | processes 5 GiB of JSON |
| System.Text.Json | `Utf8JsonReaderTests.TestJsonReaderLargestUtf8SegmentSizeOne` | O(n²) single-byte segment slicing |
| System.Text.Json | `Utf8JsonReaderTests.TestPartialJsonReaderSlicesMultiSegment` | O(n²) multi-segment slicing (one row ≈ 32 min) |
| System.Text.Json | `Utf8JsonReaderTests.TestPartialJsonReaderSlicesSpecialNumbers` | O(n²) slicing of special numbers |
| System.Text.Json | `Utf8JsonWriterTests.Writing3MBBase64Bytes` | writes a 3 MB base64 payload across every option combo |
| System.Text.Json | `Utf8JsonWriterTests.EscapeCharacters` | every writer-option combo (already `[OuterLoop]` on Mono) |
| System.Text.Json | `Utf8JsonWriterTests.WriteNumbers` | every writer-option combo (already `[OuterLoop]` on Mono) |

### Tests skipped due to memory exhaustion on the constrained wasm heap (related)

| Suite | Test | Why |
|---|---|---|
| System.Collections.Immutable | `FrozenDictionaryTests.ToFrozenDictionary_WithExtremelyLargeStrings` | allocates ~1 GB strings |
| System.Collections.Immutable | `FrozenSetTests.ToFrozenSet_WithExtremelyLargeStrings` | allocates ~1 GB strings |

### Whole `[OuterLoop]` category skipped for one suite

- **System.Linq.Parallel.Tests** — the full `[OuterLoop]` combinatorial matrix expands to ~195k cases on a single-threaded wasm session (~72 MB results file exceeding xharness's 30 MB upload limit, ~1.3 GB heap, tab crash at teardown). PLINQ has no real parallelism on a single thread, so this is dropped on the wasm/CoreCLR lane via `WithoutCategories=OuterLoop`.

### Related — different root cause (a hang, not slowness)

- **System.Private.Xml.Linq** `xNodeBuilder/FunctionalTests.RunTests` — also skipped on Browser in #129436, but because the **wasm runtime hangs at shutdown** after this suite completes (a hang, not slowness). Listed here for completeness; likely warrants its own investigation.

### Re-enablement criteria

Revisit each skip when wasm test execution is materially faster (JIT/AOT for test assemblies, multithreaded runtime) or when the individual tests are restructured to be cheaper.
Related: #125495, #129436.

> [!NOTE]
> This issue was drafted with the help of GitHub Copilot.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.