dotnet / dotnet/macios

[CI] NativeAOT DispatchGroupTest.WaitTest deadlocks on macOS

Closed
#26,537 11 comments 0 reactions 0 assignees View on GitHub
ci-failure ci-postmortem
Dominant language
C#
Stars
2.9k
Forks
576
Avg merge
2d 13h
Merged PRs (30d)
96

Description

## Failure signature

The macOS NativeAOT `monotouch-test` run stalls in `MonoTouchFixtures.CoreFoundation.DispatchGroupTest.WaitTest` until the 1,200-second harness timeout. LLDB shows the main thread blocked in `DispatchGroup.Wait` / `_dispatch_group_wait_slow`.

Stable signature:

```text
macOS NativeAOT MonoTouchFixtures.CoreFoundation.DispatchGroupTest.WaitTest deadlocks the main thread in DispatchGroup.Wait until the 1200-second harness timeout and forced exit 134
```

## Confirmed occurrences

### PR #26260

- Head: `214d20e20e6b79cc09774e59b943146a2bc19255`
- [Build 15180972, `T: monotouch_macos` / Run tests](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15180972&view=logs&j=0564bbc5-6c6b-5cb4-3d05-9ed0b2c86721&t=723ec601-2b3c-5c84-d6d6-e6999dfb9129)
- Configuration: macOS Release NativeAOT
- Main thread blocked in `DispatchGroup.Wait` / `dispatch_group_wait`; the harness killed the process after 1,200 seconds.
- Targeted simulator attempt 2 passed on `AcesShared 10`, and the PR subsequently merged.

### PR #26521

- Head: `d5e4fccb049599c0a36996106b83a8b49a3b417c`
- [Build 15182784, `T: monotouch_macos` / Run tests](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15182784&view=logs&j=0564bbc5-6c6b-5cb4-3d05-9ed0b2c86721&t=723ec601-2b3c-5c84-d6d6-e6999dfb9129)
- Worker: `AcesShared 75`
- Configuration: macOS Release, trimmable static registrar, NativeAOT
- The fixture passed `EnterLeaveTest` and `NotifyWithAction`, then stalled in `WaitTest`. LLDB shows `_dispatch_group_wait_slow` → `DispatchGroup.Wait` → `DispatchGroupTest.WaitTest`.
- TestSummary reports one timed-out configuration and 19 passed configurations. No crash report was produced.
- The PR currently has merge conflicts, so no retry is permitted.

First observed: `2026-09-02T06:45:32Z`.
Latest confirmed occurrence: `2026-09-02T12:42:35Z`.
Recurrence: **2 jobs across 2 distinct monitored PRs**.

## Classification and impact

This is a genuinely flaky NativeAOT test deadlock. The exact managed/native stack reproduces in unrelated PRs and cleared in #26260's targeted rerun. It is not a generic macOS timeout and is not the same root as #25687's broader near-suite-completion timeout family.

The deadlock blocks the complete simulator stage and prevents the NativeAOT variation from returning a test result.

## Investigation performed

- Inspected Azure timelines, task logs, TestSummary/HtmlReport artifacts, and LLDB stacks.
- Identified the exact active fixture and test immediately before each timeout.
- Compared the successful #26260 retry.
- Searched all open and closed `dotnet/macios` issues using the exact test name, managed/native stack frames, fixture name, and timeout text. No matching issue existed.
- Rejected #25687 as a broad timeout-family match with a different exact root.

## Workaround

After the run is terminal, rerun only the failed `monotouch_macos` job when the PR is conflict-free and retry budgets permit it.

Azure logs and test artifacts are retention-limited.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with MonoTouchFixtures.CoreFoundation.DispatchGroupTest.WaitTest and run the failed macOS NativeAOT monotouch_macos configuration, then inspect the DispatchGroup.Wait and _dispatch_group_wait_slow stack. Compare the failure with the successful targeted rerun from PR #26260. Done means the WaitTest no longer deadlocks and the NativeAOT simulator stage returns a test result.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, macos
Domain
desktop-dev, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.