dotnet / dotnet/macios

[CI] RegistrarTest.TestRetainReturnValue intermittently observes an extra retain

Open
#26,544 0 comments 0 reactions 0 assignees View on GitHub
ci-postmortem copilot
Dominant language
C#
Stars
2.9k
Forks
576
Avg merge
2d 13h
Merged PRs (30d)
96

Description

## Summary

`MonoTouchFixtures.ObjCRuntime.RegistrarTest.TestRetainReturnValue` has intermittently observed an Objective-C return value with retain count 3 when the test expects 2. The failure is now confirmed in two unrelated PRs, on different platforms/configurations and in different assertion subcases.

- Stable signature: `RegistrarTest.TestRetainReturnValue intermittently observes Objective-C return-value retain count 3 instead of expected 2 in different ownership subcases`
- Classification: genuinely flaky returned-object ownership/retain-count test
- Distinct monitored PRs: 2
- Confirmed occurrences: 2
- First observed: 2026-09-02T17:21:57Z
- Latest observed: 2026-09-02T17:46:13Z

## Confirmed occurrences

### PR #26513

- Head: `a354efde93df4b8e7ad9eb15a88ebd64916b2be1`
- [Build 15184608, attempt 1, X64 Mac Sonoma / Run tests](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15184608&view=logs&j=623bc498-120b-5ff2-984a-5b073efffe1b&t=4a2ee696-2483-5c21-b74f-dbb45ac9f281)
- Worker: `XAMBOT-1028.Sonoma`
- Platform/configuration: macOS x64 Sonoma lane
- Assertion subcase: `overridden nsobject`
- Expected retain count: 2; actual: 3
- Same-build controls: the exact test passed 20 times in `T: monotouch_macos`, passed on macOS x64 under Sequoia and Tahoe, and passed in the Sonoma Mac Catalyst x64 run.
- The PR changes `CFMessagePort`; no registrar or `RegistrarTest` source is modified.

### PR #26498

- Head: `6356809ff5ccece154e19a4e1a401c2b27a8cee4`
- [Build 15184965, attempt 1, `T: monotouch_ios` / Run tests](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15184965&view=logs&j=a70ed34c-97c9-5ebb-c5be-8e2604f5ffcc&t=113cecf5-45b6-5477-24c3-52ec12451502)
- Worker: `AcesShared 170`
- Platform/configuration: iOS simulator Debug, managed static registrar
- Assertion subcase: `inativeobject`
- Expected retain count: 2; actual: 3
- NUnit result: 3,861 total, 3,669 passed, 1 failed, 158 skipped, and 33 inconclusive.
- Same-build controls: the exact test passed in 24 configurations and failed in only this one.
- The PR changes xtro-sharpie tooling and baseline files; no registrar, runtime, or test source is modified.

## Impact and reproduction pattern

A single intermittent extra retain fails an otherwise passing test configuration and causes the entire simulator stage to fail. The subcase and platform vary, while the stable symptom remains `RetainCount == 3` instead of 2.

## Investigation

The published TestSummary, HTML reports, raw NUnit XML, assertion stack traces, PR file lists, and same-build passing configurations were compared directly. The test intentionally drains an `NSAutoreleasePool` before measuring ownership, but these two runs still retained one additional native reference in different return-value paths.

Separate searches across all open and closed issues used the exact test name, assertion text, subcase labels, `RetainCount`, and registrar/INativeObject terms. #26525 only mentions the #26513 test as a separate failure while tracking an unrelated Sonoma cleanup timeout. #7442, #10284, and #25232 concern different runtime or registrar designs and do not record this intermittent assertion signature.

## Current diagnosis and workaround

The cross-configuration passes and unrelated PR changes establish a genuine flake, but the source of the extra retain is not yet known. Useful follow-up would capture the native retain/autorelease transitions and managed wrapper identity for each subcase when the count differs.

No retry has been submitted specifically for either occurrence: #26513 has an indeterminate Sonoma retry reservation, while #26498 shares `simulator_tests` with a deterministic xtro failure and Azure does not expose a supported individual-job retry selector. Retrying the whole stage would violate the policy against rerunning deterministic failures.

Azure logs and artifacts are retention-limited.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with MonoTouchFixtures.ObjCRuntime.RegistrarTest.TestRetainReturnValue and compare the failing assertion subcases with the Azure logs and artifacts from PRs #26513 and #26498. Capture native retain/autorelease transitions and managed wrapper identity when RetainCount is 3; done means identifying the extra retain and making the test pass reliably across the affected configurations.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, objective-c
Domain
mobile-dev, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.