[CI] RegistrarTest.TestRetainReturnValue intermittently observes an extra retain
- Dominant language
- C#
- Stars
- 2.9k
- Forks
- 576
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 96
Description
## Summary
`MonoTouchFixtures.ObjCRuntime.RegistrarTest.TestRetainReturnValue` has intermittently observed an Objective-C return value with retain count 3 when the test expects 2. The failure is now confirmed in two unrelated PRs, on different platforms/configurations and in different assertion subcases.
- Stable signature: `RegistrarTest.TestRetainReturnValue intermittently observes Objective-C return-value retain count 3 instead of expected 2 in different ownership subcases`
- Classification: genuinely flaky returned-object ownership/retain-count test
- Distinct monitored PRs: 2
- Confirmed occurrences: 2
- First observed: 2026-09-02T17:21:57Z
- Latest observed: 2026-09-02T17:46:13Z
## Confirmed occurrences
### PR #26513
- Head: `a354efde93df4b8e7ad9eb15a88ebd64916b2be1`
- [Build 15184608, attempt 1, X64 Mac Sonoma / Run tests](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15184608&view=logs&j=623bc498-120b-5ff2-984a-5b073efffe1b&t=4a2ee696-2483-5c21-b74f-dbb45ac9f281)
- Worker: `XAMBOT-1028.Sonoma`
- Platform/configuration: macOS x64 Sonoma lane
- Assertion subcase: `overridden nsobject`
- Expected retain count: 2; actual: 3
- Same-build controls: the exact test passed 20 times in `T: monotouch_macos`, passed on macOS x64 under Sequoia and Tahoe, and passed in the Sonoma Mac Catalyst x64 run.
- The PR changes `CFMessagePort`; no registrar or `RegistrarTest` source is modified.
### PR #26498
- Head: `6356809ff5ccece154e19a4e1a401c2b27a8cee4`
- [Build 15184965, attempt 1, `T: monotouch_ios` / Run tests](https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=15184965&view=logs&j=a70ed34c-97c9-5ebb-c5be-8e2604f5ffcc&t=113cecf5-45b6-5477-24c3-52ec12451502)
- Worker: `AcesShared 170`
- Platform/configuration: iOS simulator Debug, managed static registrar
- Assertion subcase: `inativeobject`
- Expected retain count: 2; actual: 3
- NUnit result: 3,861 total, 3,669 passed, 1 failed, 158 skipped, and 33 inconclusive.
- Same-build controls: the exact test passed in 24 configurations and failed in only this one.
- The PR changes xtro-sharpie tooling and baseline files; no registrar, runtime, or test source is modified.
## Impact and reproduction pattern
A single intermittent extra retain fails an otherwise passing test configuration and causes the entire simulator stage to fail. The subcase and platform vary, while the stable symptom remains `RetainCount == 3` instead of 2.
## Investigation
The published TestSummary, HTML reports, raw NUnit XML, assertion stack traces, PR file lists, and same-build passing configurations were compared directly. The test intentionally drains an `NSAutoreleasePool` before measuring ownership, but these two runs still retained one additional native reference in different return-value paths.
Separate searches across all open and closed issues used the exact test name, assertion text, subcase labels, `RetainCount`, and registrar/INativeObject terms. #26525 only mentions the #26513 test as a separate failure while tracking an unrelated Sonoma cleanup timeout. #7442, #10284, and #25232 concern different runtime or registrar designs and do not record this intermittent assertion signature.
## Current diagnosis and workaround
The cross-configuration passes and unrelated PR changes establish a genuine flake, but the source of the extra retain is not yet known. Useful follow-up would capture the native retain/autorelease transitions and managed wrapper identity for each subcase when the count differs.
No retry has been submitted specifically for either occurrence: #26513 has an indeterminate Sonoma retry reservation, while #26498 shares `simulator_tests` with a deterministic xtro failure and Azure does not expose a supported individual-job retry selector. Retrying the whole stage would violate the policy against rerunning deterministic failures.
Azure logs and artifacts are retention-limited.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with MonoTouchFixtures.ObjCRuntime.RegistrarTest.TestRetainReturnValue and compare the failing assertion subcases with the Azure logs and artifacts from PRs #26513 and #26498. Capture native retain/autorelease transitions and managed wrapper identity when RetainCount is 3; done means identifying the extra retain and making the test pass reliably across the affected configurations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, objective-c
- Domain
- mobile-dev, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100