callstack / callstack/agent-device
investigate(ios): confirm historical XCTest tap-failure storm root cause
- Dominant language
- TypeScript
- Stars
- 4.6k
- Forks
- 299
- Avg merge
- 10h 14m
- Merged PRs (30d)
- 537
Description
## Purpose
Confirm or falsify the remaining historical root-cause hypothesis for the iOS `XCTEST_RECORDED_FAILURE` tap storm seen on hostile/profile-class screens.
The user-facing retry-spiral defect is already fixed:
- #1605 corroborates ambiguous recorded tap failures against a fresh, comparable post-action capture and returns success-with-warning when the tap landed.
- #1634 keeps corroboration on the baseline snapshot backend, covering the private-AX/tree penalty-boundary mismatch seen in the live Bluesky repro.
- After #1587, repeated live attempts could no longer reproduce the natural failure storm.
This issue no longer owns tap-outcome recovery, generic runner restart behavior, or additional retry logic.
## Remaining question
Was the archived failure storm caused by XCTest-channel saturation and abandoned capture work before #1587, or does an independent current defect remain?
## Investigation contract
1. Reproduce against the archived AppControlBench `bsky-24` evidence or a controlled pre-#1587 build using the same profile-screen action sequence.
2. Compare a pre-#1587 run with current main under the same app state and command sequence.
3. Retain runner logs and request diagnostics covering:
- capture duration and penalty state;
- abandoned or coalesced runner work;
- `XCTEST_RECORDED_FAILURE` timing;
- restart/relaunch boundaries;
- whether the UI changed despite the reported failure.
4. Distinguish causal saturation evidence from correlation. A changing timeout/failure set or a non-equivalent app state is not confirmation.
## Completion
- **Confirmed historical cause:** record the matched evidence showing the storm disappears when the #1587 channel-pressure change is applied, then close this issue.
- **Independent current defect:** provide a reproducible current-main sequence and open a narrowly scoped implementation issue for its owning boundary; do not expand this historical investigation into a speculative fix.
- **Not reproducible:** record the bounded attempts and evidence limits, then close as superseded by #1587, #1605, and #1634.
No production change is required unless a distinct current-main defect is reproduced.
Contributor guide
Research direction
Start with the archived AppControlBench bsky-24 evidence or a controlled pre-#1587 build, then compare the same action sequence with current main. Retain runner logs and request diagnostics for capture timing, abandoned work, failure timing, restart boundaries, and UI changes; done means recording confirmed causation, a reproducible current defect, or bounded non-reproduction before closing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ios, typescript
- Domain
- mobile, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100