callstack / callstack/agent-device

perf(ios): detect a presented system surface from the bridge via AX hit test, replacing the host-side process probe

Open
#2,485 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
4.6k
Forks
299
Avg merge
10h 17m
Merged PRs (30d)
515

Description

Split out of #2448 review so the design decision stays separate from that PR's validation.

**Today (shipped in #2448).** `packages/platform-apple/src/system-surface-presence.ts` decides host-side whether a registered system surface host (`com.apple.SafariViewService`) is present, by `pgrep -f` plus a `ps eww` device-scope read, and the route takes the XCTest runner for anything but a proven absent. It is ~123 lines plus ~137 of tests, a tri-state contract, a memo, and a fork/exec on every eligible Simulator capture.

**Why it could be smaller and better.** Process liveness is not presentation. Measured on iOS 26.2 with a real `ASWebAuthenticationSession` sheet, using a standalone probe built the way the bridge is:

| state | host process | today's probe | `+[AXElement applicationAtCoordinate:]` |
|---|---|---|---|
| no sheet | absent | absent → bridge | app ✓ |
| sheet presented | running | present → runner | `com.apple.SafariViewService` ✓ |
| dismissed, host lingering | still running | present → runner ✗ | app ✓ |

The third row is a false positive the current probe has by construction: while the host lingers after dismissal, every capture needlessly takes the slow path. A hit test at the centre of the primary app's frame reports the truth in all three states.

**Refuted alternative, recorded so nobody re-proposes it.** `primaryApp` disassembles to `currentApplications.firstObject`, and the bridge already computes that list three times per capture — so "scan the list we already have" looks free. It does **not** work: with the sheet presented, `currentApplications` stays `count=1` and never contains the host. Measured, not reasoned.

**Sketch.** In `SnapshotBridgeRuntime.m`, extend the existing foreground check to also answer "is a registered host presenting over this app", and return a typed `unsupported` / `system-surface-presented` failure before any tree work. Host side, replace the probe call in `snapshot-route.ts` with one early branch on that code. No protocol bump: a new `error_code` under the existing `kind` passes through `bridgeFailureFromEnvelope`, and editing the `.m` auto-invalidates the cached binary via the existing source hash.

**Not free, and these are the real risks.** It needs its own early branch rather than reusing `fallbackAfterFailure`, which would `disabledGenerations.add(...)` and disable the bridge for that generation over a transient sheet. It moves the check after `resolveTarget`, so while a sheet is up each capture pays a bridge round trip instead of a `pgrep` — still cheaper, and only while a sheet is up. `applicationAtCoordinate:` is private API, same stability class as `primaryApp`, and guards the same way (`respondsToSelector:` → typed failure → runner).

**Bonus.** `createLaunchObservationProbe` never consults the host probe, so `open`'s launch observation can currently report `observable` off an occluded tree. A guest-side check closes that for free.

**Completion.** The probe module, its tri-state and memo, and `processExecutable` in `contracts/fixtures/ios-system-surface-hosts.json` (it exists only to feed `pgrep -f`) are deleted; the lingering-host false positive is gone; live-validated through the production bridge on a real sheet, including the dismissed-but-lingering case.

Blocked by #2448.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.