callstack / callstack/agent-device

perf(ios): align AX recovery contracts and evaluate generation-scoped depth hints

Open
#2,424 0 comments 0 reactions 0 assignees View on GitHub
needs-triage
Dominant language
TypeScript
Stars
4.6k
Forks
299
Avg merge
10h 17m
Merged PRs (30d)
515

Description

## Purpose

Reduce repeated AX rejection work without weakening snapshot completeness, and prevent the host bridge and XCTest runner recovery paths from drifting.

Follow-up to #2414 and #2391; implement after #2414 lands. This is performance/design work, not a release blocker. Unknown frontier completeness must already fail closed in #2414.

## Current data flow and ownership

`open` -> launch observation -> Simulator source -> native capture/recovery. Capture failure opens the snapshot route's generation circuit; #2391 makes subsequent launch observation consult that same owner. Relaunch rebaselines the circuit. Optional coordinate-tap field observation has separate, local failure containment.

The host source defaults to traversal depth 64 (65 native levels), allows two lower-depth retries per acquisition and 32 total requests. It forgets accepted depth between captures. The runner has a 56/40/24/12 ladder, 8 frontier-extension calls and expiring accepted-depth memory. These are different private native representations and currently different failure/completeness policies, not interchangeable implementations.

Crucially, `SnapshotBridgeManager.ensureSession` reuses a process by simulator UDID. The process is not scoped to one app generation. Never use bridge lifetime as a depth-hint validity key.

## Work

First attribute the full path: launch observation currently asks for regular presentation depth1 (not raw traversal depth1), acquires a tree, and discards it before a later capture. Measure resolve/start/connect, launch observation, acquisition/continuation, presentation, and first interaction separately. Do not assume accepted-depth memory is the largest remaining cost. A cheaper readiness contract or request-local handoff requires separate evidence of equivalent readiness/freshness; do not quietly reinterpret presentation depth or add a cross-command stale-tree cache in this issue.

1. Add shared, executable recovery conformance fixtures consumed through each producer's adapter. Cover zero-based requested depth versus native levels, node budgets, complete/withheld/unknown frontier evidence, missing live elements, owner change, request exhaustion, deadline/cancellation, and explicit raw-depth requests. Document intentional backend differences. Start with shared contracts and fixtures; extract common executable policy only where equivalence is proven. Do not combine private object decoding with recovery or wire serialization.
2. Benchmark a bounded accepted-depth hint in the host acquisition owner. Use the existing resolved target identity (`targetId` plus `generation`) and producer identity; do not create another lifecycle registry or circuit. Learn only from a successful, complete recovery. Never share hints across apps, generations or native producers. A hint changes request strategy only, never delivered depth or completeness.
3. Give hints explicit expiry/probe-back behavior. Shallow successes must not indefinitely renew a low hint. Compare caching against the current bounded retry path before adopting it; drop it if total work or common-case latency regresses. Preserve existing raw-depth semantics and the request/deadline budgets.

Owning code: `apple/snapshot-bridge/SnapshotBridgeCapture.m`, `SnapshotBridgeRuntime.m`; `packages/platform-apple/src/snapshot-source/{lifecycle,adapter}.ts`; runner `RunnerAXSnapshotBridge.m` and `RunnerTests+AXSnapshotFallback.swift`. Generation circuit ownership stays in `snapshot-route.ts`.

## Done when

- Both producer adapters run the shared fixtures, with planted-red proof for missing boundary evidence, depth errors, leaked hints after relaunch and budget overruns.
- A repeated deep-screen capture avoids known rejected requests while retaining the same deep content and truthful completeness. A deep-to-ordinary screen transition proves the hint expires or probes back rather than imposing permanent continuation overhead.
- Paired results report native calls, rejected calls, continuations, total capture latency and first-interaction latency for an ordinary app and at least two structurally different difficult screens. Include cold/warm runs, sample count, median/tail results and noise. Benefits must exceed measurement noise; healthy-path regression is a reason to reject the policy.
- Benchmark raw and regular captures separately. Keep the 31% Bluesky result tied to #2414's earlier measured revision; do not attribute it to this follow-up.
- No app-name heuristics, shared backend health penalties, parallel speculative trees by default, or new lifecycle owner.

Expected benefit: fewer repeated expensive AX rejections and stronger regression protection across both recovery paths. A shared engine is an optional outcome, not the goal.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.