callstack / callstack/agent-device

wait: account startup and readiness separately from observation time, and say when infrastructure prevented any observation

Open
#2,343 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
4.6k
Forks
299
Avg merge
10h 14m
Merged PRs (30d)
537

Description

## What happens today

`wait ` charges its whole budget to whatever the polls cost, including work that is ours rather than the app's: the XCTest runner's first command on a freshly launched app, runner start or restart, and AX-bridge target discovery. The deadline also cancels the poll in flight, so a first poll that costs more than the budget ends with zero completed observations.

Four iOS Smoke runs since 2026-09-05 21:00 show it on the first `wait text "Agent Device Tester"` after a cold `open --relaunch` (`wait_capture_stalled`, `readableCaptures: 0`):

- main `80997b6b`: https://github.com/callstack/agent-device/actions/runs/33992502696
- main `f30328d0`: https://github.com/callstack/agent-device/actions/runs/34014253127
- PR #2326: https://github.com/callstack/agent-device/actions/runs/34015120372
- branch `fix/2314-memoize-platform-runtime-la`: https://github.com/callstack/agent-device/actions/runs/34016148244

Request log of the third run (`sessions/ios-e2e-smoke-14ax/requests/45ad58af8fd6970d.ndjson` in `ios-artifacts`):

| phase | cost |
|---|---:|
| runner `findText` on the just-launched app | 5.8s |
| AX-bridge target discovery, `xcrun timed out after 3000ms` | 3.4s |
| fallback XCTest snapshot | cancelled at the 10s deadline |

An earlier instance of the same accounting, before #2279, is https://github.com/callstack/agent-device/actions/runs/33984867542 (`sessions/ios-e2e-smoke-ohv/requests/6af0b46614602883.ndjson`): the runner stopped accepting connections mid-wait, the daemon invalidated and relaunched it, and the wait was cancelled 0.7s after `xcodebuild` started, reported as `wait_capture_stalled` rather than a runner restart because the cancelled capture carried no restart evidence.

#2331 takes the discovery probe off the per-capture critical path. #2325 gives `open` its own startup budget. Neither changes what a `wait` budget pays for.

## What to change

- Account startup and readiness time (runner start, runner restart, target discovery, first-command warm-up) separately from observation time in the wait loop, and surface both in the failure details.
- Keep one overall deadline and cancellation. A hung capture must still end at the deadline, so this is not a guarantee of "at least one completed observation".
- When infrastructure prevented any observation before the deadline, say so explicitly with its own reason or typed detail, distinct from "the device gave no readable picture", and keep `wait_capture_stalled` for the latter.
- Decide whether readiness time should extend the observation window (bounded, once) or only be reported. The report is the prerequisite either way.

## Related

- #2331 discovery probe off the critical path
- #2325 `open` startup deadlines
- #2329 observation runner-free, launch grace
- #2113 runner restart evidence on wait timeouts

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.