callstack / callstack/agent-device
Limrun iOS: Flutter text entry fails with “No focused accessibility element found” after tapping an editable field
- Dominant language
- TypeScript
- Stars
- 4.6k
- Forks
- 299
- Avg merge
- 10h 17m
- Merged PRs (30d)
- 515
Description
## Reproduction and observed behavior
Using agent-device 0.21.0 against a Flutter login screen on a Limrun iOS cloud simulator, snapshot exposes Email and Password as editable text fields. Filling Email, or tapping the field and using type, fails with:
```text
No focused accessibility element found — Unexpected internal error. Retry with --debug and report the diagnostics log if it persists.
```
The first attempt may display the iOS slide-to-type tutorial. Dismissing it and taking a fresh snapshot does not resolve subsequent typing failures. Repeated ref/coordinate focus and type attempts still fail.
The same Flutter app accepts input locally. A React Native app tested on Limrun does not exhibit the failure. This suggests a compatibility problem in this path, not proof of a particular Flutter engine defect. Exact Flutter/iOS versions and a standalone public reproduction are still needed.
## Current implementation
At e3880f10c826fea373cc48153996b372ef39136e, `packages/provider-limrun/src/ios.ts` sends both `type` and `fill` through `client.typeText`; `fill` first taps the supplied coordinates. The repository resolves `@limrun/api` 0.24.5, whose `typeText` signature has no focus-check option.
## Confirmed downstream workaround
We patched the bundled iOS client’s `typeText` request sender in agent-device 0.21.0:
1. Send the normal request first.
2. Only on the exact server error `No focused accessibility element found` (without the CLI’s appended diagnostic hint), retry once with the same `text` and `pressEnter` and `requireFocus: false`.
3. Propagate all other failures and any failure of the recovery request; never replay on a timeout or unknown failure.
We confirmed that this restores typing in the affected Flutter app. The workaround stays inside agent-device; the QA application does not call Limrun directly. This is a tested downstream workaround, not validation of a patch against current main.
## Proposed upstream direction
Use a supported SDK exposing `typeText(text, pressEnter, { requireFocus: false })`, and agree on a typed pre-input focus rejection or an explicit focus-bypass policy. Current upstream guidance forbids branching on error-message text; the temporary workaround should not be copied unchanged.
Do not bypass the focus check globally or retry after potentially partial writes. Preserve type/fill semantics and verify the intended field’s value. Keep keyboard tutorial handling separate.
Validation should include normal input, focus-check rejection, timeout/no replay, unsuccessful recovery, delayed character entry, existing-value fill, password fields, and live Flutter/React Native verification on Limrun. An SDK upgrade must also be validated against the provider’s existing lifecycle and recording API usage.
Would maintainers prefer a typed SDK error prerequisite or an explicit adapter option for input after a resolved field tap?
Contributor guide
Assessment
This issue has not been assessed yet.