callstack / callstack/agent-device
scroll keyboard clip follow-ups: double keyboard probe on iOS, tap/drag width-fraction gate, horizontal-scroll refusal
- Dominant language
- TypeScript
- Stars
- 4.6k
- Forks
- 299
- Avg merge
- 10h 17m
- Merged PRs (30d)
- 515
Description
## Context
The #2500 stack (#2537 → #2503 → #2514) clips a directional scroll's swipe above the on-screen keyboard and refuses with `scroll_keyboard_occludes_surface` when too little surface is left. Three things were found in review and left out of the stack on purpose. None blocks merge; each is its own small change.
## 1. iOS scroll probes the keyboard twice per command
`resolvedScrollViewport` reads `visibleKeyboardFrame` to clip the band. The dispatch then goes through `executeSynthesizedDragGesture` → `axFreeSynthesizedDragPlan` → `keyboardAvoidingSynthesizedDragPoints` (`RunnerTests+Interaction.swift:912`), which reads `visibleKeyboardFrame` again and shifts the drag if it intersects the keyboard. For a scroll the second pass is a no-op by construction: the band already ends `accessoryAllowance` above `keyboardMinY`, so the planned points never intersect. Each probe is a live `app.keyboards.firstMatch` query.
Before the stack, the first scroll of a session ran zero probes (policy `.whenAccessibilityHealthy` on `.unknown`); it now runs two. Options:
- carry the frame the clip measured into the context so the drag pass reuses it, or
- give the scroll dispatch a context with `keyboardPolicy: .never` at `RunnerTests+CommandExecution.swift:1995`, since the clip already accounted for the keyboard.
Measure the saved time on a large tree (Bluesky or the deep-tree bench) before choosing; the `AGENT_DEVICE_RUNNER_SYNTHESIZED_GESTURE_POLICY` log line must still make sense for scroll.
## 2. Tap/drag reference frame keeps the width-fraction gate
`frameAvoidingKeyboard` (`RunnerTests+Interaction.swift:780`) still gates on `intersection.width / frame.width >= 0.5` (line 790) and fails open under 25% safe height. That is the same defect #2503 removed from scroll: a floating keyboard centred at 38% of the width sits in the path and is ignored. It shapes tap and drag reference frames, so fixing it changes the tap family and needs its own device evidence. `keyboardAvoidingDragPoints` (`:672`) also keeps a local `padding = 12` that names the same fact as `SCROLL_KEYBOARD_ACCESSORY_ALLOWANCE`.
Decide whether the tap family should share `clipScrollViewportAboveKeyboard` (and its golden table) or stay separate with its own fail-open policy. Do not silently switch tap to fail-closed.
## 3. Horizontal scrolls refuse under the vertical floor
`clipScrollViewportAboveKeyboard` (`packages/contracts/src/scroll-gesture.ts:412`) applies the 15% floor regardless of direction. A `scroll left`/`right` runs along the horizontal centre line of the clipped band, so a thin band can still carry it; today it is refused. Rare and conservative. If it matters, pass the direction into the clip and only apply the floor to vertical scrolls, adding rows to `contracts/fixtures/scroll-keyboard-policy.json` so the Swift twin stays in step.
Contributor guide
Assessment
This issue has not been assessed yet.