callstack / callstack/agent-device

iOS: image lightbox swallows subsequent taps — agents end tasks stuck in the viewer

Open
#1,619 0 comments 0 reactions 0 assignees View on GitHub
needs-triage
Dominant language
TypeScript
Stars
4.6k
Forks
299
Avg merge
10h 17m
Merged PRs (30d)
515

Description

## Evidence

AppControlBench bsky re-run (2026-08-05, current main + #1587/#1601–#1606, gpt_high): three task verdicts lost to the same trap —

- **bsky-06 (fail)**: judge: "final screen is an image viewer/loading overlay with caption text, not the post detail/thread page"
- **bsky-07 (fail)**: judge: "final screen shows an image viewer with image options, not the Following feed with the overflow menu open"
- **bsky-18 (partial)**: judge: "image viewer/lightbox is open … but the image itself is not visible"

Pattern (also seen in the earlier 0.20.5 re-run analysis, previously unfiled): a tap intended for a post/thread row lands on the post's embedded image, Bluesky opens the full-screen lightbox, and from then on the agent's snapshots show viewer chrome while its thread-navigation taps hit the dimmed overlay. Models rarely recognize the modal and either keep tapping dead space or end the task there.

## What's wanted

1. Snapshot should make the modal state unmistakable — the lightbox is a full-screen presentation layer; if the capture composition can mark it (like the system-modal probe does for alerts), the agent gets "you are in an image viewer, close button at …" instead of a confusing sparse tree.
2. Investigate whether tap target resolution prefers the image subview over the enclosing pressable row when a selector matches the row — if so, that's the same class as the #1598 QuickType mis-tap: the visible target and the hit target disagree.

Repro: seeded bench feed (feedItem-by-whiskers.test has an image embed) — tap the post row near the image, then try to navigate to the thread.

Bench data: `data/gpt_high__agent-device/bsky-{06,07,18}` in the AppControlBench repo (uncommitted re-run).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.