Skip to content

iOS: image lightbox swallows subsequent taps — agents end tasks stuck in the viewer #1619

Description

@thymikee

Evidence

AppControlBench bsky re-run (2026-08-05, current main + #1587/#1601#1606, gpt_high): three task verdicts lost to the same trap —

  • bsky-06 (fail): judge: "final screen is an image viewer/loading overlay with caption text, not the post detail/thread page"
  • bsky-07 (fail): judge: "final screen shows an image viewer with image options, not the Following feed with the overflow menu open"
  • bsky-18 (partial): judge: "image viewer/lightbox is open … but the image itself is not visible"

Pattern (also seen in the earlier 0.20.5 re-run analysis, previously unfiled): a tap intended for a post/thread row lands on the post's embedded image, Bluesky opens the full-screen lightbox, and from then on the agent's snapshots show viewer chrome while its thread-navigation taps hit the dimmed overlay. Models rarely recognize the modal and either keep tapping dead space or end the task there.

What's wanted

  1. Snapshot should make the modal state unmistakable — the lightbox is a full-screen presentation layer; if the capture composition can mark it (like the system-modal probe does for alerts), the agent gets "you are in an image viewer, close button at …" instead of a confusing sparse tree.
  2. Investigate whether tap target resolution prefers the image subview over the enclosing pressable row when a selector matches the row — if so, that's the same class as the iOS: no way to dismiss the keyboard — UNSUPPORTED_OPERATION leads to QuickType-bar mis-taps #1598 QuickType mis-tap: the visible target and the hit target disagree.

Repro: seeded bench feed (feedItem-by-whiskers.test has an image embed) — tap the post row near the image, then try to navigate to the thread.

Bench data: data/gpt_high__agent-device/bsky-{06,07,18} in the AppControlBench repo (uncommitted re-run).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions