Skip to content

Sheets: confirm clicks that open or close a sheet, route agents to the parent window (#243, #245) - #247

Merged
iamnbutler merged 2 commits into
mainfrom
sheets-route
Nov 12, 2026
Merged

iamnbutler merged 2 commits into
mainfrom
sheets-route

Conversation

@iamnbutler

Copy link
Copy Markdown
Contributor

Fixes #243. Fixes #245. Also covers the sheet part of #244 (an incomplete tree right after a sheet appears).

Reproduced with a new fixture, native2/fixture/sheetfix.swift: an AppKit window with a beginSheet sheet and an NSAlert sheet, and a second window hosting a SwiftUI .alert.

What was wrong

Changes

  • Pointer.swift: when a press comes back indeterminate, it rereads the window's sheet count for up to 1 s. If the count changed, the result is completed (confirmed_change, sheet: "opened"|"closed"). Indeterminate results now carry ax_error.
  • AX.swift: sheets are read from AXSheets and from AXSheet children. A sheet's window is matched by its window ID, or by containing the sheet's frame with the same center.
  • desktop_windows: a sheet's own window gets observation_reason: attached_sheet and sheet_of_window_id. Its availability is unavailable / attached_sheet with parent_window_id and a hint: inspect the parent window, click there, and use desktop_dialog on the parent.
  • desktop_inspect on a sheet's own window (either mode) now returns ATTACHED_SHEET right away and names the parent, with no 1 s retry and no "use pixels mode" advice. The check only runs for untitled windows.
  • The parent's screenshot now draws its attached sheets, so the image matches the AXSheet elements listed.
  • desktop_dialog / desktop_dialog_answer: button titles fall back to AXDescription. A sheet's own window target is refused with a message naming the parent.
  • An action's follow-up observation retries once after 0.5 s when the tree is incomplete, as inspect already does.
  • src/tools.ts: the desktop_windows description names attached_sheet and sheet_of_window_id. The contract is regenerated.

Checked

  • swift test --package-path native2: 54 + 33 passed. npm run check is clean.
  • Live on the fixture, through the standalone host built from this branch (the dev app had no Accessibility or Screen Recording grant on this Mac yet):
    • Open Sheet, Cancel, Delete, Open Alert and alert Cancel each returned completed with sheet: opened/closed. Before this change, Open Sheet returned unknown / unsafe, as the release app still does.
    • SwiftUI Show Alert followed by Cancel or Delete, three rounds: all completed, each with a fresh observation.
    • desktop_windows lists both sheet windows as attached_sheet with the right parent.
    • desktop_dialog on each parent lists the buttons, including the SwiftUI ones. On a sheet's own window it names the parent.
    • Pixels inspect of the parent shows the sheet, for both AppKit and SwiftUI.

Not changed: Xcode's own sheets weren't retested (the brief said to use the fixture).

🤖 Generated with Claude Code

iamnbutler and others added 2 commits November 12, 2026 14:43
…o the parent window (#243, #245)

A press that opens or closes a sheet now reads back the window's sheets when the app
answers cannotComplete, and reports completed with sheet opened/closed instead of
unknown. desktop_windows marks a sheet's own window attached_sheet with
sheet_of_window_id and a hint to inspect the parent; inspecting the sheet's window
refuses at once with ATTACHED_SHEET naming the parent; the parent's screenshot draws
its attached sheets; desktop_dialog reads SwiftUI alert buttons (titled by
description) and names the parent when given the sheet's window. An action's
follow-up observation retries once when the tree is mid-change. Adds the sheetfix
fixture (beginSheet, NSAlert sheet, SwiftUI .alert).

Why: native2/Sources/DesktopMac/Pointer.swift | AppKit and SwiftUI answer an AXPress that opens or closes a sheet with cannotComplete although the press took effect, so an indeterminate press is confirmed by the window's sheet count changing
Why: native2/Sources/DesktopMac/AX.swift | AppKit windows don't support AXSheets: sheets are AXSheet children of the window, and a SwiftUI alert's window adds a transparent margin around the AXSheet frame, so match by containment and center, not exact bounds
Why: native2/Sources/DesktopMac/Capture.swift | a sheet is its own window server window, so a capture of its parent omits it; attached sheets are drawn into the parent's image so it matches the parent's elements

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant