Repository navigation
Commit 4ae9182
committed
fix(desktop): make computer use act on the app the user sees
- Hit-test coordinate input with Accessibility: Notification Center and the Dock keep
full-screen click-through windows, so the window-list check rejected every click
- Count an AX press that times out while a sheet or modal opens as dispatched
- Keep the snapshot when nothing reached the app, in the helper and the service
- Type into any focusable element (terminals, combo boxes), send keys to sheets
and popovers, accept common key aliases
- Complete menu presses that only highlight their item, observe open menus and
submenus first, and show menu shortcuts in labels
- Fit the model projection under Mothership's computer result budget1 parent ce76f14 commit 4ae9182
5 files changed
Lines changed: 149 additions & 38 deletions
File tree
- apps
- desktop
- native/computer-use
- src/main/computer-use
- sim/lib/mothership/tools/client
0 commit comments