@@ -2,19 +2,19 @@
 
 Independent native implementation using public Accessibility, Core Graphics and ScreenCaptureKit APIs. Requires macOS 14+. Package `ComputerUse.swift` as `Sim Computer Use.app/Contents/MacOS/SimComputerUse` with this directory's `Info.plist`; sign the nested app with the desktop distribution identity so TCC grants survive upgrades.
 
-Compile each architecture with `swiftc -parse-as-library -O -target arm64-apple-macosx14.0 ComputerUse.swift -o SimComputerUse` (repeat for `x86_64`, combine with `lipo`). No Apple Events, shell execution, private APIs, global event listeners or clipboard access is used. Coordinate mouse input posts public foreground events only after checking the active app, observed window geometry and topmost target point.
+Compile each architecture with `swiftc -parse-as-library -O -target arm64-apple-macosx14.0 ComputerUse.swift -o SimComputerUse` (repeat for `x86_64`, combine with `lipo`). No Apple Events, shell execution, private APIs, global event listeners or clipboard access is used. Coordinate mouse input posts public foreground events only after checking the active app, observed window geometry and that Accessibility hit-testing routes the point to the target app.
 
 ## Protocol
 
 Persistent newline-delimited JSON over stdin/stdout: `{id,method,params}` returns `{id,result}` or `{id,error:{code,message,dispatchState?}}`. The producer contract is `worker/packages/contracts/src/computer-use.ts` in mothership, copied into the desktop bridge by contract sync. `request_permission` is a local onboarding method, never a model tool; params are `{permission:"accessibility"|"screenCapture"}`. Status/preflight never triggers prompts. Errors go through the protocol, and stdout contains no logs.
 
-The helper observes up to 2000 accessibility nodes (depth 64, queue 4000, traversal deadline 8 seconds, each AX call timeout 250ms). Selected-window and focused-element ancestry are prioritized; menu traversal is deferred and capped at 80 nodes. Electron accessibility is enabled using its documented `AXManualAccessibility` attribute. The helper observes 100 windows and 1000 apps. Requests are bounded to 128 KiB during ingestion. Text/value input is capped at 32000 UTF-16 units. State references are per-process, per-app, expire after 60 seconds, and are consumed by every attempted action. Restarts invalidate all references. Screen captures exclude window shadows, preserve the selected window’s coordinate geometry, and are capped to 1600 pixels on the longest side and 8 MiB encoded. Complete JSON replies are capped at 16 MiB: oversized observations omit screenshots first, then truncate accessibility nodes, while preserving action outcomes. Screenshots are optional and failure is reported in `screenshotError` without discarding accessibility state. macOS 26 uses the screenshot-specific capture API; the macOS 14/15 API path includes only the selected window in a display filter and crops its exact frame. That older path requires the window to fit within one display and reports `capture_geometry_unavailable` otherwise.
+The helper observes up to 2000 accessibility nodes (depth 64, queue 4000, traversal deadline 8 seconds, each AX call timeout 250ms). Selected-window and focused-element ancestry are prioritized; menu traversal is deferred and capped at 80 nodes. Electron accessibility is enabled using its documented `AXManualAccessibility` attribute. The helper observes 100 windows and 1000 apps. Requests are bounded to 128 KiB during ingestion. Text/value input is capped at 32000 UTF-16 units. State references are per-process, per-app, expire after 60 seconds, and are consumed by every action that reaches the app; a request that fails before sending anything reports `dispatchState: "not_started"` and keeps them valid. Restarts invalidate all references. Screen captures exclude window shadows, preserve the selected window’s coordinate geometry, and are capped to 1600 pixels on the longest side and 8 MiB encoded. Complete JSON replies are capped at 16 MiB: oversized observations omit screenshots first, then truncate accessibility nodes, while preserving action outcomes. Screenshots are optional and failure is reported in `screenshotError` without discarding accessibility state. macOS 26 uses the screenshot-specific capture API; the macOS 14/15 API path includes only the selected window in a display filter and crops its exact frame. That older path requires the window to fit within one display and reports `capture_geometry_unavailable` otherwise.
 
 Coordinates in action inputs are window-local points; node frames are global screen points. Window bounds must still match the observed snapshot. App PID and launch time must match. Secure fields and descendants are unavailable as action targets; secure values and labels are suppressed. Sim's own apps and System Settings are protected targets. `get_app_state` can open an installed bundle ID without activating it. `list_apps` discovers installed apps under `/Applications`, `/System/Applications`, and `~/Applications` (bounded to 5000 filesystem entries and depth 3), merging running regular apps; installed apps omit `pid`.
 
-AXPress and AXSetValue are direct semantic operations. Keyboard events target the app PID. Cmd+A uses the focused editor's writable accessibility selection range where supported. Coordinate mouse input requires the target app in front; `activate_app` makes that focus change explicit and confirms it. Click/drag/scroll validate the foreground process, unchanged window geometry and topmost window at the point. Some apps ignore background keyboard events. `dispatched:true,verified:false` means only dispatch succeeded; the agent must observe again to establish the outcome. No foreground activation fallback is hidden inside the helper. Native key codes currently assume the US layout for shortcuts; literal text uses Unicode events. Unsupported shortcuts fail explicitly.
+AXPress and AXSetValue are direct semantic operations. Keyboard events target the app PID. Cmd+A uses the focused editor's writable accessibility selection range where supported. Coordinate mouse input requires the target app in front; `activate_app` makes that focus change explicit and confirms it. Click/drag/scroll validate the foreground process, unchanged window geometry and the app that receives the point; the topmost window-list entry is not the click target because Notification Center and the Dock keep full-screen click-through windows. A press that times out while the app opens a sheet or modal (`cannotComplete`) counts as dispatched. A menu-item press that leaves its menu open with the item highlighted (Calendar on macOS 26) is completed with Return. Open menus and their submenus are observed first, and menu item labels carry their shortcuts (for example ⇧⌘N). Some apps ignore background keyboard events. `dispatched:true,verified:false` means only dispatch succeeded; the agent must observe again to establish the outcome. No foreground activation fallback is hidden inside the helper. Native key codes currently assume the US layout for shortcuts; literal text uses Unicode events. Unsupported shortcuts fail explicitly.
 
-`input_sequence` executes 1–32 key/text steps against one observed editable element. It validates every step before dispatch and checks the same editor, window, and secure-input ancestry between steps. Enter can continue into another step when the same editor and window retain focus. Actual focus loss, Tab, or navigation shortcuts end the sequence before later input. Explicit `activateFirst: true` activates the target app inside the batch before input; it never enables automatic refocusing between steps. Partial execution reports completed steps and an error; callers must observe before retrying. Every mutation is followed by fresh state under the same authorization and queue entry; `observeAfter` optionally requests a screenshot. Observation failure is reported separately so the mutation is never replayed. The model receives a compact, bounded projection of the native snapshot; omitted nodes are reported explicitly.
+`type_text` and `input_sequence` target any observed element that can take keyboard focus (text fields, terminals, combo boxes, web editors). `input_sequence` executes 1–32 key/text steps against one such element. It validates every step before dispatch and checks the same editor, window, and secure-input ancestry between steps. Enter can continue into another step when the same editor and window retain focus. Actual focus loss, Tab, or navigation shortcuts end the sequence before later input. Explicit `activateFirst: true` activates the target app inside the batch before input; it never enables automatic refocusing between steps. Partial execution reports completed steps and an error; callers must observe before retrying. Every mutation is followed by fresh state under the same authorization and queue entry; `observeAfter` optionally requests a screenshot. Observation failure is reported separately so the mutation is never replayed. The model receives a compact, bounded projection of the native snapshot; omitted nodes are reported explicitly.
 
 ## Cancellation
 
 
@@ -296,6 +296,28 @@ describe('native computer use authority and lifecycle', () => {
     await expect(click('s1', 'chat', 'repeat')).rejects.toThrow('stale')
   })
 
+  it('keeps a snapshot usable after an action that sent nothing to the app', async () => {
+    const { state, native, click } = setup()
+    await state()
+    native.request.mockRejectedValueOnce(
+      new ComputerUseError({
+        code: 'window_occluded',
+        message: 'Covered.',
+        dispatchState: 'not_started',
+      })
+    )
+    await expect(click('s1', 'chat', 'refused')).rejects.toThrow('Covered.')
+    await expect(click('s1', 'chat', 'retry')).resolves.toMatchObject({ dispatched: true })
+  })
+
+  it('consumes a snapshot after an action whose dispatch is unknown', async () => {
+    const { state, native, click } = setup()
+    await state()
+    native.request.mockRejectedValueOnce(new Error('Computer Use helper exited.'))
+    await expect(click('s1', 'chat', 'lost')).rejects.toThrow('exited')
+    await expect(click('s1', 'chat', 'repeat')).rejects.toThrow('stale')
+  })
+
   it.each(['app', 'window'])('rejects a follow-up snapshot from a different %s', async (target) => {
     const { state, service, native, click } = setup()
     await state()
 
@@ -1,7 +1,8 @@
-import type {
-  ComputerUseActivity,
-  ComputerUseAppPermission,
-  ComputerUseStatus,
+import {
+  type ComputerUseActivity,
+  type ComputerUseAppPermission,
+  ComputerUseError,
+  type ComputerUseStatus,
 } from '@sim/desktop-bridge'
 import {
   type ComputerUseInput,
@@ -234,9 +235,32 @@ export class ComputerUseService {
             ? (owner?.elementWindows.get(input.elementId) ?? owner?.windowId)
             : undefined
       if ('snapshotId' in input) this.snapshots.delete(input.snapshotId)
-      const params =
-        'observeAfter' in input ? omit(input, ['action', 'observeAfter']) : omit(input, ['action'])
-      const result = await this.deps.native.request(input.action, params)
+      /** Mutations accept includeScreenshot as shorthand for observeAfter.includeScreenshot. */
+      const isMutation = input.action !== 'get_app_state'
+      const observeAfter = {
+        ...('observeAfter' in input ? input.observeAfter : {}),
+        ...(isMutation && 'includeScreenshot' in input && input.includeScreenshot !== undefined
+          ? { includeScreenshot: input.includeScreenshot }
+          : {}),
+      }
+      const fields: Record<string, unknown> = input
+      const params = isMutation
+        ? omit(fields, ['action', 'observeAfter', 'includeScreenshot'])
+        : omit(fields, ['action'])
+      let result: ComputerUseResult
+      try {
+        result = await this.deps.native.request(input.action, params)
+      } catch (error) {
+        /** The helper keeps a snapshot nothing was sent through, so its ownership stays too. */
+        if (
+          owner &&
+          'snapshotId' in input &&
+          error instanceof ComputerUseError &&
+          error.details.dispatchState === 'not_started'
+        )
+          this.snapshots.set(input.snapshotId, owner)
+        throw error
+      }
       this.check(work)
       if (result.kind === 'state') this.rememberSnapshot(work, result, windowId)
       if (result.kind === 'action') {
@@ -250,7 +274,7 @@ export class ComputerUseService {
           const observation = await this.deps.native.request('get_app_state', {
             bundleId: input.bundleId,
             ...(windowId ? { windowId } : {}),
-            ...('observeAfter' in input ? input.observeAfter : {}),
+            ...observeAfter,
           })
           this.check(work)
           if (observation.kind !== 'state')
 
@@ -25,6 +25,7 @@ const ObserveAfter = {
     .describe(
       "Every mutation returns fresh app state by default. Set includeScreenshot to true to include an image; otherwise no screenshot is captured. Observation failure never retries the action.",
     ),
+  includeScreenshot: z.boolean().optional().describe("Same as observeAfter.includeScreenshot."),
 };
 const InputSequenceStep = z.discriminatedUnion("action", [
   z.strictObject({ action: z.literal("type_text"), text: z.string().max(32_000) }),
@@ -75,7 +76,11 @@ export const ComputerUseSchema = z.union([
   z.strictObject({
     action: z.literal("press_key"),
     ...ObserveAfter,
-    ...WindowTarget,
+    bundleId: BundleId,
+    snapshotId: SnapshotId,
+    windowId: WindowId.optional().describe(
+      "Window reference from that snapshot. Omit to send the key to the app's focused window.",
+    ),
     key: z.string().min(1).max(128).describe("One key or chord, such as Enter, Tab, Escape, or Cmd+A."),
   }),
   z.strictObject({ action: z.literal("scroll"), ...ElementTarget, ...ObserveAfter, ...ScrollOffsets }),
 
@@ -117,7 +117,7 @@ describe('native computer model projection', () => {
       const result = computerToolResultForModel(nested ? sequence(state) : state)
       const textResult = { ...result, observations: undefined }
       const serialized = JSON.stringify(textResult)
-      expect(new TextEncoder().encode(serialized).length).toBeLessThan(44 * 1024)
+      expect(new TextEncoder().encode(serialized).length).toBeLessThan(22 * 1024)
       expect(serialized).toContain('editor-late AXTextArea parent=e782')
       expect(serialized).toContain('e782 AXStaticText parent=e0')
       expect(serialized).toContain('send-late AXButton parent=e782')
@@ -193,7 +193,7 @@ describe('native computer model projection', () => {
     expect(ids.size).toBeLessThan(state.nodes.length)
     for (const match of tree.matchAll(/\bparent=(\S+)/g)) expect(ids.has(match[1])).toBe(true)
     expect(Buffer.byteLength(JSON.stringify({ ...result, observations: undefined }))).toBeLessThan(
-      44 * 1024
+      22 * 1024
     )
     expect(result).toMatchObject({ dispatched: true, sequence: { completedSteps: 1 } })
   })
 
@@ -3,7 +3,9 @@ import { truncateAtCodePoint } from '@sim/utils/string'
 
 // Keep encoded pixels below the 10 MiB completion body cap, with room for state and its envelope.
 const MAX_SCREENSHOT_BASE64_BYTES = 9 * 1024 * 1024
-const MAX_STATE_BYTES = 44 * 1024
+// Mothership stores a computer result over 24,000 characters and shows the model a preview cut
+// through the middle of the tree; staying under it keeps the prioritized rows the model needs.
+const MAX_STATE_BYTES = 22 * 1024
 const encoder = new TextEncoder()
 
 /** Screenshots are image observations; a bounded readable tree keeps actionable references inline. */
@@ -105,8 +107,10 @@ function snapshotForModel(result: ComputerUseSnapshot, envelopeBytes = 0) {
     const primaryAction = node.actions.some(
       (action) => action === 'AXPress' || action === 'AXConfirm' || action === 'AXPick'
     )
+    /** Items of an open menu have an on-screen frame; closed menus report none. */
+    const openMenuItem = node.role === 'AXMenuItem' && (node.height ?? 0) > 0
     return (
-      (node.focused
+      (node.focused || openMenuItem
         ? 100
         : node.editable
           ? 80
 
@@ -25,6 +25,7 @@ const ObserveAfter = {
     .describe(
       "Every mutation returns fresh app state by default. Set includeScreenshot to true to include an image; otherwise no screenshot is captured. Observation failure never retries the action.",
     ),
+  includeScreenshot: z.boolean().optional().describe("Same as observeAfter.includeScreenshot."),
 };
 const InputSequenceStep = z.discriminatedUnion("action", [
   z.strictObject({ action: z.literal("type_text"), text: z.string().max(32_000) }),
@@ -75,7 +76,11 @@ export const ComputerUseSchema = z.union([
   z.strictObject({
     action: z.literal("press_key"),
     ...ObserveAfter,
-    ...WindowTarget,
+    bundleId: BundleId,
+    snapshotId: SnapshotId,
+    windowId: WindowId.optional().describe(
+      "Window reference from that snapshot. Omit to send the key to the app's focused window.",
+    ),
     key: z.string().min(1).max(128).describe("One key or chord, such as Enter, Tab, Escape, or Cmd+A."),
   }),
   z.strictObject({ action: z.literal("scroll"), ...ElementTarget, ...ObserveAfter, ...ScrollOffsets }),