Skip to content

fix(cua-driver): unify native action scheduling - #3391

Open
injaneity wants to merge 16 commits into
codex/2238-target-scoped-rebindfrom
rfc/3390-state-scoped-observations
Open

injaneity wants to merge 16 commits into
codex/2238-target-scoped-rebindfrom
rfc/3390-state-scoped-observations

Conversation

@injaneity

@injaneity injaneity commented Aug 26, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

Cua Driver used separate code to coordinate desktop actions and text entry. It also kept its own list of tools for each path, so a new tool could miss the right checks.

This pull request replaces that code with one small scheduler. It now:

It also removes the old desktop lock, text-process set, custom cleanup guard, and duplicate tool list.

There are no browser or public API changes.

Refs #3373

Checks

Candidate 4e7bc5b20f567217034b4d0a50d542d67bd65eb0:

  • formatting and diff checks pass;
  • focused tests cover waiting, refusal, cancellation, cleanup, and tool selection; and
  • all 23 reported CI checks pass.

This branch needs to be rebased onto the latest #3373 before merge.

@injaneity injaneity closed this Aug 26, 2026
@injaneity injaneity reopened this Aug 26, 2026
@injaneity
injaneity force-pushed the rfc/3390-state-scoped-observations branch from 554f6de to ed13b3e Compare August 26, 2026 07:54
@injaneity injaneity changed the title docs(cua-driver): propose state-scoped observations fix(cua-driver): serialize complete desktop action observations Aug 26, 2026
@injaneity
injaneity force-pushed the rfc/3390-state-scoped-observations branch from ed13b3e to d7bd829 Compare August 26, 2026 08:10
@injaneity injaneity changed the title fix(cua-driver): serialize complete desktop action observations fix(cua-driver): unify mutable resource scheduling Aug 26, 2026
@injaneity
injaneity force-pushed the rfc/3390-state-scoped-observations branch 2 times, most recently from 34fa6f5 to fec066f Compare August 26, 2026 09:49
@injaneity injaneity closed this Aug 26, 2026
@injaneity injaneity reopened this Aug 26, 2026
@injaneity
injaneity marked this pull request as ready for review August 26, 2026 09:54
@injaneity
injaneity requested a review from f-trycua as a code owner August 26, 2026 09:54
@injaneity
injaneity marked this pull request as draft August 26, 2026 09:56
@injaneity injaneity changed the title fix(cua-driver): unify mutable resource scheduling fix(cua-driver): unify native action scheduling Aug 26, 2026
@injaneity
injaneity force-pushed the rfc/3390-state-scoped-observations branch 3 times, most recently from 821ac0f to 5f1eda2 Compare August 26, 2026 10:23
@injaneity
injaneity force-pushed the rfc/3390-state-scoped-observations branch from 5f1eda2 to fc1f5c4 Compare August 26, 2026 15:41
@injaneity
injaneity marked this pull request as ready for review August 26, 2026 15:55
injaneity and others added 7 commits August 27, 2026 09:23
Keep modality and focus as internal resolver facts. Publish only the window identity needed for logical rebinding, and ignore unrelated foreground changes.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>
Adapt pi-computer-use’s cheap signal and authoritative AX diff while retaining Cua Driver ownership proof, focus suppression, and typed action accounting.

kvnloo commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

I think the unified scheduler still has one lifecycle hole that matters for the invariant it is trying to establish: the lease lifetime is tied to the async dispatch future, not necessarily to the native work it admitted.

ToolRegistry acquires _desktop_action_lease, then does:

let mut result = tool.invoke(args.clone()).await;
...
drop(_desktop_action_lease);

That is correct on normal completion. But many platform actions run irreversible work through tokio::task::spawn_blocking(...).await — macOS drag is a clear example, and the keyboard/platform paths have the same shape.

Tokio cannot cancel a spawn_blocking closure once it has started. If the outer tool future is dropped/cancelled after native entry:

  1. tool.invoke(...).await is abandoned;
  2. _desktop_action_lease is dropped with the dispatch future;
  3. the native blocking closure can keep delivering input;
  4. a second action can now acquire PhysicalDesktop and overlap the still-running first mutation.

So the scheduler can report the lane free while the resource it represents is still physically in use.

The current scheduler regression:

cancelled_waiter_does_not_keep_or_poison_a_resource

covers the opposite boundary — cancellation before acquisition — and is useful, but it does not prove cancellation after acquisition/native entry.

This appears to compose directly with #3796 rather than requiring a second cancellation design. I would record the scheduler invariant as:

A native-action lease is released only when the admitted native mutation has reached a boundary that can no longer affect that resource. Dropping the caller's waiter is not such a boundary.

For this PR, the smallest safe choices seem to be either:

The discriminating regression is straightforward:

  1. acquire the physical desktop lease;
  2. start a blocking action behind a barrier and prove native entry;
  3. cancel/drop the outer async invocation;
  4. attempt a second desktop action before releasing the barrier;
  5. require the second action not to enter;
  6. release the first native worker, then require the lane becomes available exactly once.

A held-input variant (drag/key-down) is even stronger because it observes the actual resource rather than only the scheduler.

The conceptual distinction is important: cancelled waiter != completed owner. The scheduler should model the lifetime of the effect it serializes, not only the lifetime of the task that requested it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants