Repository navigation
fix(driver): fail commands instead of hanging when the Dart VM connection closes - #937
Conversation
|
|
KazuCocoa
left a comment
There was a problem hiding this comment.
Rejecting pending RPCs on disconnect addresses the hang. I reproduced two error-handling gaps that should be covered alongside this change.
| for (const [id, {promise, timeout}] of Object.entries(queue)) { | ||
| clearTimeout(timeout); | ||
| delete queue[id]; | ||
| promise[1](new Error(`The Dart VM connection was closed before it replied`)); |
There was a problem hiding this comment.
[P1] Handle disconnect rejection during initial VM discovery
This also rejects the initial getVM call in connectSocket when no isolateId is supplied. The socket.call wrapper catches that rejection and returns undefined, after which the async onOpenListener reads vm.isolates outside its try/catch. I reproduced an unhandled TypeError (Cannot read properties of undefined, reading 'isolates') by terminating the connection while getVM was pending; the connection promise remained unsettled. Please handle this setup failure and settle the connection promise rather than allowing the event listener to reject unhandled, and add a test for a disconnect during getVM.
| this.log.debug(`>>> ${JSON.stringify(serializedCommand)}`); | ||
| const data = await (this.socket as IsolateSocket).executeSocketCommand(serializedCommand); | ||
| this.log.debug(`<<< ${JSON.stringify(data)} | previous command ${command}`); | ||
| if (!data) { |
There was a problem hiding this comment.
[P2] Propagate disconnect failures to commands that bypass executeElementCommand
This guard only covers commands routed through executeElementCommand. enterText, requestData, setFrameSync, and other handlers call executeSocketCommand directly, so the socket.call wrapper still swallows the newly rejected pending call and resolves with undefined. Using a local RPC server that terminated the connection during ext.flutter.driver, I reproduced execute('flutter:enterText', ['hello']) resolving successfully with undefined instead of rejecting. Please propagate the disconnect failure through those paths too, with a regression test for a direct command handler.
|
Thanks for reproducing both, @KazuCocoa. 4a6036f handles them.
For the P1, the setup that runs when the socket opens is now its own function, and the open listener catches any failure from it, logs the reason, settles the connection promise and closes the socket. So a disconnect while getVM is pending makes connectSocket fail with the usual "Cannot connect to the Dart Observatory URL" error instead of hanging, and there's no unhandled rejection. A getVM reply without an isolates list also fails with a clear message now.
For the P2, the socket.call wrapper rethrows a new DartVmConnectionClosedError, which is the error pending calls get on close, instead of resolving with undefined. A call made while the socket is down is rethrown as the same error. Other errors are still logged and swallowed as before. That covers enterText, requestData, setFrameSync and the other handlers that call executeSocketCommand directly.
One new test drops the socket during getVM and expects connectSocket to reject, and another drops it during ext.flutter.driver and expects the enterText command to reject. Against the previous commit 3 of the 4 socket tests fail, and with this one all 4 pass. The full npm test run (9 of 9), lint and the format check are clean locally.
I left two things alone. After connectSocket gives up, the reconnecting websocket can still reopen in the background, and a drop during the getIsolate retry loop still waits out the 15 s. I can handle either here or in a follow-up if you want.
Thank you,
Michael Lip
Founder of https://zovo.one, 108 Chrome extensions with 150,000+ users
Also building https://makeroll.com and https://ml0x.com, open-source guardrails for Claude Code agents
Open source has to win, and small fixes like this one are my part of it
GitHub Sponsors keeps these fixes coming, https://github.com/sponsors/theluckystrike
|
…tion closes
rpc-websockets 10.0.1 keeps a pending call until a reply or its timeout
arrives, and executeSocketCommand sets no timeout. When the Dart VM
socket closes during a command (app crash or relaunch), the call never
settles and the Appium command hangs until newCommandTimeout.
IsolateSocket now rejects pending calls on `close`. rpc-websockets
emits `close` only after it set `ready` to false, so calls made after
the drop are already refused with "socket not ready" and only calls from
the dead socket are in the queue. The connect wrapper turns a rejection
into an undefined result. executeElementCommand now throws a clear error
for an empty reply instead of a TypeError on `data.isError`. The wrapper
also logged `JSON.stringify(e)`, which is `{}` for an Error. It now logs
the Error itself, so the server log names the cause the new message
points to.
Fixes appium#733
I ran these in driver/ on this commit (main base e13a971), Node 22.23.2.
```
$ npm test
main lib + new tests tests 7 pass 5 fail 2
rejects a pending command when the Dart VM connection closes test timed out after 5000ms
fails an element command instead of hanging when the Dart VM ... test timed out after 5000ms
this commit tests 7 pass 7 fail 0
$ npm run lint
exit 0
$ npm run format:check
All matched files use the correct format.
```
End to end I ran the call the Appium router makes,
`FlutterDriver.executeCommand('execute', 'flutter:waitForAbsent', [finder])`,
against a fake Dart VM service. It drops the socket mid-command and
never replies, the way a crashing app does.
```
main
info Connection to ws://127.0.0.1:53066/ws closed
executeCommand('execute', 'flutter:waitForAbsent') -> STILL PENDING after 15000 ms (hang)
this commit
info Connection to ws://127.0.0.1:53104/ws closed
ERR! Error: The Dart VM connection was closed before it replied
executeCommand('execute', 'flutter:waitForAbsent') -> rejected: Cannot execute command
waitForAbsent, no response from the Dart VM. Check the server log for the connection error [4 ms]
```
I didn't run the device functional workflow (driver-function.yml) because it
needs an iOS simulator with a built Flutter app.
…nnectSocket
Review follow-up for the two gaps in the previous commit.
The socket.call wrapper in connectSocket logged and swallowed every
error, so a call rejected by the disconnect resolved with undefined.
executeElementCommand turned that into an error, but enterText,
requestData, setFrameSync and the other handlers that call
executeSocketCommand directly returned undefined as a success. The
wrapper now rethrows DartVmConnectionClosedError, the error used to
reject pending calls on `close`. A call made while the connection is
down ("socket not ready") is rethrown as the same error. Other errors
are still logged and swallowed as before.
During the initial VM lookup the rejected getVM call made the async
`open` listener read `vm.isolates` of undefined. Nothing awaited the
listener, so the TypeError was an unhandled rejection and the
connection promise never settled. The setup now runs in setUpSocket,
and the `open` listener settles the connection promise with a failure
(and closes the socket) when setUpSocket rejects. connectSocket then
throws its usual "Cannot connect to the Dart Observatory URL" error.
Since appium#948 the isolate lookup runs inside retryInterval. A closed
connection can't recover, so the lookup stops retrying on
DartVmConnectionClosedError and fails the setup right away instead of
retrying for 15 seconds.
Tests: a disconnect during getVM rejects connectSocket without an
unhandled rejection, and flutter:enterText through `execute` rejects
on a disconnect, both during the call and right after it.
4a6036f to
b42a004
Compare
|
@KazuCocoa, I rebased this on main after #948, so it merges cleanly again. Head is b42a004.
The isolate lookup now runs inside retryInterval through selectFlutterDriverIsolate, which already rejects a getVM reply without an isolates list, so I dropped that check from this PR. A closed connection can't recover, so the lookup now stops retrying on DartVmConnectionClosedError and fails the setup right away instead of retrying for about 15 seconds. The rest is unchanged. Pending calls are rejected on close, the call wrapper rethrows the disconnect error, and the open listener settles the connection promise when setup fails.
In driver/ on b42a004, `npm test` (tsc build plus the node test runner) gives 20 pass and 0 fail, and oxlint and the oxfmt format check are clean. Without the retry stop, the getVM disconnect test times out after 5 seconds.
Thank you,
Michael Lip
Founder of https://zovo.one. Chrome extensions with 14,000+ users on public Chrome Web Store pages (27 Oct 2026).
Also building https://makeroll.com and https://ml0x.com, open-source guardrails for Claude Code agents
Open source has to win, and small fixes like this one are my part of it
GitHub Sponsors keeps these fixes coming, https://github.com/sponsors/theluckystrike
|
KazuCocoa
left a comment
There was a problem hiding this comment.
The latest revision addresses both previous findings. One cleanup issue remains in the setup-failure path.
| setUpSocket().catch((e) => { | ||
| this.log.error(`Cannot set up the connection to ${dartObservatoryURL}: ${e.message}`); | ||
| removeListenerAndResolve(null); | ||
| socket.close(); |
There was a problem hiding this comment.
[P2] Cancel the scheduled reconnect when abandoning connection setup
When the connection drops during getVM, rpc-websockets has already cleared its underlying socket and scheduled a reconnect before this catch runs. Its close() method only closes an existing underlying socket, so this call does nothing: connectSocket rejects and removes its setup listeners, but the abandoned client reconnects about a second later and can leave an unmanaged connection open. The new test explicitly waits for that reconnect and closes it from the server, which masks the cleanup gap.
Please disable automatic reconnection and cancel the already scheduled reconnect timer before disposing of this client. Disabling reconnection alone is insufficient because the scheduled callback still calls _connect(). This cleanup can be encapsulated in IsolateSocket alongside the existing access to rpc-websockets internals. Update the regression test to assert that no additional connection occurs after setup fails, rather than closing the leaked reconnect from the server.
When the connection drops during getVM, rpc-websockets 10.0.1 clears its socket and schedules a reconnect (reconnect_timer_id) before the `close` event fires. Client.close() only closes an existing socket, so the setup-failure path did nothing: the abandoned client reconnected about a second later and left an unmanaged connection open. Turning `reconnect` off alone isn't enough because the scheduled callback calls _connect() anyway. IsolateSocket.dispose() turns automatic reconnection off, clears the scheduled reconnect timer and closes any open socket. connectSocket calls it when the setup fails. The regression test no longer closes the leaked reconnect from the server. It now asserts that the server sees no new connection for 2 s after the setup fails, which is longer than the 1 s reconnect interval.
|
Thanks @KazuCocoa, you're right. close() was a no-op there because rpc-websockets had already dropped its socket and scheduled the reconnect. In 68f8be3 I added IsolateSocket.dispose(), which turns reconnect off, clears reconnect_timer_id and then closes any open socket, and the setup-failure catch in connectSocket now calls it.
The regression test doesn't close the leaked connection from the server anymore. Instead it counts server connections for 2 s after connectSocket rejects, with the reconnect interval at 1 s, and asserts there are none. On the previous head it fails with 1 reconnect.
Thank you,
Michael Lip
Founder of https://zovo.one. Chrome extensions with 14,000+ users on public Chrome Web Store pages (27 Oct 2026).
Also building https://makeroll.com and https://ml0x.com, open-source guardrails for Claude Code agents
Open source has to win, and small fixes like this one are my part of it
GitHub Sponsors keeps these fixes coming, https://github.com/sponsors/theluckystrike
|
…ction closes) Conflict in the socket.call wrapper of observatory.ts: keep upstream's DartVmConnectionClosedError handling and the fork's re-throw of other errors. The new getVM-disconnect test gets 9 s instead of 5 s because the fork retries connectSocket 5 times, 1 s apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes #733
The log in #733 ends with
Connection to ws://.../ws closedright after>>> waitForAbsent, and the session then sits idle until UiAutomator2's newCommandTimeout. That's the driver waiting on a reply that can't arrive.IsolateSocket.executeSocketCommandcallsext.flutter.driverwith no timeout. In rpc-websockets 10.0.1 a pending call settles only on a reply or on its own timeout. The clientclosehandler marks the socket not ready and schedules a reconnect, but it never touches the pending queue. Any Dart VM disconnect during a command (app crash, relaunch, reinstall) leaves the Appium command pending forever.Changes
driver/lib/sessions/isolate_socket.tsrejects every pending call when the socket emitsclose. rpc-websockets emitscloseafter it setreadyto false, so calls made after the drop are refused with "socket not ready" and the queue only holds calls from the dead socket.driver/lib/sessions/observatory.tsthrowsCannot execute command <name>, no response from the Dart VMwhen the reply is empty. Thesocket.callwrapper inconnectSocketcatches the rejection and returnsundefined, and without this guard the command fails with a TypeError ondata.isError.JSON.stringify(e), which prints{}for an Error. It now logs the Error itself, so the server log showsThe Dart VM connection was closed before it replied.driver/test/isolate-socket.test.mjsadds 2 tests against an rpc-websocketsServeracting as the Dart VM service. It drops the client whileext.flutter.driveris pending.What I ran in
driver/on Node 22.23.2End to end I ran the call the Appium router makes,
FlutterDriver.executeCommand('execute', 'flutter:waitForAbsent', [finder]), against the same fake Dart VM.The device workflow (driver-function.yml) needs a simulator with a Flutter app and wasn't run here. The reconnect behavior of rpc-websockets stays as it is, and a per-command timeout (#430) is a separate change.