Repository navigation
fix(state): retry timed-out state reads once and say when we are waiting - #507
Merged
Merged
Conversation
A timed-out get_ui_tree means the device side is still busy with the request we abandoned. Portal and the mobilerun-ios portal (WDA) serve state one request at a time, so the old 7-attempt schedule queued six more reads behind a stuck app: up to ~4 minutes of `device ui --ios` with nothing printed for the first 30s (DROID-305, mobilerun#484). - Timeouts (TimeoutError, httpx.ReadTimeout) now get one retry, preceded by the recovery callback; other errors keep the full backoff schedule and late recovery. Connect timeouts are not treated as a busy device. - Log "Waiting for the UI state from the device..." at INFO once when a fetch runs past 3s. - `mobilerun device ui` sends INFO progress to stderr when no handler is configured, so the notice is visible and stdout stays the tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RasulOs
approved these changes
Oct 23, 2026
This was referenced Oct 25, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A user reported that
mobilerun device ui --ioswith TikTok open on an iPhone X running iOS 16 "does nothing and just hangs, no log lines". Their output eventually showed:fetch_state_with_retrymade 7 attempts with 1/2/3/5/8/10s backoff. Each attempt was aGET /state_fullwith a 30s httpx timeout. Worst case was about 4 minutes, with nothing printed for the first 30s. A timeout means the device side is still working on the request we abandoned. The portal (backed by WDA) and Android Portal both serve state one request at a time. So each retry queued another snapshot behind the app that was already stuck.The root cause is fixed on the device side in droidrun/WebDriverAgent#43, released in v1.3.5 and v1.3.6. When an app stops answering accessibility, WDA now responds within about 15s with the foreground package and a partial tree marked
truncated. Through the portal that takes 15–19s, 23s at worst, which fits inside the 30s driver timeout. Older WDA builds can still go past 30s. This PR is the client-side follow-up. A companion mobilerun-ios change lowers the portal's WDA state deadline below 30s.Refs DROID-305, #484 (both already closed).
Changes
mobilerun/tools/ui/provider.py: retry policyTimeoutErrororhttpx.ReadTimeout) is retried once. The recovery callback runs before that retry. All other errors keep the existing 7-attempt backoff, the late recovery after 5 failures, and the immediate recovery when the accessibility service is gone. Timeouts are counted across the whole call, so a timeout, then another error, then a timeout gives up after 3 attempts.httpx.ConnectTimeoutandPoolTimeoutare not treated as a busy device. Those requests never reached it, so retrying them doesn't pile up work, and they keep the normal schedule.max_retries.Why the timeout rule is right for every driver. The loop is shared by every driver that uses
AndroidStateProvider:IOSPortalHttpDriver) and Android Portal HTTP: aReadTimeoutmeans the single-threaded server is still busy. Each extra retry just adds to the queue.TimeoutErrorfrom a wedged on-device serializer, after 15s. Before this PR, recovery (restarting the accessibility service) ran only after 5 such timeouts, roughly 75s or more of waiting. Now it runs right after the first timeout, so Android recovers sooner._recover_portaldoes nothing for non-ADB drivers, so the single retry is a plain second read.Worst case with timeouts: about 30s + 1s + 30s ≈ 61s through the portal, down from about 4 minutes. On Android ADB it is about 15s + recovery + 15s. With WDA ≥ v1.3.5 the first read normally succeeds in 15–23s.
fetch_state_with_retry: visible wait. When a fetch is still running after 3s, it logsWaiting for the UI state from the device...at INFO, once per call. It doesn't repeat on the retry. If the caller is cancelled, the fetch is cancelled too.mobilerun/cli/device_commands.py: making that INFO line visible.device uinever set up logging, so Python's last-resort handler dropped everything below WARNING. That's why the user saw nothing for 30s.device uinow attaches a plain stderr handler at INFO when themobilerunlogger has none. stdout still carries only the tree, so scripts that parse it are unaffected. A logger that is already set up (by the SDK or the agent CLI) is left alone.Left out: The 30s per-request driver timeout (in mobilerun-core-local) is unchanged. The new WDA-plus-portal answer fits inside it. The cloud driver's
APITimeoutErroris not classified as a timeout here: that driver already retries its own reads.Tests
tests/test_state_fetch_retry_policy.py(12 tests):ReadTimeoutandTimeoutErrorConnectTimeoutkeeps the full scheduledevice uistderr handler, including leaving an existing one alonepytest tests: 1342 passed, 41 subtests passedblack==26.10.1 --check .(CI pin): 203 files unchangedruff checkon the changed files: passedNo device testing.
🤖 Generated with Claude Code