[RasulOs]

Summary

In screenshot-only mode (vision_only, Visual Remote), macro.json recorded the model screenshot size as each action's screen (for example 922x2048), while taps were recorded in device pixels (or iOS points). Replay was not affected because it taps the recorded x/y as-is, but screen did not match the tap coordinates.

Each action's screen is now the size of the space taps are sent in: device pixels on Android, points on iOS. This is the same as in accessibility runs.

• UIState gains optional device_width/device_height. It defaults to screen_width/screen_height, so no other provider changes.
• ScreenshotOnlyStateProvider sets them to the driver input size in both pixel and normalized mode.
• The macro snapshot records them.

Testing

• New test covering pixel, normalized and iOS-points states. It fails on develop in the pixel and iOS-points cases. Full suite: 1612 passed; black clean.
• Android 16 emulator, screenshot-only "Open Settings and go to Network & internet":
  • develop recorded screen 922x2048; this branch records 1080x2400, which matches wm size.
  • Replaying the macro reached the same screen.
• iOS Simulator (iPhone 17 Pro), screenshot-only "Open Settings and open General":
  • develop recorded 942x2048; this branch records 402x874 points.
  • Replaying the macro reached Settings › General.

🤖 Generated with Claude Code

[RasulOs]

@codex review

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-12T11:41:46.973847Z	9c9b5a6	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 9c9b5a678b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[lutz-grex]

Thanks, this is a focused fix. The new dimensions default to the existing screen dimensions, and the added test exercises ScreenshotOnlyStateProvider through click_at and checks both recorded screen fields.

Two nonblocking suggestions:

1. mobilerun/tools/ui/state.py:59–61 already explains that these dimensions describe the tap input space. Names such as input_width and input_height might make that clearer, especially when the units are iOS points.
2. tests/test_macro_recording_actions.py:279 expects (440, 956) for a 440×956 input space, placing the tap at the right/bottom extent rather than inside it. This comes from the existing rounding in convert_point; this PR does not change that conversion. Could the test use an interior point, or explain that it is preserving existing edge behavior?