Repository navigation
Conversation
Use an on-demand UI Automation worker and match completed remote gestures in physical screen coordinates. Publish Foundation-compatible v1 metadata to the owning encrypted control connection.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Description
When a streaming client taps a Windows text field, it needs to know whether the click reached an editor and where that editor appears in the captured frame. This change lets the client opt into a 76-byte v1 remote text context message over its encrypted control connection. The client decides whether to show its keyboard.
The Windows UI Automation worker runs while at least one client subscribes. It checks for a focused, enabled, explicitly writable Edit control and reports focus metadata and geometry. Field contents stay on the host. The router matches completed taps or clicks, translates signed desktop coordinates to capture-local pixels, and sends updates to the owning connection. The Input settings page includes a host switch.
The wire format follows Foundation Sunshine #1029. Companion client-library PR: moonlight-stream/moonlight-common-c#176
Screenshot
Issues Fixed or Closed
Roadmap Issues
Testing
sunshine.exeandtest_sunshine.exeon Windows x64 with MSYS2 UCRT64, Release, system tray disabled.test_sunshine.exewithSUNSHINE_TEST_INTERACTIVE_FOCUS=1andSKIP_SYSTEM_TESTS=1: 919 passed, 15 skipped, 0 failed. All seven new focus tests passed, including a real Win32 Edit gaining focus and deactivating when focus moved to a read-only Edit. Routing tests cover negative-origin coordinates, cancelled gestures, multitouch, ambiguous hits and session lifecycle. Skips included tray, URL launcher, device, encoder and platform-specific tests.npm run buildand the complete Vitest suite: 221 passed. A separate local smoke test decoded both host-generated state transitions with the companion common-c decoder.Still to verify: installed Windows service mode, physical mixed-DPI monitors with a negative desktop origin, Android Moonlight V+ streaming, and frame-time impact during gameplay. The coordinate tests used synthetic inputs. The decoder smoke test passed payloads directly. Linux and macOS builds were not run.
Type of Change
Checklist
AI Usage
See our AI usage policy.