feat(web-client): add an experimental browser remote control client

[yardbirds0]

This browser remote-control feature is usable in the desktop configurations tested so far, but it is still experimental and not fully polished. Broader compatibility, long-session reliability, and some interactions need more work. Community testing, review, and contributions are welcome so we can improve it together.

Summary

Adds a Console-maintained browser remote-control client that connects through the companion backend's authorized gateway and the existing RustDesk rendezvous/relay servers. It does not embed the proprietary V2 web application. This replaces the previous incremental commits with one consolidated feature commit based on official main at 5659193d88fb3bf513ad487e171f33755c89f750.

Companion backend: databk/rustdesk-console#393

Connection and session interface

• Two-column connection page with manual ID input and the currently authorized device list, online filtering, refresh, and pagination.
• Clear cancellation throughout connecting, remote approval, and password authentication. Remembered credentials are checked before showing the manual password form, avoiding the previous brief form flash.
• A centered toolbar with distinct Session, Display, and Keyboard/Mouse settings. Files, quality monitoring, chat, voice, recording, and fullscreen have direct entries. There is no right sidebar or redundant clipboard panel.
• Session-level read-only mode disables both mouse and keyboard forwarding. Clipboard synchronization and credential removal belong to session settings; mapping/touch options belong to keyboard/mouse settings. The independent screen keyboard uses real protocol input rather than a scattered row of buttons.
• Draggable/resizable chat and quality windows; bounded minimum sizes; quality-panel background opacity from 35% to 100%. File lists get the available window height, with compact headers/toolbars and no redundant explanation block.

Web access follows backend-issued scope rather than existing address-book/native permissions. Ordinary users need web_client.connect; independent files also require web_client.file_transfer. Manual IDs use the same server-side authorization as the device list. Logout/account/token changes, revocation, and stale callbacks cannot leave the previous session usable.

Native protocol and media

The self-contained Worker handles fixed RustDesk protobuf inputs, signed server/peer identity verification, authenticated key exchange, password challenges, bounded binary transport, and native session messages. Console login tokens are not passed into the native protocol. KX 1 uses direction-specific keys; compatible authenticated KX 0 sessions retain a visible legacy-encryption risk/upgrade notice. Invalid identity, downgrade, replay, or decryption failures close the connection instead of silently retrying insecurely.

Tool	Implemented behavior
Display	Monitor switching, fit/original scale, quality presets, custom bitrate ratio, requested 5–120 FPS, remote cursor scaling, and supported remote resolution changes/restoration
Encoding/color	Capability-probed VP8/VP9/AV1; VP9/AV1 8-bit 4:4:4 when browser and remote support it; actual stream headers determine the reported codec/color
Quality monitoring	Received/decoded/drawn FPS, video bitrate, native TestDelay, bounded queues/timings, decoder recovery, and separate entry/relay TCP observations
Clipboard	Direct Unicode/PNG paste and remote-copy synchronization with permission/size limits and cancellation; sent notifications do not claim remote application execution
Files	Independent authorized two-pane session, browser-authorized local directory, remote browsing, center transfer arrows, remote-pane upload/save actions, and Worker-local reuse of native authentication when available
Chat/voice	Native text messages, call negotiation, Opus/PCM handling, and microphone capture released on stop/disconnect/permission loss
Recording	Save received encoded video as a video-only WebM; bounded writer acknowledgments, cancellation, and explicit stop on incompatible codec/display changes

Smooth quality reduces the requested bitrate ratio. Requested FPS is an upper bound, not a throughput guarantee. Video flow control, cooperative Worker scheduling, decode recovery, and bounded audio credit/FIFO scheduling address observed burst/backlog problems without unbounded queues or a claim of guaranteed 120 FPS.

Input and remembered credentials

Mouse, wheel, drag, keyboard/IME, secure-attention/lock actions, mappings, touch input, and the screen keyboard use native protocol messages. Focus loss, hidden tabs, menu/modal changes, permission loss, and disconnect release held input and pending resources. Cursor visibility distinguishes local control from remote-side activity.

Normal pages cannot reliably capture OS/global shortcuts. Supported fullscreen keyboard capture requests the browser's permitted keys, displays its actual state, and releases its lease correctly across delayed API responses. A shortcut detected outside fullscreen prompts at most once per connection, with fullscreen and dismiss actions. Some OS shortcuts can still remain local; the browser's fullscreen-origin banner cannot be renamed by this application.

Remembered device credentials are derived password-equivalent material encrypted in IndexedDB with a non-exportable key, scoped to Console account/server/device, and handled inside the Worker. There is no plaintext fallback or backend password storage. Invalid authentication and explicit removal invalidate records; canceled/stale writes cannot restore them. This does not protect against malicious same-origin code.

Platform integration, locales, and packaging

• Roles and permission presentation, session/audit navigation, protected request handling, and existing server-management screens are integrated. Existing management tabs remain visible with an honest unavailable state when no node is enrolled.
• A single read-only TCP information component serves the existing Services overview and expanded frame diagnostics. Label/value association is explicit. Tooltips show system/deployment/reading scope, red failure reasons, and a compact algorithm table. System configuration is never presented as every connection's algorithm; unobservable entry/relay sockets remain unidentified.
• Complete key parity for Chinese, English, French, Brazilian Portuguese, and Russian, including new permission labels and diagnostics. Removed 32 obsolete Web Client keys from every locale.
• Content-bound page/Worker identity, hashed Worker assets, a no-store manifest, bounded service-worker transition handling, correct MIME/404 behavior, license/source notices, and upstream update metadata are built together. Old pages cannot silently start a mismatched Worker.
• Pinned protocol source attribution and AGPL notices remain. Removed the obsolete direct-connection smoke harness, old clipboard commands/conversion branch, unused styling, duplicate instructions, and dated implementation/acceptance snapshots from the PR. A private synthetic media harness remains for decoder tests only; it cannot bypass Console authorization.

Deployment and limits

Requires the companion backend, HTTPS/secure browser APIs, fixed server public key, native WS rendezvous/relay support, and proxy Upgrade forwarding to /api/web-client/socket. Protect legacy/raw anonymous browser WS bypasses; do not treat a frontend visibility check as authorization. Existing native passwords/remote approval and feature permissions still apply. No custom hbbs/hbbr fork is required for remote control, and there is no P2P media mode in this implementation.

Browser directory/file/microphone/fullscreen APIs have compatibility and user-activation requirements. Native clipboard/file restrictions remain important because the transparent backend cannot inspect all encrypted content. Audio device output and two-way human conversation need broader acceptance. Mobile/multi-monitor/native-version, long sessions, and the previously intermittent hover-without-click symptom are not declared universally resolved.

Validation

Local official-PR equivalents and additional client checks passed on the synchronized baseline:

• Fresh npm ci; 87 Jest suites / 812 tests passed, including the official API/access/update subset. The final cleanup additionally passed 64 targeted tests.
• TypeScript, Biome lint and format, pinned protocol-generation verification, and 20 Node build/PWA/source-identity/release tests passed. Biome lint has 27 warnings and 42 informational diagnostics, with no errors; this is not a zero-warning claim.
• Final production Mako/Worker build and Nginx Docker image build passed.
• Final paired-container checks verified route/assets, manifest caching, Worker MIME/hash, missing-asset 404s, licenses, backend startup/authorization, and real Nginx WS Upgrade rejection paths.

Historical real-device testing received and decoded VP8/VP9/AV1, exercised VP9/AV1 4:4:4 and remote resolution changes, verified transferred file bytes and playable video-only recording, and checked bounded decoded audio scheduling. Those results are not a complete matrix or certification of this final cleanup snapshot. No new production deployment was performed for this review.

Progress and remaining work

[████████░░] 8/10 review milestones complete

• Official-main integration and single-commit consolidation
• Native connection/authentication and lifecycle handling
• Media/input/display tools
• Clipboard, files, chat, voice, and recording implementation
• Role/audit/server-management integration and five locales
• Source/license/cache integrity and obsolete-code cleanup
• Local official-CI equivalents and client regressions
• Final paired-container HTTP/WS checks
• Broader real-device/browser/OS and long-session matrix
• Maintainer/community review and follow-up polishing

Please keep this as a draft and use the remaining checklist to guide collaborative testing.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 6d177e3b-cd80-4181-90fc-eafd898130ff

---

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}