[DaniilSmirnov]

Stream Android screens as H.264 with scrcpy 5

--need-scrcpy existed but could not work: it started a scrcpy server from 2019 (v1.0.20) that modern Android does not run, scrcpy.js used __dirname in an ES module, logged every chunk with console.log, leaked a listener per WebSocket client, and the UI had nothing to decode the stream.

This PR makes it a working alternative to minicap: the device streams H.264 from the official scrcpy-server v5.0.1, and the browser decodes it with WebCodecs. minicap stays the default.

Device side

• resources/scrcpy.js runs scrcpy-server video-only (audio=false control=false video_codec=h264). Input still goes through minitouch and STFService. It connects through localabstract:scrcpy_<scid> with openLocal, like minicap, so no adb forward port is needed.
• resources/util/scrcpy-stream.ts parses the v5 stream: codec id, session packets (new size on rotation), 12-byte frame headers with config/key-frame flags.
• plugins/screen/util/h264relay.ts sends each packet to the screen WebSocket clients unchanged:
  • a client that joins mid-stream gets the session, the config (SPS/PPS) and the packets since the last key frame, so its picture starts right away;
  • a client whose socket backs up (bufferedAmount) skips to the next key frame instead of queueing delay.
• plugins/screen/stream.js: scrcpy is a second mode of the existing FrameProducer state machine.
  • Rotation sends a new start banner instead of restarting, because scrcpy follows the rotation itself.
  • Projection and quality changes restart the server. A size or quality that arrives while the server is starting is used as is, without a second start.
  • If the server binary is missing or scrcpy cannot start on a device, the device falls back to minicap.
• Quality slider: --scrcpy-bit-rate (default 4 Mbit/s) is the bit rate at quality 80, the top of the slider, and lower values scale it down (minimum 200 kbit/s). Sending the same quality again does not restart the stream; auto quality does this every 5 s.
• --screen-frame-rate sets max_fps. Key frames come every 2 s so that new viewers and viewers that fell behind recover quickly.

Server binary

The jar is no longer committed. scripts/fetch-scrcpy-server.mjs downloads the official release into vendor/scrcpy/ and checks its SHA-256, which is the checksum Homebrew pins for the same release. The Docker build runs the script. Elsewhere, run it once by hand. Without the binary, devices use minicap.

UI

• The start banner has codec: "h264" in scrcpy mode. DeviceScreenStore then sends the packets to H264ScreenDecoder (WebCodecs VideoDecoder, Annex B with SPS/PPS sent with each key frame) and draws only the latest decoded frame on each animation frame.
• The screen WebSocket now uses binaryType = 'arraybuffer'; JPEG frames are wrapped in a Blob as before.
• Browsers without WebCodecs show an explicit error (en and ru translations). WebCodecs needs a secure context: HTTPS or localhost.

How to try

node scripts/fetch-scrcpy-server.mjs
stf local --need-scrcpy

Testing

• Unit tests: stream parser (including byte-by-byte chunks), relay (late joiner, slow client, new session), the scrcpy resource with a fake adb (start command, retries until the server listens, first packets delivered), the screen producer with fake scrcpy and minicap (quality restarts, minicap fallback), quality → bit rate, and the browser decoder.
• End to end without a device: ffmpeg encoded two sessions (480×864, then 864×480), packed in the scrcpy-server v5 format, sent through the real parser, relay and WebSocket to Chromium running the real decoder. All 210 frames were decoded with no errors, including the size change. A viewer that joined mid-stream started from a key frame.
• Not tested on a physical device or emulator yet. Please try a few Android versions before merging.

Known limitations

• Resizing the screen and changing the quality restart scrcpy-server, which takes 1–2 s (minicap restarts faster).