Skip to content

perf(recording): cut Windows readback copies and encode from textures - #2940

Open
Addin wants to merge 3 commits into
CapSoftware:mainfrom
Addin:perf/windows-capture-readback-ring
Open

Addin wants to merge 3 commits into
CapSoftware:mainfrom
Addin:perf/windows-capture-readback-ring

Conversation

@Addin

@Addin Addin commented Oct 22, 2026 •

Copy link
Copy Markdown

Summary

Windows capture currently copies every frame back to the CPU with a blocking Map(READ), then copies it again into an FFmpeg frame. This removes the second copy: the mapped staging texture is wrapped directly as an FFmpeg-backed frame, and instant/fragmented M4S recordings can encode straight from the captured D3D11 texture through Media Foundation. The existing FFmpeg encoder remains the fallback.

Changes

  • scap-direct3d: staging-texture ring with in-use tracking; bounded D3D11_MAP_FLAG_DO_NOT_WAIT polling by default (spin for up to 4 ms, then sleep in 100 µs steps to an 8 ms deadline); CAP_READBACK_BLOCKING=1 for an unbounded Map; D3D11_CREATE_DEVICE_VIDEO_SUPPORT; WARP fallback uses BGRA_SUPPORT only; one zeroed padding row.
  • scap-ffmpeg: the mapped staging texture is an AVBufferRef-backed AVFrame; the per-frame pool and memcpy are gone.
  • enc-mediafoundation / mediafoundation-ffmpeg: explicit-bitrate constructor, sequence_header(), 2 s keyframe spacing, sample_to_packet, bounded encoder-tail drain; iterate MFTs, negotiate types inside the candidate loop, and read the sequence header from GetOutputCurrentType.
  • enc-ffmpeg: SegmentedPacketMuxer writes externally encoded H.264 into the existing DASH init/m4s/manifest and declares the encoded size, not the capture size.
  • win_fragmented_m4s: MF texture input is preferred on non-AMD hardware; setup failure falls back to FFmpeg; a runtime MF failure finalizes written segments and emits MuxerCrashed.
  • scap-ffmpeg/examples/cli.rs: --bench A/B, --path validation, MinUpdateInterval, and failure counters that only count exercised paths.
  • Removed the unused win_segmented* muxers and DiskSpaceCallback plumbing.

Test machine

  • HP Pavilion Gaming Notebook
  • Intel Core i7-6700HQ, 4C/8T, 2.6 GHz
  • 16 GB DDR3L-1600 (2×8 GB)
  • NVIDIA GeForce GTX 950M, 4 GB, driver 581.94
  • Intel HD Graphics 530, driver 31.0.101.2115
  • Windows 11 24H2, build 26100.9711
  • 1920×1080 @ 60 Hz
  • Rust 1.88.0; benchmark harness x86_64-pc-windows-gnu, workspace validation x86_64-pc-windows-msvc

Performance

Readback A/B (--bench --seconds 30; 60 s runs were consistent)

resolution path p50 p95 p99 max frames / misses
1280×720 old 3.10 ms 3.66 ms 3.93 ms 4.21 ms 891 / 0
1280×720 new 2.64 ms 3.08 ms 3.16 ms 3.38 ms 891 / 0
1600×900 old 2.92 ms 3.11 ms 3.25 ms 6.28 ms 891 / 0
1600×900 new 2.03 ms 2.21 ms 2.26 ms 2.69 ms 891 / 0
1920×1080 old 4.12 ms 4.58 ms 13.62 ms 15.18 ms 892 / 0
1920×1080 new 2.82 ms 3.02 ms 8.21 ms 8.92 ms 881 / 11

The bounded default wins at every resolution: p50 3.10→2.64 ms (720p), 2.92→2.03 ms (900p), 4.12→2.82 ms (1080p); 1080p p99 drops 13.62→8.21 ms. CAP_READBACK_BLOCKING=1 is slightly faster at 1080p (2.71/2.12/2.94 ms p50 at 720p/900p/1080p with zero misses) but can stall on slow hardware.

The remaining cost is GPU readback latency, not the memcpy: CopySubresourceRegion + Flush submits in ~60 µs, while Map waits ~2–3.5 ms for the copy. Encoding from the texture avoids that map.

Instant mode

metric MF texture input FFmpeg fallback (h264_nvenc) FFmpeg software (libx264)
CPU, avg total / per core 8.1% / 1.0% 11.2% / 1.4% 52.3% / 6.5%
decoded frames / duration 758 / 30.19 s 750 / 30.58 s 766 / 30.23 s
bitrate 7.84 Mbps 7.80 Mbps 7.93 Mbps
segments 13 13 13
stop latency 70.6 ms 140.7 ms 89.0 ms

MF uses 8.1% total CPU versus 11.2% for NVENC and 52.3% for software x264, with the lowest stop latency; bitrate is comparable at 7.8–7.9 Mbps.

Validation

  • cargo fmt --all --check: pass.
  • cargo clippy --workspace --exclude cap-desktop --all-features --locked -- -D warnings (Windows MSVC): pass.
  • cargo build --workspace --exclude cap-desktop --locked (Windows MSVC): pass.
  • Affected tests: scap-direct3d 5/5, cap-mediafoundation-ffmpeg 1/1.
  • Full workspace tests: scap-direct3d and cap-mediafoundation-ffmpeg pass. Remaining failures are hardware/environment-specific (NVENC segment timing, unavailable HEVC MF, pre-existing chaos/editor tests); the camera-pts-late MP4 MF export case is outside this path and needs a follow-up.

Scope

  • AMD remains excluded from the MF path, matching the existing MP4 hardware path.
  • The Windows MP4 hardware muxer (H264StreamMuxer) is unchanged; this PR targets instant/fragmented M4S.
  • 4K and WARP performance were not measured (1080p panel; WARP accepts BGRA only).
  • On Optimus, the recording executable needs the Windows high-performance GPU preference (GpuPreference=2) for NVIDIA MFT activation.

RetriggerView in GreptileConfidence Score: 5/5

The reviewed changes appear safe to merge.

Summary

This PR reduces Windows frame copies and adds Media Foundation texture encoding for fragmented recordings, with FFmpeg as the setup fallback.

The latest changes address all five previous findings:

  • Windows capture reuses mapped frames and limits readback waits.
  • Windows M4S screen recordings can encode captured textures directly.

No new actionable issues were found. The supplied previous threads were unnumbered, so they have no numbered ledger entries.

Reviews (2) · Last reviewed commit: "fix(recording): address Windows readback..." · Reviewed by Greptile

Read Windows capture back through a staging ring with DO_NOT_WAIT instead of a blocking Map plus a second full-frame copy, and let the fragmented/instant M4S muxer encode from the captured D3D11 texture through Media Foundation with the existing FFmpeg encoder as fallback.

Also removes the unused win_segmented muxers and the never-invoked DiskSpaceCallback plumbing.
@vercel

vercel Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

@Addin is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

- Keep the readback budget authoritative by default; CAP_READBACK_BLOCKING opts into an unbounded Map.
- Negotiate MFT types inside the candidate loop and handle WARP without VIDEO_SUPPORT.
- Validate harness --path values and count failures only for exercised paths.
@Addin
Addin marked this pull request as ready for review October 23, 2026 13:16
Comment thread crates/scap-direct3d/src/lib.rs
Comment thread crates/scap-ffmpeg/examples/cli.rs
Comment thread crates/enc-mediafoundation/src/video/h264.rs Outdated
Comment thread crates/enc-mediafoundation/src/video/h264.rs
Comment thread crates/scap-ffmpeg/examples/cli.rs Outdated
- Evict idle stale-size staging textures before reporting StagingBusy.
- Size the benchmark copy from each mapped frame and rebuild old_frame on resize.
- Require METransformDrainComplete and return errors on failed or timed-out drains.
- Count drained samples in the encoder health output counters.
- Keep benchmark runs alive across capture idle timeouts.
@Addin

Addin commented Oct 23, 2026

Copy link
Copy Markdown
Author

@greptileai review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant