Skip to content

fix(recording): capture the window itself on Windows, not a crop of the display - #2301

Open
Stephen-Psaradellis wants to merge 4 commits into
CapSoftware:mainfrom
Stephen-Psaradellis:fix/windows-window-capture-item
Open

Stephen-Psaradellis wants to merge 4 commits into
CapSoftware:mainfrom
Stephen-Psaradellis:fix/windows-window-capture-item

Conversation

@Stephen-Psaradellis

@Stephen-Psaradellis Stephen-Psaradellis commented Sep 16, 2026 •

Copy link
Copy Markdown

Fixes #2151 in CapSoftware/Cap.

The bug. On Windows, a window recording never captures the window. create_d3d_capturer builds its GraphicsCaptureItem from the display and crops it to the window's bounds, so the recording is a rectangle of the screen rather than the window: anything dragged over the window is recorded with it, and switching virtual desktops records whatever now occupies that rectangle. WindowImpl::try_as_capture_item (CreateForWindow) already existed in scap-targets and had no caller.

The change.

  1. ScreenCaptureTarget::Window now carries its window id through the capture config, and the Windows capturer builds the item from the window. The crop that carved the window out of the display is dropped for window targets — WGC already hands back just that window, and the crop additionally pinned the recording to wherever the window started out.
  2. Using a window item makes the capture item resizable for the first time, which exposed a second bug in scap-direct3d: the frame pool is created at the item's size and never recreated, so after a resize each frame reports the new ContentSize while the pool's surfaces are still the old size, and every later frame reads back black. The pool is now recreated when the content size changes. The crop path also skips frames whose crop box no longer fits the source, instead of copying an out-of-range region and vending a stale surface.

macOS is untouched: it builds its content filter from the display and crops, and SCContentFilter's window path is a separate change.

How it was tested. On Windows 11 (24H2, Rust 1.88.0), with a blue window under a red always-on-top window covering part of it:

capture red pixels (the window on top) blue pixels (the window itself)
display item cropped to the window (before) 60000 0
the window's own capture item (after) 0 439796

and across a resize from 786x593 to 1086x693 while recording, sampling every frame: before the frame-pool change every frame after the resize was empty (red: 0, blue: 0 at the new size); after it the frames carry the window's contents (blue: 716496).

Both are pinned by new tests in crates/scap-direct3d/tests/window_capture.rs, which drive real windows from a fixture process the way crates/scap-targets/tests/windows_targets.rs does (Window::list() skips the calling process, so the windows have to live in another one). frames_survive_the_capture_item_resizing fails on the unpatched library with exactly the black frame above, and passes with it. Two unit tests in windows.rs pin that a window target drops its crop and a display target keeps it.

Also run: cargo test -p scap-direct3d (7 passed), cargo test -p cap-recording --lib (552 passed, 0 failed, 4 ignored), cargo clippy -p scap-direct3d -p cap-recording --all-features clean, cargo fmt --all --check clean.

Disclosure. An AI system running an autonomous software project found the issue, wrote the fix and the tests, and wrote this description; I'm the account holder submitting it and I have reviewed the patch. Take it, change it or drop it as you see fit.

This fix is offered at a fixed $100, the floor I price fixes at (CapSoftware's completed Algora bounties run a $70 median and $550 largest). If you want to pay for it, comment /bounty $100 on the issue before merging and Algora pays it on merge. If you would rather not pay, merge it anyway - the code is yours either way.

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with the previous minimized-window setup failure fully addressed and no new actionable defects identified.

Findings

  1. P1 Minimized Window Fails Setup ▶
Fix with agent prompt
### Issue 1
crates/recording/src/sources/screen_capture/windows.rs:535-540
If the user minimizes the selected window before capture setup reaches this code, `Window::from_id` cannot find it because that lookup uses `Window::list()`, whose enumeration excludes minimized windows. Capturer creation then returns “Window not found,” so the recording fails to start even though the selected window still exists. Resolve the stored HWND directly instead of requiring the target to remain eligible for enumeration.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR changes Windows recording to capture a selected window through its own GraphicsCaptureItem rather than cropping the display. It also recreates the Direct3D frame pool when the capture item is resized, rejects invalid crop regions, and adds Windows integration tests for occlusion and resizing.

  • Carries the selected window ID into Windows capture setup.
  • Resolves previously selected windows directly so minimization does not prevent setup.
  • Preserves display cropping while removing it for native window capture.
  • Recreates frame-pool surfaces when the captured item changes size.
  • Adds regression coverage for window occlusion and resize behavior.

Reviews (3) · Last reviewed commit: "fix(recording): resolve the window handl..." · Reviewed by Greptile

@richiemcilroy

Copy link
Copy Markdown
Member

Thanks for the detailed write-up. Greptile hasn't posted a score yet, so please trigger a review (comment @greptileai) and address anything it raises. We need Greptile 5/5 and green CI before we review, and we'll want to verify window resizing mid-recording on Windows.

Comment on lines +535 to +537
let capture_item = match params.window_id {
Some(window_id) => Window::from_id(window_id)
.ok_or_else(|| anyhow!("Window not found for ID: {:?}", window_id))?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Minimized Window Fails Setup

If the user minimizes the selected window before capture setup reaches this code, Window::from_id cannot find it because that lookup uses Window::list(), whose enumeration excludes minimized windows. Capturer creation then returns “Window not found,” so the recording fails to start even though the selected window still exists. Resolve the stored HWND directly instead of requiring the target to remain eligible for enumeration.

Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/recording/src/sources/screen_capture/windows.rs
Line: 535-537

Comment:
**Minimized Window Fails Setup**

If the user minimizes the selected window before capture setup reaches this code, `Window::from_id` cannot find it because that lookup uses `Window::list()`, whose enumeration excludes minimized windows. Capturer creation then returns “Window not found,” so the recording fails to start even though the selected window still exists. Resolve the stored HWND directly instead of requiring the target to remain eligible for enumeration.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Stephen-Psaradellis added a commit to Stephen-Psaradellis/Cap that referenced this pull request Oct 7, 2026
…get still records

Window::from_id searches Window::list(), and the Windows enumeration
filter skips minimized windows (is_window_valid_for_enumeration rejects
IsIconic). A window minimized between selection and capture setup was
therefore reported as "Window not found" and the recording failed to
start, even though the window still existed and was capturable.

Add WindowImpl::from_id_direct, which rebuilds the HWND from the id (on
Windows a WindowId is the HWND) and validates it with IsWindow, and use
it at the capture-item call site. Enumeration eligibility is the wrong
question once the user has already picked the target; liveness is the
right one.

Reported by Greptile on CapSoftware#2301.
@Stephen-Psaradellis

Copy link
Copy Markdown
Author

Thanks — Greptile has now run and the Greptile Review check is green. It scored 4/5 and raised one P1, which was a real bug, so thank you for pushing for the review rather than waving it through.

The P1, confirmed. Window::from_id is Self::list().into_iter().find(...) (scap-targets/src/lib.rs), and the Windows enumeration filter rejects minimized windows outright — is_window_valid_for_enumeration returns false on IsIconic(hwnd) (scap-targets/src/platform/win.rs). So a window minimized between being selected and capture setup resolving its id came back as None, and the recording failed to start with "Window not found" even though the window still existed and was perfectly capturable. Greptile read that correctly.

Fixed in 232e1b5. On Windows a WindowId is the HWND (WindowIdImpl(self.0.0 as u64)), so the id can be resolved straight back to its handle without enumerating. Added WindowImpl::from_id_direct, which rebuilds the HWND and validates it with IsWindow, exposed it as Window::from_id_direct (#[cfg(windows)]), and used it at the capture-item call site. Enumeration eligibility is the wrong question once the user has already picked a target — liveness is the right one. from_id is unchanged, so nothing else shifts behaviour.

Window resizing mid-recording. This is in the PR description, but to put it where you are looking: a resize from 786x593 to 1086x693 while recording, sampling every frame. Before the frame-pool change every frame after the resize came back empty at the new size (red: 0, blue: 0); after it they carry the window's contents (blue: 716496). The mechanism is two-part — scap-direct3d recreates the frame pool when the content size changes, and WindowsFrameScaler::scale_frame rebuilds its swscale context whenever the incoming frame dimensions differ from the last one, so the output stays pinned to the encoder's fixed target size while the window is free to change. Happy to re-run that on the current head, or to run a specific sequence if you have one in mind.

On CI: the only red check is Vercel, reporting "Authorization required to deploy" — that one needs authorization on your side, I can't clear it from here.

One caveat I would rather state than have you discover: I verified cargo check -p scap-targets passes with this change, but I could not build cap-recording locally — ffmpeg-sys-next wants pkg-config and the FFmpeg dev libraries, which this machine does not have. The call-site change is one line against a signature the compiled crate confirms, but CI is the real check on it, not me.

Disclosure: I use an AI system to do this work; I review and am responsible for everything submitted under my name.

@Stephen-Psaradellis

Copy link
Copy Markdown
Author

@greptileai

@richiemcilroy

Copy link
Copy Markdown
Member

@Stephen-Psaradellis thanks for digging into this, the before/after pixel test is really convincing. One blocker before we can take it: with the crop gone, the output size is still derived from the even-floored crop bounds (crates/recording/src/sources/screen_capture/mod.rs:499), but WGC now hands back frames at the window item's own size. Whenever those differ (odd window width/height, item size vs window rect, or max_capture_size constraining the output) every frame drops into the CPU readback + sws_scale fallback at windows.rs:593 for the entire recording, which is a big CPU and quality hit. Could you size the output from the window capture item (even-rounded) or crop the window frame to the even output size on the GPU, and add a test that a window recording never takes the scaler path? Two more things we need before merge: the Studio cursor overlay is still normalised against the window's start bounds, so it ends up in the wrong place once the window moves, and the frame pool Recreate in scap-direct3d now also changes what Display recordings do on a resolution change, so please rebase on main and include a real Windows run covering window move, resize mid-recording and a display resolution change for both Studio and Instant.

…capture item

Window targets capture the window's own GraphicsCaptureItem instead of a crop
of the display. The output is sized from that item floored to even, and odd
items are trimmed on the GPU (scap-direct3d Settings::even_dimensions), so a
window recording's frames match the encoder size and never take the CPU
scaler. The Studio cursor is placed against the window's current bounds, so it
follows a moved window. Frame pool is recreated when the item resizes.
@Stephen-Psaradellis
Stephen-Psaradellis force-pushed the fix/windows-window-capture-item branch from 232e1b5 to 7971485 Compare October 20, 2026 23:42
@vercel

vercel Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

@Stephen-Psaradellis is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

@Stephen-Psaradellis

Copy link
Copy Markdown
Author

@richiemcilroy thanks, all three were real. Rebased on main (57369c4) and pushed as one commit, 7971485.

1. Output size / scaler path. A window recording is now sized from the window's own capture item, floored to even (window_capture_size in screen_capture/windows.rs), not from the crop bounds. The odd row or column is trimmed on the GPU: scap_direct3d::Settings has a new even_dimensions flag that copies an odd-sized frame into an even-sized texture (CopySubresourceRegion, texture reallocated only when the size changes), so frames come out at exactly the output size. Your example was right. On the test fixture the window rect is 800x600, but the capture item is 786x593, and the old sizing sent every frame to the scaler.

Tests:

  • scap-direct3d/tests/window_capture.rs::window_frames_are_trimmed_to_even_dimensions_on_the_gpu captures a real window whose item is 786x593 and resizes it by 1px twice so each axis is odd at some point. Every frame is even, and the first frame is exactly the floored item size (frames seen: 786x592, 786x594, 788x594).
  • Unit test for the sizing (a_window_recording_is_sized_to_its_capture_item_floored_to_even).

max_capture_size: on Windows it is only set for Studio Compatibility with the camera on (camera_active_max_capture_size). In that case Display recordings already go through the CPU scaler on every frame, before and after this PR. Moving that downscale to the GPU for both would be a reasonable follow-up, but it is a separate change, so I left it out here. Happy to do it if you want it in this PR.

2. Studio cursor. CursorCaptureTarget.window is now carried on Windows as well as Linux. Each tick the cursor recorder re-reads the window's bounds (windows_window_crop) and normalises against where the window is now. A change in the window's bounds also counts as a cursor change, so the overlay is re-placed even when the mouse is still. Unit test: the_cursor_crop_follows_the_window_when_it_moves.

3. Real Windows run (Windows 11, 3440x1440 @ 165 Hz, the cap CLI built from this branch, cap record --window <id> --mode studio|instant --duration 10). The fixture window was moved at ~3 s and resized to an odd 1101x701 at ~6 s, with the mouse parked at the window's centre before and after each change:

Studio Instant
configured output 786x592 786x592
output file display.mp4 786x592, 529 frames output.mp4 786x592, 273 frames
scaler log line once, at the resize (frame 1086x694 into 786x592) once, at the resize
through the move direct, no scaling direct, no scaling

Studio cursor.json (time ms, x, y): (882, 0.5, 0.506) at the start, (2936, 0.246, 0.337) when the window moves out from under the still mouse, (3439, 0.5, 0.506) with the mouse at the moved window's centre, then (6426, 0.5, 0.504) at the centre of the resized window. Before this change the post-move sample read (0.754, 0.675), which is the window's starting bounds.

A Display recording on the same build (--screen, Studio, 4 s) gives 3440x1440 with no scaler line.

Display resolution change: I did not run this one. Changing the display mode on this machine is not something I can do in this setup, so I have no recording to show for it and won't claim one. What changed for Displays is limited to the frame pool. Before, after a resolution change the pool kept vending surfaces at the old size, and the frame's ContentSize no longer described its texture. Now the pool is Recreated at the new size and that one frame is skipped. After that, Display frames reach the same expected_width != frame_width scaler branch as before, just with the right pixels. frames_survive_the_capture_item_resizing covers the recreate on a real capture item. If you'd rather the Display path stay exactly as it was, I can gate the Recreate on window targets only.

@richiemcilroy

Copy link
Copy Markdown
Member

@Stephen-Psaradellis thanks for the thorough follow-up, the capture item sizing with the GPU even trim and the cursor following the moved window both look right, and the Studio/Instant run with the move and odd resize is exactly what we needed. One thing left before we can merge: the frame pool Recreate in the FrameArrived handler (crates/scap-direct3d/src/lib.rs, the pool_size block) and the new early return when the crop box falls outside the frame both apply to Display and area recordings as well, and without a display resolution change run we can't confirm those paths behave the same as today (an area recording whose crop no longer fits would now stop receiving frames instead of continuing). Could you take the option you offered and only enable the Recreate (and the crop bounds skip) for window targets, e.g. via a Settings flag that windows.rs sets alongside even_dimensions when self.config.window.is_some(), so Display and area recordings stay exactly as they are on main? Display resolution handling can go in a separate PR with its own on-device run. Thanks!

Recreating the frame pool on a size change, and skipping frames a crop box no longer fits inside, now sit behind Settings::follow_item_size. windows.rs sets it alongside even_dimensions for window targets only, so display and area recordings behave exactly as on main.
@Stephen-Psaradellis

Copy link
Copy Markdown
Author

Done in fd8daaa. Both behaviours now sit behind a new Settings::follow_item_size flag, off by default:

  • the pool_size / Recreate block in FrameArrived only runs when the flag is set;
  • the early return when the crop box falls outside the frame only applies when the flag is set, so an area recording keeps receiving frames exactly as it does on main.

windows.rs sets follow_item_size: self.config.window.is_some() next to even_dimensions, so only window targets get it. Display and area recordings build Settings the same way as on main and take the same path through the handler.

frames_survive_the_capture_item_resizing now opts in with follow_item_size: true, since that is the window-target behaviour it covers. cargo test -p scap-direct3d --test window_capture passes (4/4) on this machine, and cargo clippy -p cap-recording -p scap-direct3d --tests is clean for the touched files.

I'll leave display resolution handling for a separate PR with its own on-device run, as you suggested.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Window Recording Bug On Windows

2 participants