Skip to content

perf(rendering): recycle export RGBA readbacks and cache NV12 bind groups per texture - #2851

Merged
richiemcilroy merged 4 commits into
mainfrom
perf/export-prores-pool-nv12-bindgroups
Oct 23, 2026
Merged

richiemcilroy merged 4 commits into
mainfrom
perf/export-prores-pool-nv12-bindgroups

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Two low-risk export allocation fixes in crates/rendering/src/frame_pipeline.rs. Output is unchanged. Identity checks and numbers are below.

RGBA readback buffers (ProRes MOV, GIF, software-adapter MP4). Each RGBA readback copied into a new Vec of padded_bytes_per_row * height + 24 bytes (about 33 MB at 4K) and dropped it after the encoder copied it. PipelinedGpuReadback now keeps a clone of up to 8 of the Arc<Vec<u8>> frames it hands out, and the next readback reuses one once every consumer has dropped it (Arc::try_unwrap). RenderedFrame keeps its shape, so the MOV encoder, GIF encoder and software NV12 conversion need no changes. Only export sessions (export_deadlines) recycle, so editor, thumbnail and screenshot sessions keep no extra memory. The pool grows only when every pooled frame is still in use, so it holds no more frames than the export already had in flight, and it drops with the session.

NV12 converter bind groups. The converter keyed its bind group on the address of the &wgpu::Texture it was given and rebuilt two identical bind groups plus a view whenever that changed. Colour grade, bokeh and masks swap the session texture an odd number of times per frame, so those exports rebuilt it every frame. It now caches one bind group per source texture identity, two entries, the same way RgbaToBgraSurfaceConverter::source_bind_group does. Entries are still cleared when the NV12 buffer is recreated. This runs on the macOS VideoToolbox surface path too, but only bind group creation changes there: same shader, same buffers, same copies.

Not in this PR: moving the ProRes swscale RGBA to YUVA444P10 conversion to the GPU.

Overlap with #2770. #2770 replaces Vec::with_capacity(active_bytes + 24) in PendingReadback::wait with a fallible allocate_readback. The two PRs conflict on that line. Whichever merges second should apply try_reserve_exact to the recycled Vec (it is cleared, so the capacity math is the same). PendingReadback::wait now takes the session's recycled frames, so the standalone PendingReadback in #2770's cleanup test needs &mut Default::default() as a second argument.

Tests: unit tests for reuse after release and for the pool bound, and a macOS GPU test that alternates two source textures through one converter. It checks every output against a fresh converter and checks that the cached bind groups stay the same after the first pair.

Measurements

Scratch A/B: ci/export-prores-pool-ab (this head plus counters) against ci/export-prores-pool-base (main plus the same counters). Synthetic testsrc2 recording, colour background, padding and rounding. The mask project adds one full-length sensitive mask. Counters: bytes copied and freshly allocated per RGBA readback, time from allocation through copy, NV12 bind groups created, and process peak RSS, CPU time and page faults.

Output identity (stream md5 and decoded framemd5, base vs head, plus a base vs base control)

  • Linux (llvmpipe): ProRes MOV and MP4, plain and mask, 1920x1080 and 1280x720. All 16 comparisons are identical, and the control is identical too.
  • macOS (Metal): ProRes MOV and the "optimize file size" libx264 MP4 are identical in stream md5 and framemd5 in every case. On the default VideoToolbox MP4 path, decoded framemd5 is identical in every case. The stream md5 differs base vs head, but it also differs base vs base, so VideoToolbox is not bit-reproducible run to run.
  • Windows (WARP): inconclusive. WARP output differs between two runs of the same base binary, so md5 comparison cannot separate this change from run-to-run variation there.

ProRes MOV, macOS Metal (mov-export-benchmark, 300 frames, median of 6 runs)

size variant fps readback copy µs/frame fresh alloc bytes/frame peak RSS MB CPU s/run
1920x1080 main 50.2 683 8,294,424 688 33.9
1920x1080 head 51.5 196 165,888 491 33.1
3840x2160 main 13.9 2,598 33,146,904 1,706 125.0
3840x2160 head 14.9 719 662,938 1,713 117.4

ProRes MOV, Linux llvmpipe (export-cli, 180 frames at 3840x2160, median of 8 interleaved runs)

project variant wall s readback copy ms/frame fresh alloc bytes/frame page faults peak RSS MB
plain main 21.35 12.17 33,177,624 868k 2,454
plain head 20.71 6.45 1,105,921 617k 2,434
mask main 25.46 9.24 33,177,624 623k 2,291
mask head 24.48 5.38 368,640 607k 2,334

ProRes MOV, Windows WARP (mov-export-benchmark, 300 frames, 3 runs each)

  • 1920x1080: readback copy 1.5 to 3.9 ms becomes 0.75 to 1.04 ms per frame. Fresh allocation per frame drops from 8.29 MB to 55 KB, and page faults per run from 1.65M to 1.05M.
  • 3840x2160: readback copy 7.5 to 10.5 ms becomes 3.7 to 4.0 ms per frame, and page faults per run drop from 6.33M to 3.92M.
  • WARP renders at 1.4 fps (1080p) and 0.4 fps (4K), so wall time is entirely rendering and run-to-run noise. The job reached its time limit before the Windows MP4 benchmarks.

NV12 bind groups, macOS Metal (VideoToolbox MP4, 1920x1080, 180 frames)

project variant bind groups created µs per conversion call
mask main 360 6.0
mask head 2 0.4
plain main 2 0.1
plain head 1 0.3

Export wall time with the mask did not measurably change (887 ms vs 881 ms). The saving is small CPU work per frame, as expected.

MP4 software NV12 path, Linux llvmpipe (1920x1080, 180 frames, median of 8 interleaved runs)

The result is neutral within noise. Wall time was 3,501 vs 3,646 ms (plain) and 4,923 vs 4,338 ms (mask). glibc already reuses a freed 8 MB chunk at this size, so pooling mostly matters at 4K and on macOS and Windows.

CI

The required checks pass, except macOS Engine verify. It fails on crash/recorder-kill/{inproc,oop}/static-screen (fragment-span), the known failure on main. Recording code is unchanged here.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

Summary

Reuses released RGBA export buffers and caches NV12 bind groups for two source textures.

  • Export readbacks reuse frame storage after encoders release it.
  • NV12 conversion reuses bindings for its two source textures.

Reviews (3) · Last reviewed commit: "fix(rendering): test reserve_readback di..." · Reviewed by Greptile

@blacksmith-sh

This comment has been minimized.

@richiemcilroy
richiemcilroy marked this pull request as ready for review October 22, 2026 06:22
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

# Conflicts:
#	crates/rendering/src/frame_pipeline.rs
@vercel

vercel Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cap-status Ready Ready Preview Oct 23, 2026 5:16pm UTC

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

Open in Capy

Comment thread crates/rendering/src/frame_pipeline.rs Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

Comment thread crates/rendering/src/frame_pipeline.rs Outdated
Comment on lines +1660 to +1662
let mut data = Vec::new();
reserve_readback(&mut data, active_bytes, frame_number)?;
Ok(data)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Unused helper breaks Clippy

Production readback now calls reserve_readback directly, leaving allocate_readback used only inside #[cfg(test)]. The helper still compiles in normal builds, so it triggers dead_code. The workspace Clippy job rejects that warning with -D warnings.

Add #[cfg(test)] to the helper, or remove it and test reserve_readback directly.

Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/rendering/src/frame_pipeline.rs
Line: 1660-1662

Comment:
**Unused helper breaks Clippy**

Production readback now calls `reserve_readback` directly, leaving `allocate_readback` used only inside `#[cfg(test)]`. The helper still compiles in normal builds, so it triggers `dead_code`. The workspace Clippy job rejects that warning with `-D warnings`.

Add `#[cfg(test)]` to the helper, or remove it and test `reserve_readback` directly.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

This branch was successfully deployed

2 active deployments
Preview – cap-web — beee4fc1 Deployed Oct 23, 2026 by vercel[bot]
Preview – cap-status — beee4fc1 Deployed Oct 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant