Skip to content

perf(gpui): stream the editor preview into one texture (default on Linux) - #2854

Draft
richiemcilroy wants to merge 4 commits into
perf/gpui-streaming-imagefrom
perf/gpui-streaming-editor-preview
Draft

richiemcilroy wants to merge 4 commits into
perf/gpui-streaming-imagefrom
perf/gpui-streaming-editor-preview

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Stacked on #2852 (phase 2 of the gpui streaming-image design). Moves the editor preview's RGBA path to a StreamingImage. It is on by default on Linux, and opt-in on Windows and for macOS's RGBA fallback.

Change

  • EditorPreviewFrame::Stream(StreamingImage). The editor pump keeps one Rgba8 stream per editor instance.
  • When an RGBA frame arrives, frame_arrived's update closure hands it to the stream on the main thread. This happens after the holds_instance check, so a stale pump can't change what the preview shows.
    • The renderer's padded RGBA rows are used as they are: no un-padding, no channel swap, and no background hop.
    • The buffer is moved rather than copied when the pump owns the only Arc.
  • Every RGBA frame was going through the RenderImage path below. With the stream on, these are gone:
    • the CPU unpad + BGRA conversion on the background executor (frame_image)
    • a full-size atlas texture created and destroyed per frame (wgpu), plus its swizzle copy
    • drop_image of the previous frame: the window's dead-stream sweep releases the texture now
  • Conversion time still feeds convert_nanos / convert_samples, so the bench counters stay comparable.

Kill switch

CAP_GPUI_STREAMING_IMAGE:

  • 0: back to per-frame RenderImages
  • 1: opts Windows and macOS's RGBA fallback in
  • unset: on for Linux only

Windows is off by default because no CI runner could run the in-app A/B there (see below).

Tests

  • Frames stream with their padded stride and exact bytes. An owned buffer is moved (same pointer). A shared buffer is copied.
  • Misshapen frames are rejected and the preview stays as it was.
  • Kill-switch parsing.

In-app editor A/B

CI scratch branch ci/streaming-editor-ab, run 39670701971.

  • Fixture: a 40 s 1080p studio .cap with mic, generated with cap_verify::fixture.
  • Command: CAP_GPUI_AUTO_EDITOR=<fixture> CAP_GPUI_AUTO_TIMELINE_BENCH=1 CAP_GPUI_MUTE_AUDIO=1.
  • Runs: one binary per backend, CAP_GPUI_STREAMING_IMAGE=0/1 interleaved (order alternating per round), 3 rounds, medians.
  • Columns:
    • draw = timed_draw (paint) per frame
    • convert = PumpStats convert avg
    • CPU = process and main-thread CPU over the 300 measured frames
    • presented = PumpStats.presented
  • Backends: gpu_specs confirms the passes ran on lavapipe Vulkan and llvmpipe GL 4.5.

Linux lavapipe (Vulkan)

scenario draw p50 µs (off → on) draw p95 µs convert avg µs process CPU % main-thread CPU % frames presented runs
tick 4687 → 4694 (+0%) 6644 → 7086 (+7%) 633.9 → 7.3 (-99%) 598.4 → 587.9 (-2%) 23.0 → 21.9 (-5%) 625 → 699 3/3
tick-zoomed-out 4207 → 3414 (-19%) 6323 → 5307 (-16%) 629.5 → 7.7 (-99%) 563.7 → 582.4 (+3%) 23.1 → 22.0 (-5%) 616 → 675 3/3
hover 4428 → 3291 (-26%) 6104 → 5119 (-16%) 1018.4 → 12.5 (-99%) 554.7 → 550.1 (-1%) 24.3 → 21.3 (-12%) 299 → 300 3/3
follow 6255 → 4652 (-26%) 9067 → 7651 (-16%) 786.2 → 6.2 (-99%) 570.4 → 587.4 (+3%) 26.2 → 24.5 (-6%) 659 → 676 3/3
clean 1467 → 1589 (+8%) 1923 → 2565 (+33%) 0.0 → 0.0 409.1 → 401.9 (-2%) 10.8 → 11.7 (+8%) 0 → 0 3/3

Linux llvmpipe (GL)

scenario draw p50 µs (off → on) draw p95 µs convert avg µs process CPU % main-thread CPU % frames presented runs
tick 4761 → 3202 (-33%) 8103 → 5013 (-38%) 774.9 → 4.9 (-99%) 587.3 → 577.9 (-2%) 29.4 → 23.8 (-19%) 741 → 648 3/3
tick-zoomed-out 4673 → 3557 (-24%) 8345 → 5901 (-29%) 842.0 → 5.3 (-99%) 586.3 → 590.6 (+1%) 30.3 → 24.7 (-18%) 722 → 745 3/3
hover 4430 → 4068 (-8%) 6309 → 5986 (-5%) 866.8 → 8.9 (-99%) 558.2 → 554.8 (-1%) 31.0 → 29.0 (-6%) 300 → 299 3/3
follow 5893 → 5471 (-7%) 9038 → 8397 (-7%) 790.6 → 5.4 (-99%) 587.2 → 597.3 (+2%) 32.2 → 28.3 (-12%) 671 → 814 3/3
clean 1399 → 1198 (-14%) 1854 → 1614 (-13%) 0.0 → 0.0 407.2 → 391.7 (-4%) 15.8 → 13.5 (-15%) 0 → 0 3/3
  • clean has no preview frames; its differences are noise.
  • Frames presented vary about ±15% between runs of the same mode: tick ranged 572–753 with streaming off and 579–758 with it on. No change there is beyond noise.

An earlier version wrote wgpu stream frames during paint. On GL that made draw p50 12–24% worse, while main-thread CPU was already lower. #2852 now defers that write to before_frame, where the atlas uploads happen, and the tables above are from after that change.

Windows

The in-app A/B could not run on any available CI runner:

  • blacksmith-8vcpu-windows-2025: jobs run without an interactive desktop, so gpui cannot create its DirectComposition swap chain (DXGI 0x887A0022, DWM access denied).
  • Hosted windows-2025: its VS 18 libclang breaks the ffmpeg-sys bindgen build.
  • Hosted windows-2022: cap-rendering's DX12 device on WARP is removed on the first editor frame in both modes (0x887A0005).

The Windows evidence is therefore the D3D11 WARP atlas micro-benchmark in #2852: 1.2–2.1x faster, flat at 1080p, 0 large allocations per frame. That's why Windows is opt-in until the A/B runs on a real Windows machine.

Atlas micro-benchmark

See #2852. On Linux, wgpu on lavapipe Vulkan / llvmpipe GL makes the atlas step 1.4–3.8x faster at p50. Textures created per 600 frames drop from 600 to 1, and large allocations per frame drop from 2 to 0.

@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cap-status Ready Ready Preview Oct 21, 2026 9:23pm UTC

@blacksmith-sh

blacksmith-sh Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
crash/recorder-kill/inproc/static-screen/static-screen View Logs
crash/recorder-kill/oop/static-screen/static-screen View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@richiemcilroy richiemcilroy changed the title perf(gpui): stream the Windows/Linux editor preview into one texture perf(gpui): stream the editor preview into one texture (default on Linux) Oct 21, 2026

This branch was successfully deployed

2 active deployments
Preview – cap-web — 4f06aace Deployed Oct 21, 2026 by vercel[bot]
Preview – cap-status — 4f06aace Deployed Oct 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant