Skip to content

fix: output lifetime - validate raw pushes, defer delete, guard share… - #3

Open
rustdesk wants to merge 2 commits into
mainfrom
fix-texture-lifetime
Open

fix: output lifetime - validate raw pushes, defer delete, guard share…#3
rustdesk wants to merge 2 commits into
mainfrom
fix-texture-lifetime

Conversation

@rustdesk

@rustdesk rustdesk commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

…d texture

  • unregisterTexture deleted D3D11Output on the platform thread while the raster thread could still fetch the surface descriptor or run the release callback, and while Rust's decode thread could still push through the raw output pointer (use-after-free; hung raster thread / black remote view). Pushes now resolve the pointer through a live-object registry and deletion is deferred via UnregisterTexture(id, callback).
  • EnsureTexture waits briefly while the engine is reading the shared texture instead of overwriting it mid-read (tearing); the single fixed shared handle is kept (Windows GPU memory leaks when changing handle of FlutterDesktopGpuSurfaceDescriptor flutter/flutter#154716).
  • New FlutterGpuTextureRendererPluginCApiGetConsumed(output) exposes how many frames the engine actually consumed, for black-output detection/fallback.

Summary by CodeRabbit

  • New Features

    • Added an API to report how many frames have been consumed by the Flutter engine.
    • Texture updates now safely validate output handles before processing.
  • Bug Fixes

    • Improved stability when outputs are removed while rendering or engine callbacks are still active.
    • Prevented texture replacement from occurring while active rendering is in progress.
    • Improved synchronization for concurrent texture updates and frame consumption.

…d texture

- unregisterTexture deleted D3D11Output on the platform thread while the
  raster thread could still fetch the surface descriptor or run the
  release callback, and while Rust's decode thread could still push
  through the raw output pointer (use-after-free; hung raster thread /
  black remote view). Pushes now resolve the pointer through a live-object
  registry and deletion is deferred via UnregisterTexture(id, callback).
- EnsureTexture waits briefly while the engine is reading the shared
  texture instead of overwriting it mid-read (tearing); the single fixed
  shared handle is kept (flutter/flutter#154716).
- New FlutterGpuTextureRendererPluginCApiGetConsumed(output) exposes how
  many frames the engine actually consumed, for black-output
  detection/fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 13, 2026 01:50
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@rustdesk, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 4 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a8160dae-39c3-4cc4-ac9e-826accd18f75

📥 Commits

Reviewing files that changed from the base of the PR and between 767bb9f and 208619e.

📒 Files selected for processing (2)
  • windows/d3d11_output.cpp
  • windows/include/flutter_gpu_texture_renderer/flutter_gpu_texture_renderer_plugin_c_api.h
📝 Walkthrough

Walkthrough

Changes

D3D11 output safety and frame tracking

Layer / File(s) Summary
Output lifecycle and frame state
windows/d3d11_output.cpp, windows/d3d11_output.h
Live outputs use synchronized tracking. Rendering and consumed-frame state use atomics. Texture replacement waits for active rendering, and exported accessors validate output handles.
Output unregister lifetime
windows/flutter_gpu_texture_renderer_plugin.cpp
Texture unregistration retains the output through callback completion.
C API texture and consumed-frame bridge
windows/flutter_gpu_texture_renderer_plugin_c_api.cpp, windows/include/flutter_gpu_texture_renderer/flutter_gpu_texture_renderer_plugin_c_api.h
Texture assignment uses validated output access. The C API exposes consumed-frame counts and returns zero for null or unknown handles.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Mergeability Score: 🟠 High · up to 767bb

Stale producer handles can be accepted after address reuse and send frames to the wrong texture output, producing incorrect or black rendering. The PR is not merge-ready until output identity is validated beyond the raw address.

Sequence Diagram(s)

sequenceDiagram
  participant FlutterEngine
  participant PluginCApi
  participant D3D11Output
  participant LiveOutputRegistry
  FlutterEngine->>PluginCApi: SetTexture(output, texture)
  PluginCApi->>LiveOutputRegistry: validate output
  LiveOutputRegistry->>D3D11Output: set texture
  D3D11Output-->>PluginCApi: complete texture update
  FlutterEngine->>PluginCApi: GetConsumed(output)
  PluginCApi->>LiveOutputRegistry: validate output
  LiveOutputRegistry->>D3D11Output: read consumed count
  D3D11Output-->>PluginCApi: return consumed count
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes: output lifetime validation, deferred deletion, and shared-texture protection.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-texture-lifetime

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses Windows texture output lifetime and synchronization issues in the Flutter GPU texture renderer plugin, aiming to prevent use-after-free during unregister, reduce tearing when updating shared textures, and expose engine-consumption metrics for black-output detection/fallback.

Changes:

  • Defers D3D11Output deletion until Flutter engine unregister completes (via UnregisterTexture(id, callback) with captured ownership).
  • Adds a live-object registry to validate raw pushes coming from a decode thread before dereferencing D3D11Output*.
  • Exposes a new C-API FlutterGpuTextureRendererPluginCApiGetConsumed(output) to report how many frames the engine has consumed.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
windows/include/flutter_gpu_texture_renderer/flutter_gpu_texture_renderer_plugin_c_api.h Adds stdint.h include and exports a new consumed-frames C API.
windows/flutter_gpu_texture_renderer_plugin.cpp Changes unregister flow to defer deletion until engine-side unregister callback.
windows/flutter_gpu_texture_renderer_plugin_c_api.cpp Routes pushes through validated helpers; adds consumed-frames getter.
windows/d3d11_output.h Adds consumed counter, makes rendering_ atomic, declares validated helper functions.
windows/d3d11_output.cpp Implements live-object registry, consumed counter increment, and validated push/consume helpers; adds brief wait to reduce tearing.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread windows/d3d11_output.cpp
Comment on lines +169 to +173
bool D3D11OutputSetTexture(void *output, void *texture) {
std::shared_lock<std::shared_mutex> lock(g_live_mutex);
if (g_live_outputs.find(output) == g_live_outputs.end())
return false;
return static_cast<D3D11Output *>(output)->SetTexture(texture);
Comment thread windows/d3d11_output.cpp
Comment on lines 62 to 67
D3D11Output::~D3D11Output() {
if (texture_id_)
texture_registrar_->UnregisterTexture(texture_id_);
// Unregistration happens in the plugin's unregisterTexture; here only make
// sure no push is still running on this object and no later push reaches it.
std::unique_lock<std::shared_mutex> lock(g_live_mutex);
g_live_outputs.erase(this);
}

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@windows/d3d11_output.cpp`:
- Around line 31-32: Replace the address-only entries managed by g_live_mutex
and g_live_outputs with non-reusable opaque handle tokens or generation-tagged
handles, and associate each token with its specific output instance. Update the
producer-handle validation and lookup flow around the g_live_outputs.find check
at lines 169–180 to reject handles from destroyed outputs, even when memory
addresses are reused, while preserving valid current-output frame delivery.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e2a2f0fc-2ded-4864-b6f2-0b0ca0dd2553

📥 Commits

Reviewing files that changed from the base of the PR and between 08a471b and 767bb9f.

📒 Files selected for processing (5)
  • windows/d3d11_output.cpp
  • windows/d3d11_output.h
  • windows/flutter_gpu_texture_renderer_plugin.cpp
  • windows/flutter_gpu_texture_renderer_plugin_c_api.cpp
  • windows/include/flutter_gpu_texture_renderer/flutter_gpu_texture_renderer_plugin_c_api.h

Comment thread windows/d3d11_output.cpp
Comment on lines +31 to +32
std::shared_mutex g_live_mutex;
std::unordered_set<void *> g_live_outputs;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Prevent stale handle aliasing.

Line 32 stores only an address. If an old output is destroyed and a new output reuses that address, g_live_outputs.find at Line 171 accepts a stale producer handle and sends its frame to the new output.

Use an opaque handle with a generation value, or retain non-reusable handle tokens until the producer releases them. The registry must validate output identity, not only the current address.

Also applies to: 169-180

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@windows/d3d11_output.cpp` around lines 31 - 32, Replace the address-only
entries managed by g_live_mutex and g_live_outputs with non-reusable opaque
handle tokens or generation-tagged handles, and associate each token with its
specific output instance. Update the producer-handle validation and lookup flow
around the g_live_outputs.find check at lines 169–180 to reject handles from
destroyed outputs, even when memory addresses are reused, while preserving valid
current-output frame delivery.

- set rendering_ only when a real surface descriptor is handed out: the
  engine bails before the release callback on a null handle, which left
  the flag stuck true and cost the full wait on every push until the
  first successful populate
- document GetConsumed honestly: it counts descriptor fetches and an EGL
  bind failure still advances it, so 0 means never composited, not
  rendered correctly

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants