[API] Add client file selection on `IClientWindow` by spaingnv · Pull Request #674 · NVIDIA/flashdreams

spaingnv · 2026-09-30T21:19:26Z

Summary

Fixes #649: a UI loop calls request_file_selector(request_id, initial_path=None, *, accept=(), max_file_bytes=None, multiple=False), and the result comes back on a later tick as one SelectedFilesUserInputEvent carrying request_id, on the same input stream as keyboard and mouse.
The event has status and files (name + data bytes, no host path). status is ok, cancelled, busy, too_large, disallowed_type, or unavailable. Sizes are clamped to 32 MiB per file, 256 MiB and 256 files per set. Apps do not branch on --mode.
IClientWindow owns the rules and creates every event: every call gets exactly one event unless its session ends first; one picker at a time (a call while one is open is answered busy); results are checked against the request (types, sizes, count, base names); an internal id per picker so late results are dropped; unavailable if a picker is not shown within 30 s, and no deadline once it is shown. Session replacement withdraws an open picker without a result. Windows implement the platform hooks and the abstract _put_user_input_event; the IClientWindow docstring is the full lifecycle.
Native opens a Tk file dialog on a worker thread, so the window keeps presenting. WebRTC shows a non-modal "The app requested a file" prompt; Select… opens the browser's chooser, and the bytes are uploaded with a one-time token that only the connected page receives. MP4 and null answer unavailable.
Demo: imgui-ui-file-picker with Open file (A), Open file (B) and Open files, asking for .bin / .raw. Failed picks show in red with the status name.

Test plan

CPU:

uv run --no-sync pytest -m ci_cpu \
  flashdreams/test_v2/test_selected_files.py \
  flashdreams/test_v2/test_session_runner.py \
  flashdreams/test_v2/test_native_window_client_window.py \
  flashdreams/test_v2/test_webrtc_client_window.py \
  integrations_v2/imgui_ui_demo/imgui_ui_demo/tests/test_imgui_ui_demo.py

Native: Open file (A) → Tk dialog filtered to .bin / .raw. While it is open, A or B shows BUSY. under-32mb.bin → name (N bytes); oversize / wrong type / dismiss → red TOO_LARGE / DISALLOWED_TYPE / CANCELLED. Open files accepts several files. The window keeps presenting while the dialog is open.

uv sync --package flashdreams-imgui-ui-demo --inexact
uv run --no-sync flashdreams-run-v2 imgui-ui-file-picker --mode native-window

WebRTC: same buttons; prompt ("The app requested a file") → Select… → browser chooser (not the GPU host). Same statuses as native. Video keeps playing under the prompt.

uv sync --package flashdreams-imgui-ui-demo --inexact
uv run --no-sync flashdreams-run-v2 imgui-ui-file-picker --mode webrtc --host 127.0.0.1 --port 8080

MP4 / null: no interactive client; they answer unavailable, covered by the CPU tests above.

Apps request a file through the UI loop and read bytes from SelectedFilesUserInputEvent, so native, WebRTC, and MP4 share one contract instead of a host-path picker. Signed-off-by: Set Paing <spaing@nvidia.com>

copy-pr-bot · 2026-09-30T21:19:29Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

… session A reused request id (for example a second Open click while a picker is still open) should keep the original pick so the chosen file can still arrive. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-09-30T21:49:07Z

/ok to test fba1dc2

CI's ruff --select I hook reordered numpy after the imgui_ui_demo imports. Fold that in with the stale StepResult stub and the file-selector README line. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-09-30T22:16:51Z

/ok to test 07a23a2

greptile-apps · 2026-09-30T22:44:34Z

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with a non-blocking filename issue on Linux.

Findings

Selected filenames lose characters ▶

Summary

Adds client file selection with shared request rules and results delivered through the input stream.

UI loops can request client files and receive checked results as input events.
Native windows open a Tk file picker without stopping presentation.
WebRTC viewers choose files in their browser and return them to the app.
The ImGui demo shows separate results for single-file and multi-file picks.

Previous acknowledgments: spaingnv deferred old browser dialogs answering a replacement session because that requires handoff with a reused request ID; deferred the macOS Tk issue because native CUDA use does not run there in practice; and deferred overlapping desktop dialogs because they require session replacement with a dialog open. spaingnv also dismissed the earlier native replacement finding because reopening was unsupported at that time; Greptile accepted that explanation.

Diagram

sequenceDiagram
    participant App as UI loop
    participant Window as IClientWindow
    participant Picker as Native or browser picker
    App->>Window: Request file selection
    alt Picker already active
        Window-->>App: BUSY on a later tick
    else Slot available
        Window->>Picker: Show picker
        Picker-->>Window: Status and selected bytes
        Window->>Window: Check active ID and file limits
        Window-->>App: SelectedFilesUserInputEvent
    end

_{Reviews (30) · Last reviewed commit: "Answer busy file selections and check re..." · Reviewed by Greptile}

greptile-apps · 2026-09-30T22:46:15Z

Comments Outside Diff

These findings could not be posted inline.

Abandoned selectors survive session handoff flashdreams/flashdreams/runtime_v2/webrtc_client_window.py:141 ▶

When a session is replaced, open() clears buffered input but retains file-request IDs awaiting a result. If the previous browser never completed a picker, the new session's request using that ID is ignored. Unsent selectors also remain queued for a later peer. Resolve or invalidate pending selections when the session or peer changes.

…ploads Drop SelectedFile.path so apps only see name and bytes. WebRTC admits uploads for pending ids, completes cancel and failure, and opens the picker from a viewer click. Native picks off the UI thread with the same size cap. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-09-30T23:41:05Z

/ok to test 9ee2447

CI's ruff-format hook wrapped the pending-upload HTTPConflict and collapsed a one-line next() in the host-path omission test. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-01T16:35:12Z

/ok to test f99b69a

copy-pr-bot · 2026-10-01T16:35:16Z

/ok to test f99b69a

@spaingnv, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

Tensor/context parallel runtime (NVIDIA#619) landed on main and conflicts with this branch in the v2 loop, session runner, and docs. Signed-off-by: Set Paing <spaing@nvidia.com> # Conflicts: # flashdreams/flashdreams/api_v2/loop.py # flashdreams/flashdreams/runtime_v2/session_runner.py # flashdreams/test_v2/test_session_runner.py

Native and WebRTC both clamp max_bytes to 32 MiB and honor accept suffixes so a viewer cannot raise the cap. The UI loop only queues policy. Signed-off-by: Set Paing <spaing@nvidia.com>

Cancel, oversize, disallowed type, and unavailable are separate statuses. Windows share selected_file_policy_status so a too-large disallowed file is always disallowed_type. WebRTC confirms the request without freezing video; the demo overlay prints failures in red. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-01T22:59:03Z

/ok to test 7eaa532

copy-pr-bot · 2026-10-01T22:59:06Z

/ok to test 7eaa532

@spaingnv, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

A crashing OS chooser now reports unavailable instead of leaving the request unanswered. WebRTC stamps an internal session epoch so a POST started in the old session cannot become input for the next one.

spaingnv · 2026-10-01T23:36:31Z

Re: Greptile outside-diff (step() failure signaling on peer ranks): out of scope for this PR — that path is from main, not client file selection.

spaingnv · 2026-10-01T23:40:23Z

/ok to test 4d9bc78

The fake window recorded accept/max_bytes but was still typed as a 2-tuple, which ty rejected. Also apply the isort/format nits CI auto-fixed.

spaingnv · 2026-10-01T23:55:40Z

/ok to test a158c5c

Keep the file-picker branch current with packaging and encoder-import changes on main. Signed-off-by: Set Paing <spaing@nvidia.com>

…ntrol. Same id while a pick is active or queued is ignored so mash does not stack dialogs. Distinct ids wait in order. Apps reuse the control's id after the matching event; a new id on every click queues another dialog. Signed-off-by: Set Paing <spaing@nvidia.com>

Strip only the chooser's final newline so trailing spaces stay in a Linux filename. Do not forward key presses to the app while the browser prompt is open, including the Escape that cancels it. Note the deferred overlapping-dialog case on native session replacement. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-13T19:53:45Z

/ok to test 5352490

Move the file-selection rules into the session runner: one picker in flight, a 30 s deadline that stops once the window reports the picker opened, and dropping results for any other id. A request made while another picker is open completes as UNAVAILABLE; a repeat for the open id is ignored. Windows now only show a picker and report back. The native window uses Tk on every platform. The WebRTC server keeps only armed upload policy and its upload validation. Remove the file-selection gate, the presentation timer, generation tracking, and the browser queue. The file-picker demo shows one status line per button. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-13T23:10:03Z

/ok to test aed4a5f

…e-pick-649 Signed-off-by: Set Paing <spaing@nvidia.com> # Conflicts: # flashdreams/flashdreams/runtime_v2/native_window_client_window.py # flashdreams/test_v2/test_native_window_client_window.py

spaingnv · 2026-10-13T23:11:31Z

/ok to test cf7ca1e

…e-pick-649 Signed-off-by: Set Paing <spaing@nvidia.com> # Conflicts: # flashdreams/flashdreams/api_v2/loop.py # flashdreams/flashdreams/runtime_v2/session_runner.py

spaingnv · 2026-10-13T23:13:28Z

/ok to test cc5a619

…d check Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-13T23:56:05Z

/ok to test 6a4171e

Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-14T00:05:23Z

/ok to test 6d098da

Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-14T16:58:57Z

/ok to test 7ee3599

ArielG-NV

Discussed Offline. New Design Was Planned That Properly Assign Ownership and ensure all event creation routes through IClientWindow.

Notes on CompositeClientWindow design for other reviewers

class CompositeClientWindow{
 IClientWindow primaryWindow;
 InputSource[] secondaryInputSources;
 OutputSink[] secondaryOutputSinks;
}

This is so requests route to a primaryWindow accordingly

IClientWindow allows one picker at a time, times out a picker not shown within 30 seconds, and creates every result event. Windows only show the picker and report back. The session runner forwards requests and forgets the open picker when a session ends. IUILoop.request_file_selector queues a request; the result arrives as a SelectedFilesUserInputEvent. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv · 2026-10-15T01:25:45Z

/ok to test 938cb4d

Signed-off-by: Set Paing <spaing@nvidia.com>

greptile-apps · 2026-10-15T22:02:10Z

+
+def _base_name(name: str) -> str:
+    """Return ``name`` without any POSIX or Windows directory part."""
+    base = name.replace("\\", "/").rsplit("/", 1)[-1]


Selected filenames lose characters

_base_name treats every backslash as a directory separator, but backslashes are legal filename characters on Linux. The native picker already returns Path(path).name, so choosing part\one.bin now reports one.bin. Apps that display or save the chosen name lose part of it.

Preserve native base names and strip paths only where the source supplies them.

spaingnv · 2026-10-15T23:10:47Z

Superseded by #725, the same change as one commit rebased on current main for a clean history. Closing this one.

[API] Add client file selection on IClientWindow

317313a

Apps request a file through the UI loop and read bytes from SelectedFilesUserInputEvent, so native, WebRTC, and MP4 share one contract instead of a host-path picker. Signed-off-by: Set Paing <spaing@nvidia.com>

Ignore duplicate in-flight file-selection ids instead of aborting the…

fba1dc2

… session A reused request id (for example a second Open click while a picker is still open) should keep the original pick so the chosen file can still arrive. Signed-off-by: Set Paing <spaing@nvidia.com>

Fix CPU ruff isort and leftover review nits

07a23a2

CI's ruff --select I hook reordered numpy after the imgui_ui_demo imports. Fold that in with the stale StepResult stub and the file-selector README line. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv marked this pull request as ready for review September 30, 2026 22:39

greptile-apps Bot reviewed Sep 30, 2026

View reviewed changes

spaingnv marked this pull request as draft September 30, 2026 22:54

Apply ruff format to the file-selection WebRTC changes

f99b69a

CI's ruff-format hook wrapped the pending-upload HTTPConflict and collapsed a one-line next() in the host-path omission test. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv added 3 commits October 1, 2026 09:43

Let apps declare file-picker type and size; windows enforce the ceiling

d0c5262

Native and WebRTC both clamp max_bytes to 32 MiB and honor accept suffixes so a viewer cannot raise the cap. The UI loop only queues policy. Signed-off-by: Set Paing <spaing@nvidia.com>

spaingnv marked this pull request as ready for review October 1, 2026 22:59

greptile-apps Bot reviewed Oct 1, 2026

View reviewed changes

Comment thread flashdreams/flashdreams/runtime_v2/native_window_client_window.py Outdated

Comment thread flashdreams/flashdreams/runtime_v2/native_window_client_window.py Outdated

Comment thread flashdreams/flashdreams/runtime_v2/serving/webrtc_server.py Outdated

Complete failed native picks and drop stale WebRTC uploads on handoff.

4d9bc78

A crashing OS chooser now reports unavailable instead of leaving the request unanswered. WebRTC stamps an internal session epoch so a POST started in the old session cannot become input for the next one.

Fix CPU lint: file-selection request tuple type and ruff format.

a158c5c

The fake window recorded accept/max_bytes but was still typed as a 2-tuple, which ty rejected. Also apply the isort/format nits CI auto-fixed.

spaingnv added 2 commits October 2, 2026 10:32

Merge origin/main into client file selection

167e1c9

Keep the file-picker branch current with packaging and encoder-import changes on main. Signed-off-by: Set Paing <spaing@nvidia.com>

greptile-apps Bot reviewed Oct 13, 2026

View reviewed changes

Comment thread flashdreams/flashdreams/runtime_v2/native_window_client_window.py Outdated

Comment thread flashdreams/flashdreams/runtime_v2/native_window_client_window.py Outdated

Comment thread flashdreams/flashdreams/runtime_v2/serving/web/app.js

ArielG-NV reviewed Oct 13, 2026

View reviewed changes

Comment thread flashdreams/flashdreams/runtime_v2/native_window_client_window.py Outdated

ArielG-NV reviewed Oct 13, 2026

View reviewed changes

Comment thread flashdreams/flashdreams/runtime_v2/session_runner.py Outdated

Merge remote-tracking branch 'origin/main' into dev/spaing/client-fil…

cf7ca1e

…e-pick-649 Signed-off-by: Set Paing <spaing@nvidia.com> # Conflicts: # flashdreams/flashdreams/runtime_v2/native_window_client_window.py # flashdreams/test_v2/test_native_window_client_window.py

Merge remote-tracking branch 'origin/main' into dev/spaing/client-fil…

cc5a619

…e-pick-649 Signed-off-by: Set Paing <spaing@nvidia.com> # Conflicts: # flashdreams/flashdreams/api_v2/loop.py # flashdreams/flashdreams/runtime_v2/session_runner.py

Keep _apply_window_requests unchanged and drop the per-tick request_i…

6a4171e

…d check Signed-off-by: Set Paing <spaing@nvidia.com>

greptile-apps Bot reviewed Oct 14, 2026

View reviewed changes

Comment thread flashdreams/flashdreams/api_v2/README.md Outdated

Drop the removed per-tick request_id check from the API README

6d098da

Signed-off-by: Set Paing <spaing@nvidia.com>

Document why request_id is app-supplied and must not be shared

7ee3599

Signed-off-by: Set Paing <spaing@nvidia.com>

ArielG-NV requested changes Oct 15, 2026

View reviewed changes

greptile-apps Bot reviewed Oct 15, 2026

View reviewed changes

Comment thread flashdreams/flashdreams/runtime_v2/mp4_client_window.py

spaingnv added 4 commits October 15, 2026 11:42

Keep one file selection per UI tick

35f355d

Signed-off-by: Set Paing <spaing@nvidia.com>

Drop stale file selector id comment

290d81d

Signed-off-by: Set Paing <spaing@nvidia.com>

Harden file selection and document its contract

be952ac

Signed-off-by: Set Paing <spaing@nvidia.com>

Answer busy file selections and check results in IClientWindow

4d0eded

Signed-off-by: Set Paing <spaing@nvidia.com>

greptile-apps Bot reviewed Oct 15, 2026

View reviewed changes

spaingnv mentioned this pull request Oct 15, 2026

[API] Add client file selection on IClientWindow #725

Open

2 of 4 tasks

spaingnv closed this Oct 15, 2026

Conversation

spaingnv commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

copy-pr-bot Bot commented Sep 30, 2026

Uh oh!

spaingnv commented Sep 30, 2026

Uh oh!

spaingnv commented Sep 30, 2026

Uh oh!

greptile-apps Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comments Outside Diff

Uh oh!

spaingnv commented Sep 30, 2026

Uh oh!

spaingnv commented Oct 1, 2026

Uh oh!

copy-pr-bot Bot commented Oct 1, 2026

Uh oh!

spaingnv commented Oct 1, 2026

Uh oh!

copy-pr-bot Bot commented Oct 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

spaingnv commented Oct 1, 2026

Uh oh!

spaingnv commented Oct 1, 2026

Uh oh!

spaingnv commented Oct 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

spaingnv commented Oct 13, 2026

Uh oh!

Uh oh!

Uh oh!

spaingnv commented Oct 13, 2026

Uh oh!

spaingnv commented Oct 13, 2026

Uh oh!

spaingnv commented Oct 13, 2026

Uh oh!

spaingnv commented Oct 13, 2026

Uh oh!

Uh oh!

spaingnv commented Oct 14, 2026

Uh oh!

spaingnv commented Oct 14, 2026

Uh oh!

ArielG-NV left a comment

Choose a reason for hiding this comment

Uh oh!

spaingnv commented Oct 15, 2026

Uh oh!

Uh oh!

greptile-apps Bot Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

spaingnv commented Oct 15, 2026

spaingnv commented Sep 30, 2026 •

edited

Loading

greptile-apps Bot commented Sep 30, 2026 •

edited

Loading

greptile-apps Bot commented Sep 30, 2026 •

edited

Loading