Skip to content

fix(web): improve native settings, session history and workbar interactions - #598

Open
testikun wants to merge 20 commits into
openpi-dev:mainfrom
testikun:codex/web-ux-597
Open

testikun wants to merge 20 commits into
openpi-dev:mainfrom
testikun:codex/web-ux-597

Conversation

@testikun

@testikun testikun commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator

Problem

Related to #597; follows merged PR #561. Daily Web usage exposed missing older user messages, stale background execution, lost or duplicated pending prompts, slow workbar panels, pasted-image races, unclear settings feedback, and no complete-turn elapsed time. The embedded browser was blurry and queued input; Git review depended on an optional baseline that failed on large untracked workspaces.

Value

The workbench preserves drafts and Session identity across asynchronous operations, makes native configuration editable, and keeps visible feedback tied to actual Pi execution and saved results. Hidden panels do less work, while opening them refreshes the relevant content promptly. All usage-test rounds remain in this one PR; #597 stays open for continued iteration.

Approach

  • Page older native Session entries without raising the 250-entry/2 MiB response limit. Preserve entry ancestry and reading anchors, cap the browser reading window, and keep oversized entries reachable with explicit truncation. Improve response typography, list numbering and file-link display while retaining the execution-process design.
  • Project execution for the exact selected Session separately from the current input controller. Reuse retained Pi managers, report bounded background progress, and preserve cancellation/timing ownership across Session switches. Reading a Session never acquires input control; interactive controls require the current native file identity, including when copied JSONL files share an ID.
  • Preserve same-view pending messages through controller changes, require actual admission before display merging, and remember UI/native-entry associations so old inputs do not reappear after the snapshot window moves. Native queue counts prevent premature merging; these are display associations, not invented SDK delivery or disk receipts. Active drafts expose Send and Stop separately; observer mode hides another controller's model and stale receipts.
  • Keep side-conversation responses scoped to navigation and drafts, avoid terminal connection/restart focus theft, and clear outer artifact state when a preview closes due to context changes. External controller changes preserve the mounted history reader while input stays disabled until its native identity is confirmed. Report Chromium process signals and launch errors promptly; Linux CI uses the runner's installed Chrome with its native sandbox profile.
  • Preserve Pi ownership. First-use controls prepare a real Session without losing its draft; headless startup resolves unused search binaries lazily. Package-owned preferences, skills, plugins and agent-role choices still use /openpi-setup. Native models and write-only API keys use Pi's existing model/configuration and login APIs.
  • Make settings useful on desktop and narrow screens: model-tree selection edits the corresponding definition, unsaved drafts survive navigation, conflicts reload authoritative values, and late save receipts remain scoped to their provider. Add labeled mobile navigation, keyboard tabs, default-size resets and busy feedback. Show original setup operations and their results instead of hiding the episode. New setup requests explicitly distinguish their episode from previous completion notices.
  • Preserve native mixed text/image paste, caret and undo behavior, import images in a bounded queue, and transfer pending imports only with the exact Session-creation receipt. Keep terminal queues, file references and other asynchronous work scoped and cancellable.
  • Keep one Git summary read in flight, reduce hidden-panel polling, avoid unchanged diff reads and focus theft, and promote a deferred refresh when Git becomes visible. Default to native unstaged/staged/branch scopes, with quota-limited Session baselines as an optional view. External edited files still require explicit single-file authorization.
  • Discover native rename pairs before limiting a file's patch to literal paths. Keep raw metadata and patch in the same detail invocation, and include complete Git object/mode/path identities in summary revisions so index-only updates refresh an already-open diff even when worktree timestamps and line counts stay unchanged.
  • Preview authorized PNG/JPEG/GIF/WebP files with byte validation and bounded downloads. Reuse identical content, release images on cancellation/revocation, keep long Session titles compact, and place file identity details below the content.
  • Reuse Chromium/CDP with DPR-aware PNG screencast, newest-frame backpressure, one decoder and coalesced input. Preserve local HTTP navigation, native editing shortcuts, mouse-button identity and pointer release outside the pane. Late state restoration and navigation receipts preserve newer address drafts; real-browser assertions distinguish native text drag-and-drop from ordinary pointer release.
  • Wait for the native browser process and inherited stderr to close before deleting its profile, with bounded cleanup and retryable Session ownership. Cleanup errors preserve the original startup failure. Startup failures retain at most 4 KiB of stderr plus stage/exit evidence in host diagnostics; raw stderr stays out of the HTTP error and is released after successful startup.
  • Add source-verified native setup commands and Web-panel shortcuts to /, with reasons for unavailable commands. Display running and settled elapsed time from Pi's actual start/settlement events; persist optional display evidence in the native Session without adding model context or changing terminal outcomes.
  • Preserve main's structured questions, plan cards, native command feedback, configuration recovery and workspace identity checks. Plan writes carry the native Session path and ignore stale receipts; /usage uses native feedback. Keep setup requests visible, distinguish React keys from native history IDs, reconcile legacy live messages one-to-one, and keep Send reachable when the plan bar appears in short viewports.

Sources, reproductions, ablation and boundaries: native interactions, round 3, round 4, round 5, history and lifecycle. Pi Web and Maka are source-scoped references, not embedded products or claims of full parity.

The review follow-up's six real-Git failures, fixes and browser evidence are recorded in Git comparison identity.

Validation

  • Synced main 43cd2e9 in merge commit 0bc3470, including the earlier structured-question/plan work in 35ebf01 and foreground post-edit lifecycle.
  • Node 24.20.0 / Bun 1.4.2, Git implementation 81d4c67 and CI environment 250731f: bun run check passed; bun run test reports Node 1931 passed / 5 skipped and Vitest 497 passed.
  • Production-build Chromium E2E: 66/66 passed, including real Git rename details and index-only refresh, clipboard PNG/text paste, authorized image preview, responsive settings, accessibility, command discovery, running/settled timing, PTY and embedded-browser interaction. Popup accessibility waits for the actual entry animation; cold browser startup verifies its open receipt before the first frame.
  • The real-SDK/faux-provider browser fixture verifies A remains observable while B controls input, repeated queued text stays in its Session, and stopping A leaves B running. History fixtures recover the original question, preserve the visible anchor within 2px, and reject late pages from a different branch.
  • At 0bc3470, the separate Pi SDK + scripted-provider Chromium suite passed 12/12, covering structured answers, refresh and controller isolation, cancellation, handoff, plan controls and native command results. This version-scoped result was not rerun for the Git-only change and is not real-model reliability evidence.
  • Native Pi integration covers configuration/authentication persistence and secret-free projection, continuous setup episodes, cancellation identities, monotonic timing, Session-file reopen and timing persistence failure. Windows retains content/security assertions while POSIX mode checks run only where supported.
  • Isolated real-model UI smoke saved a requested theme and showed its original command, result and 25s elapsed time. A later unapplied request remained visible. Separate sampled real-provider runs confirm writer reactivation; after episode wording clarification, two further theme changes saved successfully, each applying exactly once. This does not establish a universal model-reliability guarantee.
  • Real Chromium DPI/input evidence and prior real-model/PTY smokes remain version-scoped in the linked records. Ablation removed redundant wrappers/delays and demonstrated that Session-path transfer checks and Pi execution boundaries are necessary.
  • The resident 57161 service was updated to 0bc3470 at the user's request, with configuration and Sessions preserved. The Git review follow-up is tested separately. Credentials, private Sessions and raw evidence remain outside Git.
  • Current CI passed on 250731f: Node 22.19.0 / 24 / 26, Linux Web E2E 66/66, three independent Chromium cold starts/cleanups, and Windows process lifecycle/full-suite isolation. This includes the wheel regression on the current Node 24 job. After a cold-start diagnostic identified an invalid D-Bus address, Linux tests run in an isolated dbus-run-session; the original debug-port deadline, sandbox and failure assertions remain.
  • A real Linux ENOTEMPTY cleanup failure and two old-version regressions establish the shutdown ordering bug. A separate prior eight-second debug-port timeout still has no confirmed root cause because its original stderr was discarded; the new diagnostics preserve that evidence if it recurs. Local cleanup, logging-failure and bounded-UTF-8-tail regressions passed.
  • The live-service network diagnosis remains qualified: a same-command isolated comparison succeeded with both original startup variables (2.97s) and proxy candidate variables (1.70s). Missing proxy variables are not established as the sole cause of the earlier Git timeouts, and no candidate configuration was applied to the resident service.

Impact

  • User-visible: usable first-use controls, editable native settings, preserved paste/drafts, clearer setup and command feedback, compact sidebar, readable authorized images, lower hidden-panel work, sharper browser rendering and complete-turn elapsed time.
  • Model-visible: no new model tool, orchestration workflow or provider stack. Setup requests and completion notices clarify the existing episode boundary; typed writer authority remains runtime-enforced. Credentials and elapsed-time records do not enter model context.
  • Runtime/lifecycle: existing Pi Session/admission/cancellation ownership remains. Optional timing records are bounded native custom entries; failure to persist them cannot change execution outcomes. Browser streams and file grants are released on their existing lifecycle boundaries.
  • Persisted configuration: explicit saves update existing Pi files; unrelated provider fields remain intact. No new package preference store or hardcoded default model. UI display associations are transient and do not alter Pi history or execution outcomes.
  • Compatibility/risk: model forms support Responses, Chat Completions and Anthropic Messages; OAuth and multi-field authentication remain native Pi. Git scopes inspect the selected workspace, while external edit evidence is separate. PNG has bandwidth costs; this is not a full desktop-browser replacement. Old history without reliable timing is not backfilled. TIFF/HEIC, Safari, full IME and extended disconnect scenarios are not claimed covered. The existing bundle-size warning remains.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Sep 20, 2026
@github-actions github-actions Bot added the area:setup OpenPI setup, configuration, or setup documentation label Sep 21, 2026
@testikun testikun changed the title fix(web): improve session isolation and workbar interactions fix(web): enable native settings and improve browser, Git and file interactions Sep 21, 2026

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查提交:ece3e4201cc22b5ef719266fc55d3fa717ef6fd1

需要修改后再合并。

具体问题

P2 — 逐文件详情丢失重命名的旧路径 · web/host/git-review.ts:283

真实临时 Git 仓库执行 git mv old.txt new.txt:staged summary 为 renamed、+0/-0;请求 new.txt 详情却变成 added、/dev/null 全文件新增。filePath pathspec 只包含新路径,截掉 rename source。

P2 — 摘要 revision 未包含 index 内容身份,刷新仍保留旧暂存差异 · web/host/git-review.ts:540

保持工作文件不变,用 git update-index 将暂存 blob 从 staged-a 改为 staged-b,两者 +1/-1;两个 summary revision 完全相同,detail 内容不同。ReviewPanel effect 依赖该 revision 并保留 loadedFile,故刷新相同路径时不会重新读,继续显示旧暂存内容。应包含所选比较两端的 Git 对象身份。

P2 — POSIX 权限位断言阻断 Windows CI · tests/web/model-configuration.test.ts:67

当前 GitHub Windows job 在新增用例此行失败,实际 mode 438(0666)对期望 384(0600);Windows 不提供同样的 POSIX 权限位语义。应按平台验证适用的文件权限,不能把此失败当作 Windows 凭据泄漏证据。run 35565898489。

验证范围

已审阅生产代码、相关原生配置测试及五份历史验证记录。80 个 Node 测试通过,218 个 UI/集成测试通过。独立真实 Git 复现上述两个差异问题。

限制与后续

本轮未重新运行完整浏览器 E2E、真实模型或 Windows 本地测试;当前远端 E2E 在真实嵌入 Chromium 首帧仍 hidden 处失败,原因未定位,不能引用 PR 描述的旧本机 56/56 替代当前 CI。源码/原生集成验证不等于浏览器全面验收。

已核对的 CI 失败

Run 35565898489:Web E2E:真实 Chromium 初始图像 5 秒后仍为 hidden;根因未定。Windows:新增 models.json 用例无条件断言 POSIX 0600,实际 0666。前者需要复现,后者是明确的平台测试问题。

@testikun testikun changed the title fix(web): enable native settings and improve browser, Git and file interactions fix(web): improve settings, workbar interactions and execution feedback Sep 22, 2026
@testikun
testikun requested a review from a team as a code owner September 22, 2026 12:26
@github-actions github-actions Bot added the area:github GitHub workflows, templates, ownership, or tests label Sep 22, 2026
@testikun testikun changed the title fix(web): improve settings, workbar interactions and execution feedback fix(web): improve native settings, session history and workbar interactions Sep 22, 2026

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复审提交 4480b8d88a9453eea7b2cfed3e26a50d7927770b

本轮新增了会话历史、后台执行投影、地址草稿和 Chromium 启动/清理诊断;此前 Windows POSIX 权限断言已按平台修正。不过两个确定性 Git review 问题在当前 head 仍存在,且当前 Node 24 检查新增失败,因此仍需修改。

P2 — 逐文件详情仍丢失 rename 的旧路径 · web/host/git-review.ts:283

详情查询仍把 pathspec 限定为新路径。真实 git mv old.txt new.txt 的摘要能识别 rename,但打开 new.txt 详情会截掉 source,退化为 /dev/null 到新文件的 added diff。当前 head 没有修改这条生产路径或增加 rename detail 回归。

P2 — summary revision 仍不包含 index 内容身份 · web/host/git-review.ts:540

revision 目前使用工作树文件的 size/mtime/ctime;纯 git update-index 可以在工作文件完全不变时替换 staged blob。相同路径和相同 +/- 统计仍产生相同 revision,UI 会继续保留旧的暂存详情。revision 需要纳入所选比较两端的 Git object/index 身份。

P2 — 当前 Node 24 CI 在本 PR 修改的 wheel 合并用例失败 · tests/web/workbar-input.spec.ts:202

Run 35736169313 中 coalesces wheel deltas while an input request is in flight 期望首次 browserAction 已调用一次,实际为 0;全量结果 464 pass / 1 fail。请定位时序问题或修正确定性等待后重跑,不应以其他 job 或旧本地结果代替当前 head 的集成检查。

验证边界:没有对 115 个文件做完整浏览器人工验收;以上前两项来自可重复的真实 Git 行为,第三项来自当前远端日志。最新 Chromium cleanup 增量保留原始启动错误并等待 close 后删 profile,未发现额外阻断。

@testikun

Copy link
Copy Markdown
Collaborator Author

Problem

The composer-level file bubble accumulated pre-existing dirty files, while a long Session could say its history was exhausted even when an individual message body was only a truncated preview (observed on the resident 57161 service).

Value

The conversation now shows only changes recorded during each user turn, next to its answer. Older native turns and the full visible body of a truncated user/assistant item can be restored without raising the bounded snapshot size.

Approach

  • Follow the public Codex App Server separation of paged turns and explicit item reads, using Pi native Session entry IDs, branch ancestry, and 32k-character item pages. The UI no longer shows a misleading global truncation/start banner.
  • A Web-only Pi extension captures a pre-turn Git baseline and persists bounded temporal change evidence as a native custom entry. The inline card previews three files and opens the saved diff; the existing manual Git Review workbar remains.
  • Reconcile optimistic, live, and native user messages by exact ancestry and a bounded one-for-one display association. Older Sessions are not backfilled. Temporal Git comparison cannot attribute a writer, so no unreliable Undo command is exposed.
  • Fix the file-reference dialog contrast and wait for its entry transition before accessibility inspection. The separate 57162 preview uses an isolated Pi configuration; the resident 57161 was not restarted.

Validation

Commit 94059b5: bun run check passed; bun run test passed (Node 1,948 pass / 5 skip, Vitest 498 pass); production-build Chromium E2E passed 67/67, including desktop/mobile turn review, full-message recovery, history/observer reconciliation, and file-reference accessibility. Two earlier full browser runs encountered independent input/transition timing failures; the final full run passed after the dialog correction. Git diff whitespace check passed. GitHub CI on this commit remains pending.

Impact

No model-facing tool or package configuration change. Turn diffs are bounded private Session evidence (200 files / 256 KiB, 64 MiB baseline quota); a partial or unavailable baseline is explicitly distinguished from verified zero changes. Existing Session data and the resident service were left untouched. Research, limits, and ablation: https://github.com/testikun/openpi/blob/codex/web-ux-597/docs/research/WEB_TURN_EVIDENCE_AND_ITEM_RECOVERY_2026-09-23.md

@testikun

Copy link
Copy Markdown
Collaborator Author

CI follow-up for commit 94059b5: all checks passed. Node 22.19.0 / 24 / 26, Linux Web E2E (including the new turn/history cases), Windows background-terminal lifecycle/full-suite isolation, and area labels are green: https://github.com/openpi-dev/openpi/actions/runs/35840808353 . The earlier validation comment was written while CI was pending. The resident 57161 service remains unchanged.

@testikun

testikun commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator Author

Problem

Issue #597 的 Web 跟进:运行中的状态栏显示排队数量,但正文只存在当前浏览器的临时消息里;刷新、切换控制会话或回看历史后,用户可能只看到“1 条消息正在排队”却找不到自己的消息。

Value

排队内容在当前与后台会话中持续可见,刷新后仍可核对;消息真正进入 Pi 历史时不留下重复的“已发送”气泡。

Approach

  • 提交 f0e85cd 从所选 Pi Session 的原生 follow-up 队列投影 FIFO 正文,最多 20 条、每条最多 12,000 字符;原有总数准确保留,超出部分显示数量。
  • 输入框上方展示队列,独立于会话历史与更早消息的阅读窗口;队列变化触发快照刷新。乐观消息仍用于提交前过渡,但与原生队列中的同文消息一对一去重。
  • 参考 Codex 公开 TUI 的独立输入队列提交和出队逻辑。这不是对未公开桌面 App 布局的等价性声明;Pi 仍拥有真实队列和执行生命周期。

Validation

  • Node 24.20.0 / Bun 1.3.14:bun run check 通过;bun run test 的 Node 1948 通过、5 跳过,Vitest 500 通过;bun run test:web:e2e 67/67 通过。
  • 真实 Pi Session + Chromium 回归覆盖相同正文两次排队、浏览器刷新、后台只读会话、控制权返回、出列进入历史、停止后避免重复,以及 390px 移动端无重叠和横向溢出。57162 独立预览已更新;57161 未修改。GitHub CI run 35844954954 全部通过:Linux Web E2E、Node 22/24/26、Windows 全量隔离。
  • 消融检查:只保留原生队列投影、一对一显示去重和队列变更刷新所需状态;删除了重复的 UI 条件。

Impact

  • 用户可见:队列正文在输入框上方显示,历史仍只表示真实进入 Pi 的消息。
  • 模型可见:无新工具或上下文。运行时:只读、有界的原生队列投影及刷新事件;不改变发送、取消或出列顺序。
  • 持久化配置和 Session:无新存储或配置字段。风险:最多预览 20 条,超出仅显示剩余条数;浏览器刷新可恢复视图,但进程终止后的原生内存队列不在本变更的恢复范围。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:github GitHub workflows, templates, ownership, or tests area:setup OpenPI setup, configuration, or setup documentation documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants