Skip to content

feat(webui): rebindable shortcuts, voice input, and a dev origin fix - #33

Merged
antianqi merged 4 commits into
webuifrom
fix/webui-settings-voice-shortcuts-i18n
Oct 9, 2026
Merged

antianqi merged 4 commits into
webuifrom
fix/webui-settings-voice-shortcuts-i18n

Conversation

@antianqi

@antianqi antianqi commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Roadmap P 区, two rows plus the two fixes that fell out of building them.

Rebased onto webui @ d29c87a — the baseline that removed scheduled tasks and
split the server layers. No conflict: the baseline touched only two of the six
source files this change edits, and both of those edits are scheduled-task
deletions.

P-1 快捷键管理

Ctrl+, was a label on a menu row and nothing ever handled it. Every command
in WEBUI_SHORTCUT_COMMANDS now has a branch behind it, and any of them can be
rebound. Bindings live in localStorage and are resolved per keystroke; a
binding the user is actively typing must not swallow the character, so
matchesWebuiShortcut reports what it matched rather than always answering a
boolean. A command with no branch would render in the settings page and do
nothing — the defect the registry's tests now assert in both directions.

P-2 语音输入

There was nothing behind the voice tab: no getUserMedia, no
SpeechRecognition, no voice state of any kind. The feature turns on one
honest question — what does this browser do when nobody is listening? The Web
Speech API is Chromium and partly Safari; Firefox has never shipped it. A mic
button that appears and then fails costs the user a click and tells them
nothing, so "can this browser listen" is a first-class state in the reducer.
When the answer is no, the composer renders no button at all and the settings
page says which browsers would work.

What a recognition result does to text already typed is the other decision
worth freezing. The recogniser revises interim text continuously while the user
keeps typing, so the state records where this utterance began and every interim
update replaces exactly the segment this utterance contributed — everything
before and after it survives. Cancelling throws the segment away entirely: the
user pressed stop, they did not ask for half a word. A failure keeps whatever
was already recognised, because those words were on screen.

Two fixes this work had to carry

afaafbb — the dev shell could not connect. ADR 0004 pins the packaged server
to same-origin requests, so the origin port has to equal the bound port; the
Vite dev server necessarily listens on a different one, and every dev-mode
upgrade and API call was rejected with 403, leaving the shell on "WebUI
connection failed". isAllowedOrigin now takes the dev flag the service
already had and accepts any loopback origin in dev. The loopback-host and
http/https checks still apply, so a page on another machine still cannot reach
the service.

4e8c870 — the voice switch only took effect after a reload. The hook read
localStorage once at mount. voice-state.ts now exposes a module-level
subscribeWebuiVoiceSettings, writeWebuiVoiceSettings persists before it
notifies, and the hook returns the subscription from its mount effect so it
detaches on unmount. SettingsModal.tsx goes through that writer.

Validation

Run on webui @ d29c87a + this branch:

  • pnpm check:source exit 0 (4783 files)
  • pnpm typecheck:webui-full exit 0
  • run-vitest-suite.mjs webui 10 failed | 1883 passed | 4 skipped
  • shortcut-state.test.ts + voice-state.test.ts 79 passed

The 10 failures are the pre-existing Windows baseline: 7
webui-boundary-check path assertions, 1 webui-design-tokens path assertion,
1 profile-files path assertion, 1 webui-service shutdown timeout.

Confirmed by negative control, not by inspection: the same suite run on a
clean d29c87a with this work absent fails those same 10 and passes 1804.
This change adds 79 passing tests and no new failure.

CI is currently absent for every PR in this repository. GitHub Actions returns
HTTP 422: Actions has been disabled for this repository, and the
repository-wide run list is empty after 2026-10-07. mergeStateStatus: CLEAN
therefore means "no conflicting changes", not "checks passed".

Test evidence

79 tests: 48 in shortcut-state.test.ts, 31 in voice-state.test.ts, both
registered in test/vitest-suites.json.

voice-state.ts is pure and DOM-free, which is what makes those decisions
testable at all — the webui suite has no jsdom and the recogniser itself needs
a microphone. use-webui-voice.ts holds the lifecycle glue and declares the
non-standard global narrowly rather than pulling in a DOM library that has it;
the one rule that file exists to enforce is that no recogniser is constructed
until the user asks for one.

Two holes negative injection found, both in this change's own wiring. The
composer assertion only checked that data-testid="composer-voice" appears in
the file, so replacing the availability guard left every assertion green while
removing the button from the composer entirely; it now asserts the guard and
the control as one piece of source. And the toggle fix needed a wiring
assertion at all because the projection tests could not reach the hook —
deleting the subscription line from the mount effect left the whole suite
green.

One limitation stated rather than papered over: that hook wiring is covered by
source assertions, not behavioural tests. The browser fixture hand-rolls its
own textarea, so no browser test here reaches the hook. A green suite did not
verify it.

Existing tests moved their tab lists rather than losing their teeth:
settings-modal.test.tsx and settings-account-tab.spec.mjs place voice in
the implemented group and still assert connection is disabled, which is what
keeps either check from being vacuous.

Scope

多语言 (P-3) is deliberately not here: 8604 CJK characters across 77 of 109
client files, with mavis-locale currently read by a useState that has no
setter. That is a full extraction and wants its own PR.

Roadmap P 区「快捷键管理」. The ledger records this row as "the tab is
disabled", which undersells it: there is no shortcut system at all. The user
menu has always labelled its settings row `Ctrl+,`, and nothing anywhere in the
client honoured that combination. The three document-level keydown listeners
that do exist belong to the context menu, the goal banner and the user-menu
popover, and each handles only Escape while its own popover is open. So the
first thing this has to survive is being the thing the label already promised.

Three commands are registered, and only three, because they are the only ones
with a real handler behind them: openSettings (Ctrl+,), toggleSidebar (Ctrl+B,
which toggles the rail that already exists) and focusComposer (Ctrl+L). A
registry full of rows nobody wired would be the same defect wearing a
different hat, so the tests assert the other direction too — every command id
must appear as a branch in the shell.

`shortcut-state.ts` is pure and DOM-free, which is what makes the interesting
decisions testable without jsdom: parsing and canonicalising a binding,
matching an event, resolving overrides over defaults, refusing a combination
another command already holds. Bindings are stored platform-neutrally with Ctrl
as the primary modifier and translated for display on macOS; storing the
display string would mean a binding written on one platform does not parse on
the other.

The settings page is presentational like the E-area panels, so its rows are
asserted directly, and the click paths that a static render cannot reach are
asserted against the source — the same shape `rail-pin-affordance.test.ts`
already uses in this repo.

Two things are deliberately refused when rebinding, and the tests say why: a
lone modifier is a press in progress, and Escape is the cancel gesture. Escape
closes every popover in this shell, so no command defaults to it either.

The settings dialog is mounted by UserMenu, which only exists while the rail
is expanded. Firing `Ctrl+,` into nothing is worse than not having the
shortcut, so the handler opens the rail first and delivers the signal once the
menu has mounted.

Validation
- pnpm check:source                          exit 0  (4755 files)
- pnpm typecheck:webui-full                  exit 0
- pnpm build:webui                           exit 0
- run-vitest-suite.mjs webui    10 failed | 1890 passed | 4 skipped
- playwright test test/webui-browser/          96 passed

The 10 failures are the unchanged pre-existing Windows baseline, confirmed on
a clean tree of this same base by stashing: 7 webui-boundary-check path
assertions, 2 webui-design-tokens path assertions, 1 webui-service shutdown
timeout. This branch adds none.

Two existing tests moved their lists rather than losing their teeth:
settings-modal.test.tsx and settings-account-tab.spec.mjs both asserted that
`shortcuts` sat in the disabled group. Both now assert it in the implemented
group while still asserting `voice` and `connection` are disabled — which is
what keeps the check from being vacuous.

Test evidence
48 new tests in shortcut-state.test.ts, registered in test/vitest-suites.json.

Red before the implementation: 11 of them failed against an empty stub — the
component was not exported, the tab was not routed, the tab was still
disabled, and the shell had no key handler.

Negative injection: 24 mutations across the registry, the page and the wiring,
0 survivors, implementation restored byte-for-byte. The first run found four
problems, two of them real:

- Removing `"shortcuts"` from the empty-pane guard left the suite green. The
  routing assertion and the guard are independent: without the tab in the
  guard, both the page and the empty pane render at once. There is now an
  assertion scoped to the guard, alongside the one for the E-area tabs.
- Replacing the handler's override read with an empty list left the suite
  green, because the only wiring assertion was `resolveWebuiShortcutBindings`
  appearing somewhere in the file. This is the seam between the two halves of
  the feature: the page would show saved keys while the shell kept firing the
  defaults. Both halves are now asserted.

The other two first-run entries were my own harness writing `{(true ? (`,
which is not valid JS — the run crashed rather than reporting a result, and
the harness counts a crash as inconclusive rather than killed, which is the
conservative call.

The suite also caught a real bug while the capture path was being written:
`F2` was being case-folded to `f2`, and matching compares named keys verbatim,
so the binding would have been stored in a form that could never fire.

Scope: this is the first of the three rows in P 区. 语音输入 has no
implementation anywhere in the client yet — no getUserMedia, no
SpeechRecognition — and 多语言 is a full extraction: 8604 CJK characters across
77 of 109 client files, with `mavis-locale` currently read by a `useState`
that has no setter. Neither belongs in this change.
Roadmap P 区「语音输入」. The ledger records this row as "the tab is
disabled", which is accurate and also understates it: there is nothing behind
the tab at all. The client has no getUserMedia, no SpeechRecognition, and no
voice state of any kind.

The whole feature turns on one honest question — what does this browser do
when nobody is listening? The Web Speech API is Chromium and partly Safari;
Firefox has never shipped it. A mic button that appears and then fails costs
the user a click and tells them nothing, so "can this browser listen" is a
first-class state in the reducer rather than a boolean folded away at the call
site. When the answer is no, the composer renders no button at all and the
settings page says which browsers would work.

The other decision worth freezing is what a recognition result does to text
the user has already typed. The recogniser revises its interim text
continuously while the user keeps typing, so the state records where this
utterance began and every interim update replaces exactly the segment this
utterance contributed — everything before and after it survives. Cancelling
throws the segment away entirely: the user pressed stop, they did not ask for
half a word. A failure keeps whatever was already recognised, because those
words were on screen.

`voice-state.ts` is pure and DOM-free, which is what makes those decisions
testable at all — the webui suite has no jsdom and the recogniser itself needs
a microphone. `use-webui-voice.ts` holds the lifecycle glue and declares the
non-standard global narrowly rather than pulling in a DOM lib that has it. The
one rule that file exists to enforce: no recogniser is constructed until the
user asks for one.

Validation
- pnpm check:source                          exit 0  (4758 files)
- pnpm typecheck:webui-full                  exit 0
- pnpm build:webui                           exit 0
- run-vitest-suite.mjs webui    10 failed | 1917 passed | 4 skipped

The 10 failures are the unchanged pre-existing Windows baseline: 7
webui-boundary-check path assertions, 2 webui-design-tokens path assertions,
1 webui-service shutdown timeout. Confirmed on a clean tree by stashing.

Browser suite: NOT clean, and not attributable to this change. On this branch
19 of 96 fail; on a clean tree of the same base with this work stashed, 20 of
96 fail — the clean tree is one worse. The run also went from 56s earlier today
to ~9.3 minutes. Ruled out so far: `reuseExistingServer` is false by default in
playwright.config.mjs, so no stale server was adopted; killing 25 orphaned
chrome and 17 orphaned node processes left both numbers unchanged; and the
config sets no test timeout, so the `--timeout=30000` used in these runs was a
no-op. What remains is not something I can attribute, and it is not something I
can call a passing gate. Someone should look at the machine.

Test evidence
27 tests in voice-state.test.ts, registered in test/vitest-suites.json. Red
before the implementation: the module did not exist, the component was not
exported, the tab was not routed, the tab was still disabled, and the composer
had no mic.

Negative injection: 21 mutations across the reducer, the page, the composer and
the hook, 0 survivors, implementation restored byte-for-byte. Two real holes,
both from this change's own wiring:

- The composer assertion only checked that `data-testid="composer-voice"`
  appears in the file. Replacing `{voice.available ? (` with `{false ? (`
  left every assertion green while removing the button from the composer
  entirely. It now asserts the guard and the control as one piece of source.
- A guard assertion for "the mic must not appear before the feature is on"
  passed against the wrong file: the guard lives in `use-webui-voice.ts`, and
  the mutation was filed against the composer, where it matched nothing and was
  correctly reported STALE rather than counted as killed.

A third hole was in my own code and is worth recording. The hook mirrors the
reducer's draft back into the composer, and unguarded it compares a fresh
state's `""` against whatever the user has typed — so it clears the composer on
the first keystroke. It is fixed by a touched-flag, and negative injection now
covers that guard. The fix itself is a source assertion, not a behavioural
test: the browser fixture hand-rolls its own textarea, so no browser test here
reaches this hook. I am not claiming a green suite verified it.

Existing tests moved their tab lists rather than losing their teeth:
settings-modal.test.tsx and settings-account-tab.spec.mjs now place `voice` in
the implemented group and still assert `connection` is disabled, which is what
keeps either check from being vacuous.

Scope: second of the three rows in P 区. 多语言 is a full extraction — 8604 CJK
characters across 77 of 109 client files, with `mavis-locale` currently read by
a `useState` that has no setter — and is deliberately not in this change.
…nect

Every dev-mode upgrade and API call was rejected with 403 Forbidden Origin,
and the shell sat on "WebUI connection failed". Sessions and account state came
back empty because nothing had ever connected: the websocket handshake never
completed.

`isAllowedOrigin` requires the origin's port to equal the bound port, which is
correct for the packaged server — ADR 0004 has it serve the built client from
the same listener, so same-origin is the right rule there. The Vite dev server
necessarily listens on a different port (5173 against 8787), so every request
the browser made carried an origin the check rejected.

Measured, before this change:

    ws://127.0.0.1:8787/ws  Origin http://127.0.0.1:8787  ->  OPEN
    ws://127.0.0.1:5173/ws  Origin http://127.0.0.1:5173  ->  ERROR

After: both OPEN.

The port check is skipped only in dev, and only after the checks above it have
already held — the origin must still be http or https and its host must still be
`127.0.0.1` or `localhost`. A page on another machine still cannot reach the
service.

This is a port of another contributor's uncommitted work on this file, which I
had to drop while syncing the base: their patch no longer applies, because
PR #29 added 153 lines to `service.ts` and PR #32 moved the other two files in
the same change set. The reasoning and the shape are theirs, carried over
rather than rewritten; if they would rather land it themselves, this commit
should be dropped in favour of theirs.

Validation
- pnpm typecheck:webui-full                  exit 0
- run-vitest-suite.mjs webui    10 failed | 1917 passed | 4 skipped

The 10 failures are the unchanged pre-existing Windows baseline. The browser
suite's own harness does not exercise the dev proxy path, so it neither caught
this nor proves the fix; the handshake measurement above is what does.

Not covered here: `vite.config.ts` and `build-webui-styles.mjs` carried related
edits in the same dropped patch. Those address dev-server packaging rather than
the handshake, and neither was needed to restore the connection.
The composer reads its voice settings once, in the mount effect, and nothing
re-runs that effect when the settings page writes. So turning voice on left
the switch reading "on" in settings and no mic button in the composer until
the user reloaded the page — the same "the structure is right, the thing does
not work" failure this row started as, caught on the first manual pass.

The two ends only agree if the settings page writes through a writer that also
tells its readers, and the composer subscribes to that writer:

- `voice-state.ts` grows a module-level `subscribeWebuiVoiceSettings` and a
  `writeWebuiVoiceSettings` that persists and then notifies. A subscription
  rather than a DOM event: both ends are React state in the same bundle, so an
  event bus would buy nothing here.
- `use-webui-voice.ts` returns the subscription from the mount effect. It is
  returned, not merely called, so React tears the listener down on unmount.
- `SettingsModal.tsx` writes through `writeWebuiVoiceSettings` instead of
  calling `localStorage.setItem` directly. A bare `setItem` persists the value
  and tells nobody, which is the bug itself.

## Test evidence

31 tests in `voice-state.test.ts` (was 27), all green. The four new ones cover
the join between the two ends:

- two behavioural tests on the pub/sub itself — a write reaches every reader,
  a write after unsubscribing does not, and the value written is the value
  stored
- two source assertions pinning the wiring. These are source assertions for the
  same reason the rest of that block is: the subscription lives in a hook
  effect, the webui suite runs in `environment: "node"` with no jsdom and no
  test renderer, and `renderToStaticMarkup` does not run effects. They exist
  because the projection tests provably cannot reach them — deleting
  `return subscribeWebuiVoiceSettings(setSettings);` leaves all 31 green.

Negative injection: 30 mutations, 0 survivors (`neg-inject-voice.mjs`). The
nine added for this fix include the three that only the wiring assertions can
kill — the composer not subscribing, subscribing without cleanup, and
subscribing from the wrong effect.

Gates: `pnpm check:source` 4758 files clean, `pnpm typecheck:webui-full` clean,
full webui suite 10 failed / 1921 passed / 4 skipped. The 10 failures are the
pre-existing Windows baseline (path-separator assertions in
`webui-boundary-check`, `webui-design-tokens`, `profile-files`, plus one
`webui-service` shutdown timeout); the count is unchanged from the baseline at
1917 passed, and the delta is exactly these four new tests.

Verified in a real browser at 127.0.0.1:5173 — opening Settings → Voice,
flipping the switch, and returning to the app without a reload puts
`[data-testid="composer-voice"]` in the composer, between the `+` button and
the authorisation-mode button.
@antianqi
antianqi force-pushed the fix/webui-settings-voice-shortcuts-i18n branch from e678d2a to 4e8c870 Compare October 9, 2026 17:18
@antianqi antianqi changed the title feat(webui): make Ctrl+, true, and let it be rebound feat(webui): rebindable shortcuts, voice input, and a dev origin fix Oct 9, 2026
@antianqi
antianqi merged commit cf30548 into webui Oct 9, 2026
antianqi added a commit that referenced this pull request Oct 10, 2026
…-shortcuts-i18n"

This reverts commit cf30548, reversing
changes made to d29c87a.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant