Repository navigation
feat(kiosk): audio on a display: radio stations (incl. operator-ticked home-network streams) through a pinned stream relay, display tickets, transport verbs, ducking, now playing; voice turn: optional STT prompt/hotwords, repetition-loop and tool-syntax guards; kiosk 0.3.6 - #443
Merged
Merged
Conversation
kh0pper
force-pushed
the
feat/kiosk-wm1a
branch
from
October 5, 2026 17:37
e1f47df to
c403103
Compare
kh0pper
force-pushed
the
feat/kiosk-wm1b1
branch
from
October 5, 2026 18:50
c16c2cc to
b85031f
Compare
kh0pper
force-pushed
the
feat/kiosk-wm1a
branch
3 times, most recently
from
October 6, 2026 02:02
3849872 to
ec606c2
Compare
kh0pper
force-pushed
the
feat/kiosk-wm1b1
branch
from
October 6, 2026 17:55
b85031f to
d57fb80
Compare
kh0pper
force-pushed
the
feat/kiosk-wm1b1
branch
from
October 6, 2026 22:46
d57fb80 to
c255873
Compare
kh0pper
force-pushed
the
feat/kiosk-wm1b1
branch
from
October 7, 2026 00:37
c255873 to
613e92a
Compare
added 17 commits
October 7, 2026 12:44
…ved address checked from this host's view (its own addresses and every interface's on-link prefix are never public), the dialled address is the checked one, credentials on the first hop only, bounded redirects and timeouts, GET only, audio content types only
…ay/t, bound to a device and a resource, at most two concurrent requests, cut at expiry by a timer, never logged
… per display, a ticket per item, station reloads after a drop); media state on every hello; every turn_start gets exactly one closing frame; effect_ms in turn metrics; optional max_volume and pause_media_on_listen device settings
… station presets, with one canonical form for how a station is said; a home-network station needs an explicit per-station tick
…on plays at T1 with no model, 'Which one?' is remembered for 30 s, transport phrases fire only when they would change something and only in forms that name the music, the model's next is the playback verb, close everything also stops audio
…play turns; the ticket mount behind the display network gate; station admin routes behind the dashboard session (one invalid row refuses the save, a ticked row is checked by the server and its address set recorded, Test reads headers only); the turn check plays when there is a station
…fault; Save checks every ticked row first), loudest volume and pause-while-listening per display
…e now-playing window, ducking that follows the mic, the bird and the speech with a 30 s backstop, a watchdog for streams that never start or stall, a failure the socket could not carry is sent after reconnect
…tickets and the relay, transport, ducking, now playing); kiosk 0.3.0
…ware shared rule is marked for the public rule alone; a guard test: a ticked station still reaches the LAN's own global IPv6 /64 when the shared classifier knows this host's interfaces
… this host's networks as the shared classifier sees them (isOwnNetworkIp); the public test stays address-only, so a ticked station still reaches the LAN's own global IPv6 /64
…odel's next alone (it is the playback verb and never touches a window), so with nothing playing it says so; the panel test counts the theme select; the play test steps a recipe with words that ask for it
…audio defaults, page/server agreement, an offline chip, an automatic now-playing window; kiosk 0.3.2
- Station names by voice: the display's station names (and aliases) bias the STT through an optional, bounded prompt, and a call sign that STT spelled by its sounds finds the one station that sounds the same; two that sound alike are asked about.
- A playback word with filler ("louder louder", "a bit quieter please", "skip it") acts at once while something plays; a turn that asks the playback to change must end with a real call or the truthful "Sorry, I couldn't change the playback."
- Phones and tablets pause music while listening by default and let the mic go after each turn (Android leaves call audio mode); the reopen cost is measured per turn.
- The page never reports a pause or a block the media element did not make; a no-model turn records its verb in the metrics line.
- An offline display says so: the chip dims and does nothing, the window's buttons are off.
- The now-playing window opens by itself when playback starts on a display with a screen (one, reused, behind a card just asked for); the chip opens it.
Core change: the voice turn takes an optional STT prompt (opts.sttPrompt, bounded and cleaned) and records timings.verb on a no-model turn; the faster-whisper STT adapter sends the prompt.
…pen at a time; sound-alike stations are asked about; the now-playing window keeps out of the way; 60 s offline grace
- A playback word counts only when it acts on the music (nothing after it, a pronoun or a music word); another object (lights, heat, a timer, the TV) or the voice is never forced onto the playback tool, and crow_wm is forced only over music or when the music is named.
- "Skip this one." skips at once while something plays; a bare "Stop." with nothing playing answers "Nothing is playing." at once.
- A resume the browser refuses after pause-while-listening is reported as blocked again (chip shows play, never pause over silence).
- One microphone open at a time: taps during an open share it, a second tap is ignored and the bird shows the first was heard; an open released meanwhile is closed when it lands.
- A station that only sounds like ordinary words is asked about ("Did you mean …? Say yes or its name."); only a call-sign shape plays at once.
- The now-playing window comes to the front by itself only on an empty screen; otherwise it opens behind what is open.
- An offline page keeps its audio for 60 s (the server's grace plus two pings) and reconnects every few seconds while audio is live.
- With follow-up on, a phone keeps its mic through the follow-up and lets it go when the conversation settles; the per-turn release is Android-only until an iPhone is checked.
- No English station prompt on a Spanish display that detects its language; a station name that is itself a command is refused; the docs say a cloud STT receives the names.
…ry word is not a call sign; "Did you mean …?" lives one utterance; command-word station names are refused at the save
- "Pon la música más alta", "ponla más fuerte", "más bajito" ask the playback to change (crow_wm, never crow_play); the TV or the heating before them do not.
- A single ordinary word counts as a call sign only with four consonant sounds or more ("Play Cats." is asked about, never played).
- "Did you mean …?" is answered by the next utterance only; "No." clears it at once with "Okay."; a later yes never starts the radio.
- A command-word station name is refused at the save with the word; one already saved keeps loading, stays out of the speech hint, and the panel names it.
…hose audio never finishes (bounded by its own length plus a grace) and drops a decode that never answers; a phone lets the mic go when the turn settles instead of at the end of speech, so the audio-mode switch never lands in the reply; the turn report says whether the reply stalled and the audio context's state
…odel spell letters, loop and mishear short commands; only the call-sign-shaped words go to faster-whisper as hotwords; a call sign one sound off is asked about instead of going to the model; a transcript that is a repetition loop ends the turn as an empty one, before any fast path or model
added 14 commits
October 7, 2026 12:44
…red with no model even when nothing would change — the state is sent to the page again, so a page that disagrees is put right and 'Play.' never starts something else; every display turn records the server's media state before and after it
…d, spoken or kept; with radio presets a station asked for in words speech-to-text mangled still offers the play tool; 'already playing' re-sends the play to the page so it is true where it is heard
…n the element instead of a tenth of its linear gain, and a new session starts two steps below full
…nd-off call sign is asked about, repetition loops end the turn, bare Pause/Play re-send the state, 5 dB volume steps, the phone mic goes after the reply; kiosk 0.3.4
… plays (never on a pause between two sentences) and before the music comes back; a reply is cut only when its audio clock has stopped, never while a late speaker is still playing it; the metrics say how long a reply was and what the audio did at a stall
…ly when the display and the speech profile are both English, and taken from the operator's own all-capitals call signs
…ole current item again, so a page that shows another item, none, or the other state is put right; the turn report names the page's own media state at the tap and at the end
…repeated word in Spanish or English, laughter, a title, long numbers) and still drops the long loops; tool-call syntax in the other common formats is never heard either
…Nothing is paused. What would you like to hear?') instead of going to the model; a timer that has gone off still comes first; interjections without a vowel are not taken for call signs
…oot, to the step on the new volume curve that is as loud as it was (never louder); caps saved from now on are marked as on the new scale so the move never repeats
… loops only, Play with nothing loaded, the loudest-volume move
… suggested station plays at once, several are asked again by name
…sent (a true/false flag, never the words), the early transcript included
…d a fenced json block, are never heard either
kh0pper
force-pushed
the
feat/kiosk-wm1b1
branch
from
October 7, 2026 17:51
613e92a to
701352b
Compare
added 8 commits
October 7, 2026 17:22
…ered with no model in every form; a model's playback call never runs against the sentence's own command (the sentence's verb runs instead); the turn metrics name the playback verb the model asked for; more speech-to-text forms of the short commands are recognised
… a phone); a new session starts one step below full; stored loudest-volume caps move once to the step that is as loud or quieter, from the old linear scale or the previous 5 dB one
…wn playback verb wins
…uieter moves 10 dB; Quieter stops at a quiet but audible floor; a spoken level is that point on the scale; stored caps move once to the equally-or-less loud level, from the linear scale or the previous 10 dB one
…a short command; anywhere else it is part of the sentence
…so a failing test ends the run instead of hanging it
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds (kiosk 0.3.6)
"Play " on a paired display plays the stream through the page's own
<audio>element with no model call. "Pause", "keep playing", "louder", "what's playing" and "stop" act at once with no model. A question asked over music ducks it (or, on a phone or tablet, pauses it) and brings it back. The display never receives an upstream URL or a credential.Merge only inside a deploy window. This PR must not be merged outside an attended deploy window for the kiosk (it also moves stored volume caps once at boot; see "Volume").
server/tickets.js): one stream for one display under/display/t/<id>/stream, behind the display network gate (not a Funnel prefix). A ticket is bound to one device and one resource. At most 8 per device and 2 concurrent requests. Tickets last 12 h, and a timer aborts open requests at expiry. GET only,no-store, never logged.server/relay.js,server/netscope.js): each upstream carries a hop policy. On every hop the name is resolved once, every resolved address is checked, and the connection goes to the address that was checked (pinned through the request'slookup). A credential is sent on the first hop only, and https→http is refused when a credential was sent. At most 3 redirects. 10 s to headers, 30 s body idle. Only audio content types pass, and only oneRangeis passed through.server/media.js,session.js): queue, state, volume with a per-display cap. A station that drops after it has played is reloaded up to three times. Everyhellocarries the media state, and everyturn_startgets exactly one closing frame (turn_doneorturn_over). Two optional device settings,max_volumeandpause_media_on_listen; no default ever writes them.server/sources/): a source contract (contract 1) and its first adapter, operator-entered station presets with one canonical spoken form. The repo ships no presets.play.js,executor.js,patterns.js,phrases.js,strings.jsen/es): a confident station match plays at T1. "Which one?" is remembered for 30 s. The model'snextis the playback verb, and "close everything" also stops audio.runtime.js): station admin routes behind the dashboard session and CSRF. One invalid row refuses the whole save. Unpairing a display, or the end of the login behind a session display, ends its stream and revokes its tickets. When the instance has a station, the turn check adds a fourth sentence, which must start playback with no model.<audio>element and a now-playing chip. Ducking follows the mic, the bird and the speech, with a 30 s backstop. A watchdog catches streams that never start or that stall. A blocked autoplay waits for a tap. Newnowplayingwindow.Core changes (voice turn) — this PR is not kiosk-only
servers/gateway/voice/turn.js,servers/gateway/voice/turn-helpers.js,servers/gateway/ai/stt/adapters/fasterwhisper.js:opts.sttPrompt(the OpenAI-styleprompt, ≤ 200 characters) andopts.sttHotwords(faster-whisper'shotwords, ≤ 6 words), each a string or a function of the STT profile; the faster-whisper adapter sends both fields. Callers that pass neither are unchanged. The kiosk sends no prompt (a station-name prompt made the speech model spell letters, repeat itself and mishear short commands), and its hotwords are a switch that is OFF by default (CROW_KIOSK_STT_HOTWORDS=1), used only when both the display and the STT profile are English, from the operator's own all-capitals call signs.[TOOL_CALLS],<|python_tag|>,<function_call>, a bare or pretty-printed JSON call, a fenced json block; any case; split across deltas) is never captioned, spoken or kept in the history. Plain prose that quotes a{"name": …}object is cut too (accepted for speech).runVoiceTurn: the kiosk AND the glasses endpoint. The companion runs its own loop and is not covered.verb(a fixed word on a no-model turn),stt_degenerate,tool_text,spoken_chars,stt_hotwords(true/false; never the words). Never transcript text.Changed since the last push (third attended smoke and its review)
model_verbs) and the one that ran instead (model_verbs_kept), as fixed words.Earlier fixes (second attended smoke and two reviews)
tts_stalled,tts_audio_ms,stall_ctx_state,stall_clock_moved,ctx_state.media_before,media_after) and the page's own (page_media_start,page_media_end).Known gaps
Home-network ("local stream") station policy
A station may point at a stream on the home network or the tailnet only when the operator ticks Home network for that row. The tick is off by default and can be set only through the admin route (dashboard session + CSRF). The model never supplies a URL.
On every hop of every station, ticked or not: an address of this host itself is refused (
own_address); any address inside the on-link prefix of any interface of this host is refused (private_address), with the interface table and default-route interfaces read again on every hop (fail closed); address classes come from the shared classifier (servers/shared/ip-classify.js), and the relay additionally refuses an IPv4 embedded in a SIIT, IPv4-compatible, NAT64, 6to4 or Teredo form even when it is public.For a ticked station the entered host:port may resolve only to an address inside a prefix of the default-route interface (RFC 1918, CGNAT, ULA, or that LAN's own global IPv6 prefix; never the network or broadcast address) or the tailnet ranges (
100.64.0.0/10,fd7a:115c:a1e0::/48) when not inside another interface's prefix. Refused even with the tick: other interfaces' prefixes, an RFC 1918 address on no interface, loopback, link-local (metadata included), multicast, broadcast, reserved, embedded or translated IPv4, IPv6 site-local, this host, and any public answer for a ticked host. Pinning: the server resolves the ticked host once at save, judges every address and stores the set; every play and same-host redirect must resolve inside it (address_changedotherwise). Redirects from a ticked station may go only to the same host:port (http→https allowed) or to a public address under the public rule; no credential is ever sent on this policy.The relay's public rule also refuses any address the shared classifier counts as one of this host's networks (
isOwnNetworkIp), atnetscope.jsjudgeAddress, on the public rule only; a guard test keeps the production predicate address-only (kiosk-relay).How this meets the display tools on
mainnextas a window verb (here it is only the playback verb). The guard's parity test still passes for every window verb.Tests
main(54e682d5). Full suite on this head, scratch env: 7,918 tests, 7,917 pass, 0 fail, 1 skipped.build-registry --checkOK,check-public-hygieneclean.Not run here: anything on a live host (evaluation, attended smoke, deploy window). Those happen separately before merge.