From effc6b9f441f00e2c9e3e5852837b46a6536b482 Mon Sep 17 00:00:00 2001 From: Andres Cera Date: Sun, 26 Jul 2026 11:39:55 -0500 Subject: [PATCH] fix(live): a frozen audio meter is a dead meter, not a live one The audio-level meter could sit indefinitely on fixed bars with peak ticks instead of falling to its documented "unavailable" state whenever something upstream broke. Its staleness watchdog stamped its clock on every incoming frame, so it only ever proved that frames were still flowing. That is not the same as proving the audio path behind them is alive, and the gap is reachable on real hardware: every liveness check in this stack -- cerastream's 2 s sidecar delivery probe and its streaming-owner deadline, the bridge's foreign-card gate, this watchdog -- measures buffer DELIVERY, and a capture device that keeps clocking ALSA buffers of frozen content passes all of them. Confirmed on a Rock 5B+: a RODE HDMI-to-USB-C with nothing on its HDMI input held its capture substream RUNNING with hw_ptr advancing, so the engine truthfully published a bit-identical rms_db of [-41.522344822589105,-44.116395350676385] for 226 frames out of 226 across 45 s. At the meter's -60 dBFS floor that is a 31% bar that never moves. Key the watchdog on CONTENT instead, which is the rule stores/hud/staleness.ts already uses for per-interface freshness ("reference identity cannot tell which source actually got new data; the content fingerprint can"). The meter was the last live-data surface still stamping on arrival. The same change also stops the clock being reset by cerastream's late-subscriber hydration, which replays its cached last observation to every reconnecting subscriber -- and the idle sidecar, unlike the streaming owner, has no delivery deadline to replace it. Two readings stay exempt and always count as life however long they repeat: an engine `unavailable` marker, which already states the gap and would only lose its typed reason by ageing out, and genuine digital silence, which is a live meter correctly reporting a muted mic and already renders as its own `silent` state. Only a non-silent frozen reading draws bars an operator reads as signal. The rule is pure and rune-free so it is testable without the component; the component test drives a reactive feed, because a non-reactive mock would make the meter go stale for the "feed stopped" reason and prove nothing. Four of the eight new component cases fail against the pre-fix tree with the exact bug signature; the other four are controls that pass on both. No engine or bridge change: the engine is correctly reporting what it measures, and collapsing repeats upstream would leave the consumer unable to tell "stopped" from "unchanged". --- apps/frontend/AGENTS.md | 1 + .../preview/LiveAudioMeter.frozen.test.ts | 213 +++++++++++++++++ .../components/preview/LiveAudioMeter.svelte | 39 ++- .../__fixtures__/audio-level-source.svelte.ts | 30 +++ .../preview/audio-meter-liveness.test.ts | 222 ++++++++++++++++++ .../preview/audio-meter-liveness.ts | 152 ++++++++++++ 6 files changed, 644 insertions(+), 13 deletions(-) create mode 100644 apps/frontend/src/lib/components/preview/LiveAudioMeter.frozen.test.ts create mode 100644 apps/frontend/src/lib/components/preview/__fixtures__/audio-level-source.svelte.ts create mode 100644 apps/frontend/src/lib/components/preview/audio-meter-liveness.test.ts create mode 100644 apps/frontend/src/lib/components/preview/audio-meter-liveness.ts diff --git a/apps/frontend/AGENTS.md b/apps/frontend/AGENTS.md index 06094f7e2..4828c17c8 100644 --- a/apps/frontend/AGENTS.md +++ b/apps/frontend/AGENTS.md @@ -241,6 +241,7 @@ CI job that uploads the signed bundles to R2. Pipeline (each step gates the next - Selected-but-unbound coarse row: warn loudly, point at the real device [EXISTS]: the state above is real, but it used to look identical to a working selection — a live operator report on the RK3588 board sat on the coarse "HDMI Capture" row (permanently "Not connected") while their actual camera, a RØDE HDMI-to-USB-C adapter, waited one row below under a name they did not recognise. The small "?" popover was too easy to miss against a lime `Check` + "Selected". Fix: when a coarse row IS `config.source`, `SourceSection` swaps the lime affirmation for the amber warning register — `border-status-warning/60 bg-status-warning/10` on the row, a `TriangleAlert` in the selected chip (`source-selected-`), `data-unbound="true"` on the `
  • ` — and renders a `role="status"` band below it (`source-coarse-unbound-`). When a CONNECTED capture device's REAL hardware name contains the coarse row's own pipeline token (`hdmi`, `usb`, …) the band adds a "Did you mean ?" lead (`source-coarse-suggestion-lead-`) plus one-tap switch buttons (`source-coarse-suggestion-`) that reuse the ordinary `handleSelectSource` write. The rule is the pure, rune-free `lib/streaming/coarse-source-hint.ts` (`suggestedCapturesForCoarse`/`deriveCoarseUnboundState`, capped at `MAX_COARSE_SUGGESTIONS` = 3): matching is on the PIPELINE TOKEN (not the translated label, so it is locale-independent), candidates must be live (`available !== false && lost !== true`), and no match renders NO pointer — a wrong "did you mean" is worse than none. HONESTY IS UNCHANGED: the "Not connected" pill and its explainer stay exactly where they were, the row stays selectable, and nothing fakes a connection. This is a UX-clarity fix only — actually binding the on-board HDMI-RX to a real pipeline (the "camlink bridge") remains a separate, deferred decision. Copy: `live.source.{coarseUnboundTitle,coarseUnboundBody,coarseSuggestionOne,coarseSuggestionMany,coarseSuggestionAction}` (10 locales). Coverage: `lib/streaming/coarse-source-hint.test.ts`, `SourceSection.test.ts` ("selected-but-unbound coarse row"), and the capability-truthfulness e2e gate (`tests/e2e/truthfulness.spec.ts`). - Source codec labelling (#196, `fa16cea9`) [EXISTS]: a capture row's codec badge is derived from `source.modes[].media_type` via `captureModeCodecs` — NOT from the engine's single collapsed `kind`. cerastream collapses a dual-codec UVC camera to one H.265-priority `kind`, but its per-mode `media_type` retains both, so a camera advertising H.264 AND H.265 renders one combined "UVC H.264/H.265" badge instead of silently claiming only one. `SourceSection.svelte` derives the real codec set from `source.modes[].media_type`, and the same modes-derived set threads into `sourceOffersCodec` (`lib/streaming/passthrough.ts`) so the EncoderDialog passthrough disclosure stays correct for a dual device. No engine change was needed — the truthful set was already on the wire. The separate COARSE USB-camera fallback label stays short and codec-free while still carrying the UVC device-family tag across all 10 locales. Coverage: `SourceSection.test.ts`, `passthrough.test.ts`, `truthfulness.spec.ts`. - Inline audio-level meter (Todo 22; inlined 2026-07) [EXISTS]: `lib/components/preview/LiveAudioMeter.svelte` is the store-connected wrapper around the presentational `AudioLevelMeter.svelte`, mounted OUTSIDE the preview at TWO inline sites — `SourceSection.svelte`'s audio-source block (idle) and `LiveSummaryStrip.svelte`'s audio line (streaming). Exactly one is on screen at a time (LiveView switches cockpits), so the meter is continuous across start/stop and moves on the selected device's real audio WHILE IDLE with no preview open. It is deliberately NOT its own full-width page section: an operator reads the level next to the device it belongs to. **It ALWAYS renders** — `AudioLevelMeter` owns the active/silent/unavailable states, so a feed that has never delivered a frame (`data-pending="true"`), a stalled feed (`data-stale="true"`, 2 s deadline), or an engine `unavailable` marker all show the meter's own unavailable copy. Never gate the wrapper on `level !== undefined`; unmounting it makes "no meter" indistinguishable from "no meter here". It consumes the cerastream `audio-level` IPC topic bridged over the MAIN authenticated backend WS broadcast (`apps/backend/src/modules/streaming/audio-meter-bridge.ts` → `subscriptions.svelte.ts`) — NOT a second frontend socket, and NOT the preview socket (which exists only while a preview is open). Device labels render real product names + transport tags (` · USB`) from `audio-naming.ts`, and saved audio selections migrate to the stable identity (label display-only). Coverage: `LiveAudioMeter.test.ts`, `AudioLevelMeter.test.ts`, backend `audio-meter-bridge.test.ts` + `audio-naming.test.ts`. +- The meter's staleness watchdog keys on CONTENT, never ARRIVAL [EXISTS]: `lib/components/preview/audio-meter-liveness.ts` is the pure, rune-free rule (`meterFingerprint` / `trackMeterFreshness` / `isMeterStale`); `LiveAudioMeter` only layers runes on it. This is the SAME rule `stores/hud/staleness.ts` already codified for per-interface freshness — "reference identity cannot tell which source actually got new data; the content fingerprint can" — and the meter was the last live-data surface still stamping its clock on every incoming frame. That is not a style point: **every** liveness check in this stack measures buffer DELIVERY (cerastream's 2 s sidecar delivery probe and its streaming-owner deadline, the bridge's foreign-card gate, this watchdog), and a capture device that keeps clocking ALSA buffers of FROZEN content passes all of them. Confirmed on a Rock 5B+ — a RØDE HDMI-to-USB-C with nothing on its HDMI input held its capture substream RUNNING with `hw_ptr` advancing while the engine truthfully published a bit-identical `rms_db:[-41.522344822589105,-44.116395350676385]` for 226 frames out of 226 over 45 s, which draws a 31 % bar that never moves. Content-stamping also stops the watchdog being reset by cerastream's late-subscriber hydration, which replays the cached last observation to every reconnecting subscriber (and the IDLE sidecar, unlike the streaming owner, has no delivery deadline to replace it). TWO readings are deliberately exempt and always count as life however long they repeat: an engine `unavailable` marker (it already states the gap, and ageing it out would only discard its typed reason) and genuine digital silence (an unchanging floor reading is a live meter correctly reporting a muted mic, and `AudioLevelMeter` renders it as its own `silent` state) — only a NON-silent frozen reading draws bars an operator reads as live signal. `AUDIO_METER_FLOOR_DB` MUST stay equal to `AudioLevelMeter.svelte`'s `FLOOR_DB` or the two disagree about what is on screen. Do NOT "simplify" this back to stamping on arrival, and do NOT push the dedupe into the engine or the bridge — the engine is correctly reporting what it measures, and collapsing repeats upstream would leave the consumer unable to tell "stopped" from "unchanged". Coverage: `audio-meter-liveness.test.ts` (pure rule) + `LiveAudioMeter.frozen.test.ts` (rendered DOM, driven through the reactive `__fixtures__/audio-level-source.svelte.ts` feed — a non-reactive mock would make the meter go stale for the "feed stopped" reason and prove nothing). - Re-enumeration is MOVED, not GONE [EXISTS]: a capture device replugged WHILE STREAMING comes back on a NEW node path (the engine still holds the old one, so the kernel cannot recycle it), and the engine keeps reporting the node it opened at start. `lib/streaming/live-source-state.ts` is the pure, rune-free ONE verdict both mid-stream source surfaces read: `deriveLiveSourceState` (drives `LiveCockpit`'s `active-source-lost-banner`) and `canOfferLiveSourceSwitch` (drives `LiveSourceSwitch`'s render gate). They were derived independently before and DRIFTED — after a replug the alert stayed up while the switch card unmounted, so the banner told the operator to "switch to another source to keep your stream alive" with nothing left to switch with. `sourceLost` therefore OPENS the switch gate on its own: a running id that resolves to no row can never satisfy the capture-origin test, and that is exactly the state the alert points at. **Id PRECEDENCE is unchanged** — the engine's `active_input` still wins over `config.source`, so a source the engine switched to and then genuinely lost still alarms; only the LOOKUP is identity-aware (`findSourceById` in `sourceSummary.ts`, which accepts the backend's proven `previousIds` aliases). Do NOT "simplify" this into a `??` chain that prefers whichever id resolves — that silences a real loss. Backend contract: `apps/backend/AGENTS.md` → "Staying ITSELF is not the same as keeping its NODE PATH". Coverage: `lib/streaming/live-source-state.test.ts` (incl. the exhaustive alert⇒affordance invariant) + `src/tests/live-source-renumber-recovery.test.ts`. - Engine-truth-clears-on-stop [EXISTS]: `getStatus()?.active_encode` follows the SAME contract as `getLinkTelemetry()` below, and for the same reason. `deriveActiveSummary` derives its `live` boolean from `Boolean(activeEncode)` and then prefers `activeEncode.active_input` over the fresh `config.source`, so a stale object does not merely show old numbers — it asserts the device IS streaming. Found live: a stopped ~36 min session kept rendering `"● Live RØDE HDMI to USB-C … H.265"` while the picker row below correctly showed the newly selected HDMI Input. Both ends now guarantee the clear: the backend drops `active_encode` from engine telemetry when the engine reports it is not streaming AND on `stop()` itself (a crashed engine sends no final idle frame), and pushes it as an EXPLICIT value on every status nudge; the frontend additionally sets it to `null` on the `wasStreaming && !isStreamingState` edge. That second half is load-bearing because the status merge deliberately preserves an omitted field — the same raise-but-never-retract latch as `policy_route_missing`. Coverage: `subscriptions-sources-telemetry.test.ts` ("active_encode clears on stream stop", incl. a stop frame that omits it and a steady-state streaming tick that must NOT clear). - Telemetry-clears-on-stop [EXISTS]: `getLinkTelemetry()` (subscriptions) is guaranteed `null` (never a stale object) on the streaming→stopped transition edge — belt-and-braces on both ends: the backend's 5 s heartbeat emits exactly one `{linkTelemetry: null}` frame after stop, and the frontend additionally clears `linkTelemetryState` on the `wasStreaming && !isStreamingState` edge as a second guarantee even if a stop frame omits the field. The tri-state distinction is load-bearing: `undefined` = pre-first-status (skeleton), `null` = delivered-empty/stopped (dashes), object = live values — do not collapse `undefined`→`null`. HUD bitrate and per-interface throughput (`buildLinks(..., isStreaming)`) follow the same never-stale-past-stop rule. diff --git a/apps/frontend/src/lib/components/preview/LiveAudioMeter.frozen.test.ts b/apps/frontend/src/lib/components/preview/LiveAudioMeter.frozen.test.ts new file mode 100644 index 000000000..5edc3da8d --- /dev/null +++ b/apps/frontend/src/lib/components/preview/LiveAudioMeter.frozen.test.ts @@ -0,0 +1,213 @@ +// @vitest-environment jsdom +/** + * LiveAudioMeter — the frozen-meter regression, driven through a REACTIVE feed. + * + * `LiveAudioMeter.test.ts` covers the always-mounted/pending/stall/unavailable + * contract against a static level. It cannot cover this bug: its mock getter + * reads a plain `let`, which `$derived` does not track, so the component sees + * exactly one frame per test. The defect here is frames that KEEP ARRIVING while + * their content never changes, so the feed has to be genuinely reactive — + * otherwise the meter would go stale for the "the feed stopped" reason and the + * assertion would pass while proving nothing. + * + * Reproduces the Rock 5B+ finding byte for byte: a RØDE HDMI-to-USB-C with no + * source on its HDMI input kept its ALSA capture substream RUNNING and its + * `hw_ptr` advancing, so cerastream truthfully published an unchanging + * `rms_db:[-41.522344822589105,-44.116395350676385]` at 5 Hz — 226 identical + * frames out of 226 over 45 s. At a −60 dBFS floor that draws a 31 % bar that + * never moves. + */ +import type { AudioLevelMessage } from "@ceraui/rpc/schemas"; +import { render } from "@testing-library/svelte"; +import { flushSync } from "svelte"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +vi.mock( + "$lib/rpc/subscriptions.svelte", + async () => await import("./__fixtures__/audio-level-source.svelte"), +); + +import { setAudioLevel } from "./__fixtures__/audio-level-source.svelte"; +import LiveAudioMeter from "./LiveAudioMeter.svelte"; + +const FROZEN: AudioLevelMessage = { + source: { owner: "sidecar", identity: "card:usbaudio" }, + channels: 2, + rms_db: [-41.522344822589105, -44.116395350676385], + peak_db: [-41.522344822589105, -44.116395350676385], + floor_db: -1_000_000, +}; + +function meter(container: HTMLElement): HTMLElement | null { + return container.querySelector( + '[data-testid="live-audio-meter"]', + ); +} +function innerMeter(container: HTMLElement): HTMLElement | null { + return container.querySelector( + '[data-testid="audio-level-meter"]', + ); +} +function channelBars(container: HTMLElement): NodeListOf | undefined { + return innerMeter(container)?.querySelectorAll( + '[data-testid="audio-channel"]', + ); +} + +/** + * Deliver `count` frames at the engine's real 5 Hz cadence. Each is a DISTINCT + * object (a fresh broadcast), exactly like `subscriptions.svelte`'s handler. + */ +function pump(count: number, build: (i: number) => AudioLevelMessage): void { + for (let i = 0; i < count; i += 1) { + vi.advanceTimersByTime(200); + setAudioLevel(structuredClone(build(i))); + flushSync(); + } +} + +describe("LiveAudioMeter — a frozen feed is a DEAD feed", () => { + beforeEach(() => { + vi.useFakeTimers(); + setAudioLevel(undefined); + }); + afterEach(() => { + vi.useRealTimers(); + setAudioLevel(undefined); + }); + + it("falls to unavailable when the backend keeps re-emitting the SAME level", () => { + setAudioLevel(structuredClone(FROZEN)); + const { container } = render(LiveAudioMeter); + flushSync(); + + // Live at first — one real reading is a real reading. + expect(meter(container)?.getAttribute("data-stale")).toBe("false"); + expect(channelBars(container)).toHaveLength(2); + + // 4 s of unchanging frames at the engine cadence. + pump(20, () => FROZEN); + + expect(meter(container)?.getAttribute("data-stale")).toBe("true"); + expect(innerMeter(container)?.getAttribute("data-unavailable")).toBe( + "true", + ); + // The whole point: NO frozen bars retained. + expect(channelBars(container)).toHaveLength(0); + }); + + it("crosses over within the documented 2 s deadline, not later", () => { + setAudioLevel(structuredClone(FROZEN)); + const { container } = render(LiveAudioMeter); + flushSync(); + + pump(9, () => FROZEN); // 1.8 s — still inside the window + expect(meter(container)?.getAttribute("data-stale")).toBe("false"); + + pump(4, () => FROZEN); // 2.6 s total — past it + expect(meter(container)?.getAttribute("data-stale")).toBe("true"); + }); + + it("keeps a genuinely moving signal live for as long as it moves", () => { + setAudioLevel(structuredClone(FROZEN)); + const { container } = render(LiveAudioMeter); + flushSync(); + + // 10 s of real, changing audio. + pump(50, (i) => ({ + ...FROZEN, + rms_db: [-40 - (i % 7), -44 - (i % 5)], + peak_db: [-30 - (i % 7), -34 - (i % 5)], + })); + + expect(meter(container)?.getAttribute("data-stale")).toBe("false"); + expect(innerMeter(container)?.getAttribute("data-unavailable")).toBe( + "false", + ); + expect(channelBars(container)).toHaveLength(2); + }); + + it("recovers the moment the signal moves again", () => { + setAudioLevel(structuredClone(FROZEN)); + const { container } = render(LiveAudioMeter); + flushSync(); + + pump(20, () => FROZEN); + expect(meter(container)?.getAttribute("data-stale")).toBe("true"); + + pump(1, () => ({ ...FROZEN, rms_db: [-12, -13], peak_db: [-6, -7] })); + expect(meter(container)?.getAttribute("data-stale")).toBe("false"); + expect(channelBars(container)).toHaveLength(2); + }); + + it("leaves repeated digital silence as `silent` — a muted mic is a WORKING meter", () => { + const silence: AudioLevelMessage = { + source: { owner: "sidecar", identity: "card:usbaudio" }, + channels: 2, + rms_db: [-1e6, -1e6], + peak_db: [-1e6, -1e6], + }; + setAudioLevel(structuredClone(silence)); + const { container } = render(LiveAudioMeter); + flushSync(); + + pump(50, () => silence); + + expect(meter(container)?.getAttribute("data-stale")).toBe("false"); + expect(innerMeter(container)?.getAttribute("data-unavailable")).toBe( + "false", + ); + expect( + innerMeter(container)?.querySelector('[data-testid="audio-silent"]'), + ).not.toBeNull(); + }); + + it("keeps the engine reason on a repeated `unavailable` marker", () => { + const gap: AudioLevelMessage = { unavailable: true, reason: "no_device" }; + setAudioLevel(structuredClone(gap)); + const { container } = render(LiveAudioMeter); + flushSync(); + + pump(20, () => gap); + + // Still unavailable, but via the engine's own marker (which carries a + // reason) rather than being aged out into a reasonless stale render. + expect(innerMeter(container)?.getAttribute("data-unavailable")).toBe( + "true", + ); + expect(meter(container)?.getAttribute("data-stale")).toBe("false"); + }); + + it("still ages out a feed that stops entirely", () => { + setAudioLevel(structuredClone(FROZEN)); + const { container } = render(LiveAudioMeter); + flushSync(); + + vi.advanceTimersByTime(2_600); + flushSync(); + + expect(meter(container)?.getAttribute("data-stale")).toBe("true"); + expect(channelBars(container)).toHaveLength(0); + }); + + it("does not restamp on the engine replaying its cached level to a reconnecting subscriber", () => { + // cerastream hydrates a NEW audio-level subscriber with its cached last + // observation, and the idle sidecar has no delivery deadline — so a stalled + // sidecar's stale level is replayed on every bridge reconnect. Arrival + // stamping reset the watchdog each time; content stamping must not. + setAudioLevel(structuredClone(FROZEN)); + const { container } = render(LiveAudioMeter); + flushSync(); + + vi.advanceTimersByTime(2_600); + flushSync(); + expect(meter(container)?.getAttribute("data-stale")).toBe("true"); + + // Reconnect → hydration replay of the very same cached observation. + setAudioLevel(structuredClone(FROZEN)); + flushSync(); + + expect(meter(container)?.getAttribute("data-stale")).toBe("true"); + expect(channelBars(container)).toHaveLength(0); + }); +}); diff --git a/apps/frontend/src/lib/components/preview/LiveAudioMeter.svelte b/apps/frontend/src/lib/components/preview/LiveAudioMeter.svelte index 36d96604f..880fc7a14 100644 --- a/apps/frontend/src/lib/components/preview/LiveAudioMeter.svelte +++ b/apps/frontend/src/lib/components/preview/LiveAudioMeter.svelte @@ -12,9 +12,22 @@ states, so a missing feed shows the meter's own "unavailable" copy rather than vanishing: an operator must be able to tell "the meter says nothing is coming in" apart from "the meter isn't here". + + The staleness watchdog keys on CONTENT, not arrival — see + `audio-meter-liveness.ts` for why a frame landing is not evidence the audio path + behind it is alive. -->