Skip to content

feat(keyer): lightweight GStreamer graphics keyer (half CPU, full 30fps, no CEF) - #43

Draft
guycochran wants to merge 2 commits into
masterfrom
feat/gst-keyer
Draft

guycochran wants to merge 2 commits into
masterfrom
feat/gst-keyer

Conversation

@guycochran

Copy link
Copy Markdown
Owner

The keyer, finally cheap. The cbcrc html5-keyer composites graphics via headless Chromium (CEF). Without GPU-accelerated CEF it software-renders (SwiftShader): ~225% CPU / ~13-17fps for a simple lower-third.

Tried the obvious fix first — a GTX 970 was installed and passed through (nvidia-container-toolkit set up, GPU visible in containers ✅). But the CEF image hardcodes --use-gl=angle --use-angle=swiftshader-webgl in its compiled libgstcef.so — no env override — so the GPU stayed at 0% util, CPU still 225%. Confirmed dead end without rebuilding their image.

This PR replaces the keyer for the common case (PNG / logo / lower-third) with a plain GStreamer pipeline — no browser, no GPU needed:

mxlsrc(program) → videoconvert → gdkpixbufoverlay(PNG+alpha) → videoconvert → mxlsink

Drop-in on the keyer I/O contract (reads the program flow, writes the keyed flow).

HW-measured on the $400 box:

  • Full 30fps (103 grains/3s) vs CEF's ~13fps
  • ~1 core (105% CPU) vs CEF's 2+ cores
  • Real composited video verified (frame dump: live program + overlay)

For animated/HTML graphics the CEF keyer stays the tool; this is the cheap 90% case (logos, bugs, static/semi-static lower-thirds). +5 tests.

Next step (not in this PR): wire it into the control plane (mxl-routes) + a service so it replaces the bypassed CEF keyer on the live box, with a real lower-third PNG.

🤖 Generated with Claude Code

guycochran and others added 2 commits October 9, 2026 19:25
…l 30fps, no browser

The cbcrc html5-keyer composites via headless Chromium (CEF). Without GPU-accelerated CEF
it falls back to SwiftShader software rendering: ~225% CPU (2+ cores) at only ~13-17fps.
And the CEF image HARDCODES --use-gl=angle --use-angle=swiftshader-webgl in its compiled
libgstcef.so, so even a GPU doesn't help (confirmed on a GTX 970: GPU stays 0%, CPU 225%).

tools/gst-keyer.py replaces it for the common case (PNG/logo/lower-third overlay) with a
plain GStreamer pipeline, no browser:
  mxlsrc(program) -> videoconvert -> gdkpixbufoverlay(PNG+alpha) -> videoconvert -> mxlsink
Drop-in on the keyer I/O contract (reads program flow, writes keyed flow).

HW-measured on the 00 box: 103 grains/3s = full 30fps, ~105% CPU (1 core, HALF of CEF),
real composited video verified (frame dump shows live program + overlay). For animated/HTML
graphics the CEF keyer remains the tool; this is the cheap 90% case. +5 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwSKEZZz4yEc2CGq5hcTmT
…service (lower-third ON air)

- tools/make-lower-third.py: renders a lower-third PNG (title/subtitle, cyan stripe, alpha)
  with pycairo (present in the keyer image) — no PIL, no browser. The customizable 'Skin'
  layer; adopters edit text/colors.
- Service wiring (on the box, box-specific so not all committed): mxl-gst-keyer.service runs
  run-gst-keyer.sh (selector -> gdkpixbufoverlay(PNG) -> keyer-out 86efffa4); its
  ExecStartPost points the encoder at the keyed output so the lower-third reaches WebRTC.
  Ordered After=mxl-pgm-heal so it locks onto a settled selector; it owns the encoder
  wiring (last-writer-wins converges to KEYED even if pgm-heal picks bypass first).

HW-verified end-to-end: lower-third 'MXL SWITCHER / Live on one $400 box' composited over
live program, flowing through WebRTC to viewers; survives service restarts (ExecStartPost
re-wires). 30fps, ~1 core. +test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwSKEZZz4yEc2CGq5hcTmT
@guycochran

Copy link
Copy Markdown
Owner Author

Wired into a service with a real lower-third — ON AIR ✅

Deployed on the box and HW-verified end-to-end: the lower-third 'MXL SWITCHER / Live on one $400 box' (cairo-rendered, cyan stripe, alpha) is composited over live program and flowing through WebRTC to viewers at 30fps / ~1 core.

Added:

  • tools/make-lower-third.py — pycairo PNG generator (title/subtitle/colors = the Skin layer, no PIL/browser).
  • Box service wiring: mxl-gst-keyer.service runs the gst-keyer (selector → gdkpixbufoverlay(PNG) → keyer-out 86efffa4); its ExecStartPost points the encoder at the keyed output. Ordered After=mxl-pgm-heal and it owns the encoder wiring (last-writer-wins → converges to KEYED).

Durability verified: survives gst-keyer service restarts and pgm-heal runs (ExecStartPost re-wires the encoder to keyed). The gst-keyer owns its wiring rather than pgm-heal orchestrating it (avoids the restart-ripple fragility).

For animated/HTML graphics the CEF keyer stays the tool; this covers the common static/semi-static lower-third + logo case at a fraction of the CPU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant