Skip to content

feat(chat): render Twitch subs, gifts, raids, announcements and cheers - #11

Merged
Blackwellcode merged 8 commits into
masterfrom
feat/twitch-chat-fidelity
Nov 14, 2026
Merged

Blackwellcode merged 8 commits into
masterfrom
feat/twitch-chat-fidelity

Conversation

@Blackwellcode

@Blackwellcode Blackwellcode commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

Summary

VOD replay and live chat now show Twitch chat events almost 1:1: subs, resubs, gifted subs, community gifts, pay-forwards, raids, announcements (with colour), bits/cheers, highlighted messages, first messages, replies, power-ups and channel point rewards.

Chat preview

Rendered from chat_emotes.js + style.css with one case each: resub, mystery gift, subgift, raid, BLUE announcement, cheer + first message, reply, highlight, reward, timeout, plus a normal line for comparison.

Changes per file

File Change
recorder/chat_events.py irc_command(line) skips tags and prefix and returns the real command. parse_usernotice(tags, message) maps the msg-param-* tags into a {"type": "notice", "kind": ...} event (sub/resub, subgift/anonsubgift, submysterygift, giftpaidupgrade, pay-forward, raid, bitsbadgetier, announcement, viewermilestone, charitydonation, sharedchatnotice → source-msg-id, anon, system). sanitize_event validates notices with a strict allowlist: kind ^[a-z0-9_-]{1,40}$, plan/color from fixed sets, ints > 0 and capped, text fields whitespace-normalised and cut to 80 characters (system-msg 300), unknown fields dropped. parse_privmsg_meta / sanitize_meta handle bits, highlight, first, reply {user, msg ≤ 140} and power.
recorder/Streamrecorder.py The IRC loop routes on irc_cmd == 'PING' / 'CLEARCHAT' / 'CLEARMSG' / 'USERNOTICE' / 'PRIVMSG'. A message text like CLEARCHAT #x is no longer misrouted. USERNOTICE is written through append_chat_log (live log + session log with rel), with emotes, badges and colour taken from process_twitch_message. PRIVMSG entries get an optional meta.
core/tts_bot.py Second IRC reader: parse_privmsg now checks the command token instead of " PRIVMSG " in rest, so a USERNOTICE whose text contains "PRIVMSG" no longer triggers TTS.
WEBUI/webui.py format_notice_text(event, user) builds Twitch-style headlines with Flask-Babel (gettext/ngettext) and falls back to event.system for unknown kinds. chat_item_from_row sets event.text, never sends system, and passes meta through. The duplicate token for notices includes kind and the notice text, so several gift subs from one gifter to different recipients stay separate. VOD chat chunks (?chunk=N) used to send the raw JSONL rows unfiltered. They now go through chat_item_from_row with the same duplicate check, served as private, max-age=600 with Vary: Cookie, Accept-Language because the text depends on the locale.
WEBUI/static/chat_emotes.js renderNoticeHtml (header with an SVG icon per group sub/gift/raid/announcement/bits/other, then badges, name in the user's colour and the message with emotes). chatLineClassName adds `chat-line-notice[-
WEBUI/static/style.css Notices: 4px #9147ff left border on a light purple background. Announcement border-image gradients: BLUE #00d6d6→#9146ff, GREEN #00db84→#57bee6, ORANGE #ffb31a→#e0e000, PURPLE/PRIMARY #9146ff→#ff75e6. Highlight #755ebc, first message, reply context, cheer tiers, gigantified/animated power-ups.
WEBUI/templates/vod.html, live.html, mobile_player.html, twitch_watch_chat.js.html, chat_obs.html Call decorateLine after .chat-msg is filled and use chatEventKey in the duplicate keys. The VOD text filter also searches event.text, and notices stay visible under badge filters. chat_obs.html gets the notice classes and inline styles. It also fixes two bugs on lines this PR already touches: renderBadgesHtml got the array instead of the item, and userColor got the user name, so the overlay showed no badges and no name colours.
WEBUI/i18n.py, WEBUI/translations/* New JS strings registered in JS_MESSAGES. 26 new msgids are translated in all 11 catalogs (pl with 3 plural forms; ja, ko, zh_Hans and tr with 1). .po and .mo are committed. Catalogs were updated with --no-wrap to match the existing unwrapped .po files; the .pot uses the standard extract.
tests/test_chat_events.py 28 new tests (see below).
docs/screenshots/chat_twitch_events.png Screenshot above.

Limits

  • Channel points rewards WITHOUT text input do not arrive over IRC. They would need EventSub channel.channel_points_custom_reward_redemption.add with broadcaster authorisation. This PR only documents that and does not implement it. Rewards with text input keep working as before (custom-reward-id on PRIVMSG).
  • Existing VOD logs contain no USERNOTICE data. The new display only applies to recordings made after deploying the recorder change. Old logs render as before.
  • Reply messages keep Twitch's leading @parent mention in the body. Removing it would shift the native emote offsets.
  • Moderation lines (format_moderation_message) are still German strings from the recorder. That predates this PR and is unchanged here.

Test plan

  • python -m unittest discover -s tests -p "test_*.py" → Ran 206 tests, OK (skipped=1), exit 0 (178 existing + 28 new)
  • python -m compileall -q core recorder WEBUI bin run_transcribe.py tests → exit 0
  • uvx ruff@0.17.3 check . → All checks passed!
  • node --check WEBUI/static/chat_emotes.js → exit 0
  • python -m py_compile on every changed .py file → exit 0
  • python scripts/check_translations.py → 11 locales, 1109/1109 translated, 0 problems each
  • Visual check: a temporary local page (deleted afterwards) loaded chat_emotes.js + style.css; screenshot above, no console errors

New tests cover: irc_command (USERNOTICE; PRIVMSG with "CLEARCHAT #x" in the text; PING), resub with message and streak, subgift with sender_count, anonymous submysterygift, raid (displayName/viewerCount), announcement colour, unknown kind → system-msg, invalid kind → None, sanitize_event dropping bad plan/color/negative numbers/foreign fields, parse_privmsg_meta (bits/first/highlight/reply; first-msg=0 → None), the TTS bot routing, process_twitch_message on USERNOTICE, and API tests where two gift subs from one gifter return 2 items with the recipients in event.text, no system, and meta passed through (range API and chunk API).

Design pass (commit 50f9ad4)

Checked in all five themes (vault, aurora, cyber, ember, ocean), the VOD/live grid at 360 px, mobile at 375 px and the OBS overlay.

  • Twitch colours are now --sv-* tokens; text colours come from the theme (--tube-*).
  • Contrast (WCAG AA): Twitch's bits tier colours failed as text on dark backgrounds (tier 100 2.58:1, 10000 3.20:1, 5000 4.22:1, 1 4.34:1). The text tints are the same hues lightened to at least 4.5:1. Measured in the browser on rendered colours, worst case over all themes: notice text 17.37, reply context 8.10, first-message label 7.19, bits 6.09, highlight text 12.96.
  • The reply context uses --tube-text-2 instead of --tube-muted (3.78:1 in cyber).
  • Notice headers align with the timestamp. The templates' empty badge and name slots added an empty line above the header.
  • A notice without badges no longer renders an empty badge wrapper, which caused a stray indent.
  • OBS overlay: same tokens, plus first message, power-ups and the points/timeout pills that rendered as plain text.

Merge with master (commit 8083a46)

  • webui.py: keeps master's blueprint import, and chat_vod now registers on vod_bp with the sanitized chunk serving.
  • de/messages.mo is rebuilt.
  • The catalogs are re-synced. Two login messages that master changed had stale translations in 10 locales; they are now translated.
  • After the merge: Ran 338 tests, OK (skipped=20), ruff and compileall exit 0, check_translations 0 problems.

🤖 Generated with Claude Code

Pierre and others added 8 commits November 11, 2026 18:29
Capture USERNOTICE in the recorder and show it in VOD replay and live
chat close to the Twitch web chat.

- irc_command() routes IRC lines on the real command token; message text
  such as "CLEARCHAT #x" is no longer misrouted (recorder and TTS bot).
- parse_usernotice() maps sub/resub/subgift/mystery gifts/pay-forward/
  raid/bits badge/announcement/milestone/charity into a strictly
  allowlisted "notice" event; sanitize_event() validates it.
- parse_privmsg_meta()/sanitize_meta() keep bits, highlighted message,
  first message, reply parent and power-ups as optional "meta".
- API adds a localized event.text (Flask-Babel, 11 catalogs), never sends
  Twitch's raw system-msg, passes meta through, and includes kind + text
  in the duplicate token so several gift subs stay separate.
- VOD chat chunks now go through the same sanitizer as the range API.
- chat_emotes.js: notice renderer with icons, bits pills in Twitch tier
  colours, meta line classes, decorateLine() for reply/first context;
  wired into vod, live, mobile, Twitch watch and the OBS overlay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WEBUI.webui runs init_db() only on its first import. When another test
module imports it first inside a patched temp base, the tables land in
that (later deleted) database and ChatApiTests hit an empty one:
"sqlite3.OperationalError: no such table: users" in reversed order.

Give each ChatApiTests test its own base directory and an initialized
database, like StorageTestBase. No assertion changed (78 before/after).

Verified: full suite 206 tests OK (skipped=1) in normal and reversed
order; storage -> ChatApiTests and reliability -> ChatApiTests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Share button built its onclick handler with
shareClip('{{ request.args.get('clip_name') }}'). Jinja HTML-escapes the
quote to &#39;, but the browser decodes attribute entities before the JS
runs, so a crafted ?clip_ok=1&clip_name= link could break out of the
string and run script when a signed-in user clicked Share.

Render the value with |tojson inside a single-quoted attribute, so the
argument is always one JS string literal. A double-quoted attribute
would be cut short by tojson's double quotes; the new test caught that.

ClipEditorShareXssTests requests the editor with five payloads, decodes
the onclick attribute as a browser does and requires the single argument
to equal the payload. Red before the fix (errors=5), green after; full
suite 207 tests OK (skipped=1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Twitch colours become --sv-* tokens; text colours come from the theme.
- Bits text tints lightened to >= 4.5:1 on all five themes and tints
  (tier 100 was 2.58:1, 10000 3.20:1, 5000 4.22:1, 1 4.34:1).
- Reply context uses --tube-text-2 instead of --tube-muted (3.78:1 in cyber).
- Notice lines hide the empty badge/name slots, so the header aligns
  with the timestamp instead of sitting below an empty line box.
- Highlight, first message and notices share stripe and radius tokens.
- Notice body omits an empty badge wrapper (stray indent).
- OBS overlay: same tokens, plus first message, power-ups and the
  points/mod pills that rendered as plain text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts:
- WEBUI/webui.py: keep master's BlueprintSetupState import and the new
  chat_items_from_chunk_file(); chat_vod now registers on vod_bp.
- messages.mo (de): rebuilt from the merged messages.po.

The catalogs were re-synced with the merged sources (pybabel update).
Two login messages that master changed had stale translations in 10
locales; they are translated, check_translations reports 0 problems.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…elity

Conflicts:
- WEBUI/static/style.css: master reformatted the neighbouring chat rules;
  the Twitch chat block (--sv-* tokens) is kept unchanged. The tokens it
  uses (--tube-text, --tube-text-2) still exist on master.
- Catalogs (.po/.pot/.mo, 11 locales): merged per message. Master's
  catalogs are the base (new UI strings from PR #13); entries missing
  there come from this branch (Twitch chat notices). Then extract,
  update --ignore-obsolete (master keeps no obsolete entries) and compile.
  check_translations: 1132/1132 in every locale, 0 problems.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Blackwellcode
Blackwellcode merged commit 15914f6 into master Nov 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant