Skip to content

Fix EPG scheduler storm, dead image host, add plugin.json - #4

Open
kilo-WATT wants to merge 2 commits into
jesmannstl:mainfrom
kilo-WATT:fix/scheduler-storm-and-plugin-key-lookup
Open

Fix EPG scheduler storm, dead image host, add plugin.json#4
kilo-WATT wants to merge 2 commits into
jesmannstl:mainfrom
kilo-WATT:fix/scheduler-storm-and-plugin-key-lookup

Conversation

@kilo-WATT

@kilo-WATT kilo-WATT commented Aug 5, 2026

Copy link
Copy Markdown

Fixes #3

Summary

Two independent bugs found while debugging why a live Dispatcharr instance's Jellyfin Live TV guide had stopped getting fresh data and thumbnails, both fixed in plugin.py / zap2xml.py.

1. EPG auto-refresh scheduler storm — last_generated never persists

_create_or_update_epg_source() looks up its own settings row with:

plugin_key = __name__.split(".")[0]

Under Dispatcharr's real plugin loader, this module's __name__ is _dispatcharr_plugin_zap2xml.plugin, so plugin_key evaluates to _dispatcharr_plugin_zap2xml — not zap2xml, the actual PluginConfig.key (see the existing _plugin_key() helper, which gets this right and is already used elsewhere). The lookup silently returns None every time, so last_generated is never written, even after a fully successful fetch + parse.

Because last_generated never persists, _scheduler_loop's should_run check always evaluates true. Dispatcharr runs multiple worker processes (gunicorn + celery), each with its own independent scheduler thread and no cross-process coordination (_SCHEDULER_LOCK only dedupes within one process), so every worker fires download_epg on every ~60s tick, forever. Concurrent runs then race on the shared workspace/output file — one run's cleanup step can delete the produced XML out from under another run's copy-to-/data/epgs step, so the copy silently fails (caught by a bare except Exception, never logged), the EPGSource never gets refreshed, and the guide quietly runs out of programme data days later, while Gracenote's API gets hammered roughly once a minute in the meantime.

Fix: reuse _plugin_key() instead of the broken __name__-based lookup, and add a flock()-based cross-process lock (_SCHED_LOCK_PATH) around the scheduler's run, so only one worker process across the deployment ever executes download_epg at a time (with a re-check of "still due" after acquiring the lock, to skip a redundant back-to-back run at the interval boundary).

2. Dead dshm.tmsimg.com image host (fixes #3)

Gracenote retired the dshm.tmsimg.com image CDN subdomain (NXDOMAIN, confirmed against 1.1.1.1), so every <programme> <icon> in the generated XMLTV points at a dead host — Jellyfin logs thousands of Unable to pre-cache warnings per guide refresh and no thumbnails load. This is a recurring problem — the code already migrated once before (zap2it.tmsimg.comdshm.tmsimg.com).

Fix: swap the two hardcoded dshm.tmsimg.com references (~line 481, ~line 499) to demo.tmsimg.com, which resolves and serves the identical /assets/pXXXX_*.jpg paths (verified curl -I200 OK).

3. Add plugin.json manifest

The plugin ships with no manifest, so Dispatcharr flags it as legacy ("Please update or ask the developer to add plugin.json"). Added one generated from the plugin's own fields/actions, which clears the warning. Bumped plugin.py's version 2.1.4 → 2.1.5 to match.

Test plan

Verified end-to-end against a live Dispatcharr instance with real Zap2it/Gracenote credentials:

  • Scheduler storm — before: last_generated stayed empty indefinitely, Auto-download tick… running download_epg fired roughly every 60 seconds continuously, /data/epgs/<file>.xml never updated past its initial creation date despite the underlying fetch itself succeeding repeatedly. After: exactly one tick fires per configured interval, last_generated is correctly persisted, and the EPG source's programme data ingests successfully and advances (confirmed via epg_programdata row counts / max end_time).
  • Image hostdemo.tmsimg.com has been running in production for about a week; thumbnails load correctly in Jellyfin's Live TV guide, no more Unable to pre-cache warnings.
  • plugin.json — clears the legacy-plugin warning in Dispatcharr's plugin UI with no other behavior change.

Diffed and reviewed manually (this repo ships the plugin as zap2xml.zip, so a raw git diff on the zip isn't reviewable in GitHub's UI — all behavioral changes are described in full above; channels.db and __init__.py are untouched).

🤖 Assisted by Claude Code

Two bugs in plugin.py compound to make the auto-refresh scheduler
effectively broken from the first install:

1. _create_or_update_epg_source() looked up its own PluginConfig row
   with `__name__.split(".")[0]`. Under Dispatcharr's real plugin
   loader the module's __name__ is "_dispatcharr_plugin_zap2xml.plugin",
   so this evaluated to "_dispatcharr_plugin_zap2xml" instead of the
   actual config key "zap2xml" (see _plugin_key()). The lookup always
   returned nothing, so `last_generated` was never persisted, even on
   a fully successful run.

2. Because `last_generated` was never set, _scheduler_loop's should_run
   check always concluded a refresh was overdue. Combined with the
   fact that Dispatcharr runs multiple worker processes (gunicorn +
   celery), each with its own independent scheduler thread and no
   cross-process coordination, every worker fired download_epg on
   every ~60s tick, forever. Concurrent runs then raced on the shared
   workspace/output file (one run's cleanup step could delete the
   produced XML out from under another run's copy-to-/data/epgs step),
   so the copy silently failed and EPG data quietly went stale while
   Gracenote got hammered continuously.

Fixes:
- Reuse the existing _plugin_key() helper instead of the broken
  __name__-based lookup, so last_generated actually gets recorded.
- Add a flock()-based cross-process lock (_SCHED_LOCK_PATH) around
  the scheduler's run, so only one worker process executes
  download_epg at a time, with a re-check of "still due" after
  acquiring the lock to avoid a redundant back-to-back run.

Verified end-to-end against a live Dispatcharr instance: prior to
the fix, last_generated was never set and the scheduler fired every
~60s indefinitely, hammering Gracenote's API and never actually
completing the copy+ingest step reliably. After the fix, exactly one
tick fires per interval, last_generated is correctly persisted, and
guide data ingests successfully into the EPG source.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Folded into this PR rather than a separate one, since both bugs were
found while debugging the same broken guide-data pipeline:

- zap2xml.py: swap the two hardcoded dshm.tmsimg.com references to
  demo.tmsimg.com (dshm no longer resolves — NXDOMAIN; demo serves
  the identical /assets/pXXXX_*.jpg paths, verified HTTP 200).
- Add plugin.json (generated from the plugin's own fields/actions),
  clearing Dispatcharr's "legacy plugin" warning.
- Bump plugin.json's version to 2.1.5 to match plugin.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kilo-WATT kilo-WATT changed the title Fix EPG auto-refresh scheduler storm — last_generated never persists Fix EPG scheduler storm, dead image host, add plugin.json Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: Program thumbnails broken: dshm.tmsimg.com image host no longer resolves (+ add plugin.json)

1 participant