Reports as jobs and as automations#2297
Draft
Flix6x wants to merge 6 commits into
Draft
Conversation
- Activate the reporting queue (it was prepared but commented out), including worker help texts and queue cleanup. - Reporters accept as_job: a job is queued (with trigger meta data) that rebuilds the reporter from its data source, computes the report and saves the results to the database. - `flexmeasures add report --as-job` queues such a job; reporting jobs show up in the asset's jobs overview (status page and API). Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Automations can now compute reports on a recurring basis: - `flexmeasures add automation --type reports --reporter <class>` stores the reporter config on a data source (steady across runs, so all report results attribute to the same source) and validates the report parameters. - The report window resolves freshly on each run: 'start-offset'/'end-offset' fields (comma-separated Pandas offsets, applied to the run time in the first output sensor's timezone) express a rolling window, and without any timing fields the window defaults to the last cron period (from the previous cron fire time until the run time). Absolute start/end still work, but draw a warning. - The API creation field 'forecaster' is generalized to 'generator' (also accepting reporter classes), and the UI's New automation modal gains data generator and config fields; the Reports tab is now enabled. Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Documentation build overview
34 files changed ·
|
Each automation run is recorded in Redis; a report automation without timing fields then reports on the period since its actual last run, falling back to the last cron period when no last run is known (e.g. on the first run, or after a Redis flush). This gives gapless coverage even when runs are missed. Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Consolidates the shared automations concept (model, lifecycle, runner deployment, provenance) into documentation/features/automations.rst, with the per-feature pages linking to it and keeping only their type-specific parameter semantics. Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
This was referenced Jul 11, 2026
- API automation creation now checks that the caller may read every sensor referenced in the parameters/config and record data on the sensors the automation writes to, closing a cross-account data read/write hole. - `add report --as-job` implies --save-config (the worker rebuilds the reporter from its data source, so jobs without stored config always crashed). - Report automation parameters are validated with the chosen reporter's own parameters schema, not the base schema. - Invalid start-offset/end-offset strings are rejected at creation instead of being silently skipped at run time (which yielded empty report windows). - run_report_job wipes the shared cached reporter's parameter state, like the automation runner already did, so consecutive jobs in one worker process don't pollute each other. - Default report windows now anchor to the end of the last *successfully* covered window, recorded by the reporting job upon success — failed jobs no longer create permanent reporting gaps, and the enqueue-time minute-rollover gap is gone (the recorded anchor is the window end itself). - The cron-period fallback window is computed in the platform timezone, matching how the runner decides when automations fire. - Job stats for schedules automations also scan flex-model device sensors (which may belong to child assets), so failed per-device jobs show up. - The trigger provenance kwarg is excluded from the job cache hash, so identical schedule requests from different origins dedupe again. Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Milestone 4 of #2288 (stacked on #2294): reports can run as background jobs, and automations can compute them on a recurring basis.
Reports as jobs
run-workersupport and queue cleanup.as_job: a job is queued that rebuilds the reporter from its data source, computes the report and saves the results.flexmeasures add report --as-jobuses it; reporting jobs appear in the asset's jobs overview with their Created Via provenance.Reports as automations — rolling window design
Following the data-generator philosophy: the reporter class + config live on a steady data source (same ID, immutable row, pinned via the automation's
generator_id), while the automation'sparametersJSONB holds the per-run parameters whose timing resolves freshly on each run:start-offset/end-offsetfields (comma-separated offsets incl.DBday-begin, e.g."-1D,DB"), applied to the run time in the first output sensor's timezone — exact parity withadd report --start-offset.start/endstill validate but draw a warning (each run would compute the same period).Also
forecasteris generalized togenerator(accepts forecaster or reporter classes); the UI's New automation modal gains Data generator and config fields, and the Reports tab is enabled.How to test
Tests:
pytest flexmeasures/cli/tests/test_automations.pyincludes a full roundtrip (create → runner queues reporting job with provenance → worker computes and saves the report, values verified).🤖 Generated with Claude Code
https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX