Fix: CEM work session#188

Draft

Flix6x wants to merge 170 commits into

mainfrom

dev/fix-handshake-handler

Flix6x commented Feb 25, 2026 •

edited

Loading

Member

Follow-up to #187.

Fix handshake handler now sending back two responses against a queue that is periodically flushed (every 0.3 seconds)
Docker-compose stack skips running toy tutorial every time -> this is actually just for devs at the moment, because the toy account is actually needed for the CEM to run; specifically, the CEM uses the toy user credentials
Ported the FRBC.FillLevelTargetProfile and FRBC.UsageForecast to the FRBCSimple handler.
Test CEM now configures its own data dashboard in FlexMeasures
Test CEM now polls FlexMeasures much faster, which is more suitable for simulations.
A few minor fixes to the SimpleFRBC handler to get to the point that FRBCInstructions are sent back.
Ported the FRBC.LeakageBehaviour to the FRBCSimple handler.
Support wrapped S2 messages in an envelope together with metadata containing the simulation time.
Made the start of the schedule equal to the most recent storage status

Stijn van Houwelingen and others added 29 commits

February 19, 2026 16:44


          refactor: migrate from tox/PyScaffold to uv for dependency management…

733ef40

…, consolidating all dependencies into pyproject.toml and a single uv.lock file.

This simplifies and speeds up the development setup greatly.

Changes:
- Switch build backend from setuptools to hatchling
- Remove setup.cfg, tox.ini, .isort.cfg and setup.py in favour of .flake8 and pyproject.toml
- Add Poethepoet tasks
- Upgrade main Python version (CI/CD, .python-version, etc.) to 3.12
- Deprecate Python 3.9
- Add .python-version for consistent Python version management
- Create separate Dockerfile for flexmeasures-client
- Replace pip-tools with uv in all CI/CD workflows
- Remove ci/run_mypy.sh in favour of Poethepoet task
- Update documentation

Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@github.io>


          fix: fix whitespace issues

d661056

Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@github.io>


          feat: move from pre-commit git repos to using uv for pre-commit scripts

cef07f3

Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@github.io>


          dev: try using the _sending_queue instead

d63afe5

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          Revert "dev: try using the _sending_queue instead"

564b8f1

This reverts commit d63afe5.


          fix: give the sending task enough time to flush the queue before the …

2db7a4f

…connection shuts down

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          dev: use DEBUG logging level

35cc9ad

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: await message sent rather than message queued

375b49c

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: also close WS if closing CEM

14f7792

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: skip setting up the toy account

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: post prices in background task

89d4338

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          Revert "feat: skip setting up the toy account"

60ff6af

This reverts commit 4799585.


          chore: use modern method name

404ed6b

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: fall back on now in case FRBCActuatorStatus.transition_timestamp…

5d4de10

… is missing

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: actuator status unit

f8ea658

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: skip setting up the toy account

d59f340

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: copy-paste mistake

6abd2a1

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: discharge unit

5328de9

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          docs: update instruction to run docker-compose stack

eae232b

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          Revert "fix: discharge unit"

0e72c25

This reverts commit 5328de9.


          fix: schedule power sensor instead of dimensionless discharge sensor

abb0a74

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: trigger schedule with each storage status (not yet rate limited…

bbaac73

…) and make system_description_id optional

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: post 1 year of data in background tasks

374e44a

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          Revert "feat: post 1 year of data in background tasks"

a1d6ba7

This reverts commit 374e44a.


          fix: set prior knowledge of prices and test with now

f8e0d62

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: floor the schedule start

be6d8d3

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: messages should now be routed through cem.send_message

5334c0c

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: messages should now be routed through cem.send_message; update h…

…andlers accordingly

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: roll 3 days of test prices

e01176d

Signed-off-by: F.N. Claessen <claessen@seita.nl>

Flix6x self-assigned this

Flix6x and others added 30 commits

June 22, 2026 19:12


          refactor: introduce extra variables for better debugging

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: infer next schedule_duration from previous usage_forecast duration

6cfe972

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          Revert "feat: infer next schedule_duration from previous usage_foreca…

a6b0c1d

…st duration"; it works after the first schedule, but I fear it may sometimes lead to a very short schedule duration; I prefer not to make this dependent on the usage forecast duration

This reverts commit 10fa1556532e789c42e81f2fa26af18ade611257.


          dev: add debugging statements

14d21d1

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: support running multiple CEM instances on different ports

ad1cf67

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: load the seita_ems plugin into the server and worker containers

a230540

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: match returning resource managers to assets by name when externa…

04b3999

…l_id is unavailable

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: use more gunicorn worker processes instead of threads, to isolat…

dedfe9f

…e slow requests

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: add trace logging around asset reuse and schedule triggering; s…

51346d6

…top silently swallowing exceptions in the fire-and-forget storage-status task

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          chore: scale gunicorn workers to match the number of CEM instances

bc709df


          fix: relax all site-capacity constraints, not just SOC, so infeasibil…

c0ddc15

…ity degrades gracefully


          fix: place back command for running a worker against the ingestion queue

c19cf8d

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          fix: revert worker to the scheduling|forecasting queue list (ingestio…

52843ff

…n is not a valid queue on this branch)


          fix: raise the CEM's schedule-polling budget to match FlexMeasures' l…

d2805b6

…onger job timeout


          fix: relax site capacity specifically instead of the broad relax-cons…

…traints flag, which pushed HiGHS into 10+ minute solves


          revert: drop the CEM polling-timeout increase, no longer needed now t…

2d2d97c

…he real solver-slowdown cause is fixed


          dev: let py-spy attach to the RQ worker subprocess for live debugging

9aa318d

Signed-off-by: F.N. Claessen <claessen@seita.nl>


          feat: scale RQ workers to 3, one per concurrently-simulated apartment…

b3540dd

…, so scheduling jobs run in parallel instead of serializing through a single worker


          fix: retry a transient scheduling-job failure instead of permanently …

117cdac

…stalling the timestep

Observed a pandas 'cannot reindex on an axis with duplicate labels' error from a
scheduling job during a multi-day, frequently-re-triggered (implementation_horizon_hours=3)
run; re-running the exact same job manually succeeded immediately, confirming it's
transient. Without a retry, trigger_schedule's fire-and-forget task just logs and
drops the exception, so the RM never receives an instruction for that timestep and
polls forever. Retries up to 3 times on a ValueError containing 'Scheduling job
failed' before giving up for real.


          refactor: move the seita_ems plugin mounts to an overlay in the ems repo

52bfa82

The base override now runs without access to the private ems repo (needed
for single-house simulations by collaborators without that repo); community
runs add ../ems/docker-compose.community.yml as a third compose file.


          feat: pass S2 operation-mode power ranges to FlexMeasures as power bands

292b646

Builds the new flex-model "operation-modes" field from the FRBC system
description's operation-mode element power ranges (only when they actually
restrict the range), so FlexMeasures schedules only power values the device
can run at. Requires flexmeasures feat/2113-operation-mode-power-bands.


          fix: tear down the CEM session when its websocket closes

f924543

A plain disconnect (RMs reconnect at every replan boundary) previously left
the producer and watchdog tasks and the CEM itself running forever; the old
session lived on as a zombie able to keep posting SoC and triggering
schedules for an asset the RM no longer implements.


          fix: retry schedule triggers that race the initial battery-SoC post

7df1a4b

At startup, the CEM's first trigger_schedule can beat the controller's
initial battery-SoC post (a separate client writing to the flex-model's
state-of-charge sensor). FlexMeasures then rejects the trigger with a
422 "No recent state-of-charge value found", and since this whole call
runs as a fire-and-forget task, that single failure permanently stalled
the house at its barrier. Treat it as transient alongside "Scheduling
job failed" (5 attempts, 2s backoff) - it resolves itself once the SoC
post lands moments later. Heavier stacks lose this race more often.


          feat: route S2 PowerMeasurements to a measured-power sensor at simula…

aae8c23

…ted time

The CEM now creates a dedicated 'measured-power' sensor (kW,
consumption-positive) per site, separate from the StorageScheduler 'power'
schedule sensor, and maps ELECTRIC.POWER.L1 measurements to it (replacing a
hardcoded fallback sensor id). Measurements are binned and posted at their
own SIMULATED timestamp instead of wall-clock time, and posted directly
per incoming value (the RM sends one aggregated value per interval, so the
old 5-minute wall-clock buffered-averaging would have suppressed nearly
every post in a co-simulation). Values are posted in Watts (S2
PowerMeasurement convention) and converted to the kW sensor by FlexMeasures.


          fix: give control-type handlers a default close() so CEM teardown can…

e44619f

…'t crash

CEM.close() calls close() on every registered control-type handler when a
websocket tears down, but only FRBCTunes implemented it - FRBCSimple (and any
other handler) raised AttributeError, which killed the CEM's request handler
and left it unable to serve the next RM connection (houses then hang forever
waiting for a schedule). Add a default no-op close() on the ControlTypeHandler
base class.


          fix: install and run the CEM with one and the same Python interpreter

80a7e51

Images built from FlexMeasures' uv-based Dockerfile put a pip-less
/app/.venv first on PATH while bare `pip` is the system one under
/usr/local, so packages installed at container start were invisible to
the runtime interpreter (ModuleNotFoundError: aiohttp) on any freshly
built image - notably Samuel's Windows setups - while long-cached
single-Python images kept working. Use `python3 -m pip` throughout
(with an ensurepip bootstrap for pip-less venvs) and exec the server
with that same python3. Also set the generic
SETUPTOOLS_SCM_PRETEND_VERSION alongside the client-specific one for
the git-less editable install.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki


          fix: stop deriving a phantom production capacity from S2 operation modes

d409b51

An S2 PowerRange runs from start_of_range (lower bound) to end_of_range
(upper bound), but the capacity derivation introduced in 9f20ab4 (March
2026) read them the other way around: production-capacity became the
smallest upper bound across operation modes (e.g. 2.5 kW for a
consumption-only heat pump) and consumption-capacity the largest lower
bound. The scheduler consequently discharged the heat pump's thermal
storage as if it were an electric battery, and capped its consumption
below the device's real maximum. Derive the overall [min, max] power
range instead, with production only where ranges extend below zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki


          Fix CEM co-sim belief time, timezone anchor, and live aggregate-power…

a7f154a

… realizations

Belief time (defect 4b): PowerMeasurements posted by the CEM now carry a
simulated prior (the binned interval's end), so realized apartment power
is stamped at simulation time instead of wall-clock 2026, and the UI
horizon view no longer shows data "recorded in 2026".

Timezone (defect 2): naive S2 measurement timestamps are now localized as
Europe/Amsterdam (was UTC), matching the community orchestrator, RM and
controller which all localize the same naive sim times as Europe/Amsterdam.

Live apartment realizations (defect 4a): besides the dedicated
measured-power sensor, the CEM now also mirrors the realized apartment
aggregate onto the apartment's flex-context "aggregate-power" sensor,
resolved lazily from the asset's flex_context (it is attached by the
community runner only after the CEM connects, so it appears from step 1).
Realized posts stay distinguishable from the StorageScheduler schedule
data on that same sensor by their own source and simulated belief time.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki


          fix: stamp power measurements with zero belief horizon

cfd562e

The RM sends one PowerMeasurement per 15-minute interval (stamped at
interval start), but the posting path binned and stamped priors with
the 5-minute _minimum_measurement_period, yielding belief times 10
minutes before each event's end - an ex-ante horizon on measurements.
Bin and stamp at the measured-power sensor's 15-minute event resolution
instead, so each realized value is believed exactly when its interval
elapses (zero horizon).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki


          fix: hard-bound the FRBC heat pump against production

89b5893

An explicit production-capacity of 0 kW gets softened by FlexMeasures'
relax-constraints defaults (100 EUR/kW device breach price), so under a
tight community capacity the scheduler chose to 'produce' 1.764 kW from
a heat pump rather than pay the 10000 EUR/kW site breach. Setting
is_strictly_non_positive on the FRBC power sensor keeps production
hard-bounded at zero regardless of relaxation settings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LQFkeGxDwErydUV5ZmHmki

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S2