Balance internal commodity nodes (heat/steam networks) via first-class balance groups

documentation/changelog.rst

@@ -22,6 +22,8 @@ New features
 * Let storage scheduling infer missing ``power-capacity`` from directional device capacities before falling back to site capacity, and default the missing opposite capacity to zero when only a non-zero ``consumption-capacity`` or ``production-capacity`` is configured [see `PR #2222 <https://www.github.com/FlexMeasures/flexmeasures/pull/2222>`_]
 * Support multiple feeders to a shared storage [see `PR #2001 <https://www.github.com/FlexMeasures/flexmeasures/pull/2001>`_, `PR #2321 <https://www.github.com/FlexMeasures/flexmeasures/pull/2321>`_, `PR #2322 <https://www.github.com/FlexMeasures/flexmeasures/pull/2322>`_ and `PR #2325 <https://www.github.com/FlexMeasures/flexmeasures/pull/2325>`_]
 * The flex-context can now define multiple commodities, each specifying their own prices and grid capacities [see `PR #1946 <https://www.github.com/FlexMeasures/flexmeasures/pull/1946>`_, `PR #2172 <https://www.github.com/FlexMeasures/flexmeasures/pull/2172>`_, `PR #2235 <https://www.github.com/FlexMeasures/flexmeasures/pull/2235>`_ and `PR #2271 <https://www.github.com/FlexMeasures/flexmeasures/pull/2271>`_]
+* Support commodity-converting devices, such as a CHP, e-boiler or heat pump, by describing each of the device's commodity ports as a flex-model entry sharing one ``coupling`` group with fixed flow ratios (``coupling-coefficient``) [see `PR #2218 <https://www.github.com/FlexMeasures/flexmeasures/pull/2218>`_]
+* Commodities without energy prices in the flex-context (e.g. a heat or steam network without a grid connection) are scheduled as internal nodes whose devices must balance each other at every time step [see `PR #2289 <https://www.github.com/FlexMeasures/flexmeasures/pull/2289>`_]
 * Commodity contexts that omit grid-connection fields (prices and site capacities) now get smart defaults instead of failing or silently leaving the grid unconstrained — for instance, a bare ``{"commodity": "gas"}`` is treated as having no grid connection; see :ref:`commodity_context_defaults` for the full rules [see `PR #2272 <https://www.github.com/FlexMeasures/flexmeasures/pull/2272>`_]
 * CLI support for adding/editing account attributes [see `PR #2242 <https://www.github.com/FlexMeasures/flexmeasures/pull/2242>`_]
 * Improve chart axis domain for event values not around zero, with a per-sub-chart ``y-axis`` option in ``sensors_to_show`` (default ``zero``, which pads the axis out to include zero) that can be set to ``data`` to fit a sub-chart's y-axis to the values shown, to an explicit ``[min, max]`` domain that the axis will cover at least (expanding to fit data beyond it), or to a strict ``{"min": min, "max": max}`` domain that the axis will never exceed (clamping data beyond it, with a warning when that happens), editable from the graph editor [see `PR #2244 <https://www.github.com/FlexMeasures/flexmeasures/pull/2244>`_]

documentation/features/scheduling.rst

@@ -41,6 +41,10 @@ The ``flex-context`` is independent of the type of flexible device that is optim
 With the flexibility context, we aim to describe the system in which the flexible assets operate, such as its physical and contractual limitations.
 For multi-commodity scheduling problems, the flex-context can be defined separately per commodity (e.g. electricity and gas). See :ref:`tut_multi_commodity` for a hands-on example.
 
+A commodity that defines no energy prices in the flex-context (e.g. a heat or steam network without a grid connection) is treated as an internal node:
+its devices must balance each other at every time step, so everything produced into the node is consumed from it within the same time step.
+Devices that convert between commodities (such as a CHP unit, gas boiler or electric heater) are described in the flex-model, one entry per commodity port, tied together by a ``coupling`` group. See :ref:`tut_converters` for a worked example flex-model.
+
 Fields can have fixed values, but some fields can also point to sensors, so they will always represent the dynamics of the asset's environment (as long as that sensor has current data).
 The full list of flex-context fields follows below.
 For more details on the possible formats for field values, see :ref:`variable_quantities`.

documentation/tut/multi-commodity.rst

@@ -257,6 +257,34 @@ The commitment-cost result keeps these as separate entries — ``electricity net
 
 .. note:: This same pattern extends to more devices and more commodities. Add further entries to the ``flex-model`` list (each with its ``commodity``) and a matching entry in the ``flex-context`` ``commodities`` list. As long as all commodities share one currency, FlexMeasures optimises them together and reports each commodity's cost on its own.
 
+
+.. _tut_converters:
+
+Converters between commodities
+==============================
+
+A **converter** turns one commodity into another — a CHP unit (gas → electricity + steam), a gas boiler (gas → heat) or an electric heater (electricity → heat).
+In the ``flex-model`` a converter is not a single device but **one entry per commodity port**, tied together by a shared ``coupling`` name.
+The ``coupling-coefficient`` on each port fixes the conversion ratio relative to the shared coupling variable, so the ports always move together.
+
+For example, a CHP that turns gas into steam and electricity:
+
+.. code-block:: json
+
+    [
+      {"sensor": 6, "commodity": "gas",         "coupling": "chp", "coupling-coefficient": 1.0, "power-capacity": "20 kW", "production-capacity": "0 kW"},
+      {"sensor": 7, "commodity": "steam",       "coupling": "chp", "coupling-coefficient": 0.5, "power-capacity": "1 MW", "consumption-capacity": "0 kW"},
+      {"sensor": 8, "commodity": "electricity", "coupling": "chp", "coupling-coefficient": 0.3, "power-capacity": "1 MW", "consumption-capacity": "0 kW"}
+    ]
+
+Here each kW of gas input produces 0.5 kW of steam and 0.3 kW of electricity.
+The gas port is import-only (``production-capacity: 0 kW``) and the steam and electricity ports are export-only (``consumption-capacity: 0 kW``).
+
+**Internal nodes.** A commodity that lists no energy price in the ``flex-context`` (e.g. a steam or heat network with no grid connection) is treated as an **internal node**: its devices must balance each other at every time step, so everything converters produce into the node is consumed from it within the same step.
+Give such a commodity only an ``inflexible-device-sensors`` entry (its fixed demand), or omit it from the ``flex-context`` entirely.
+
+This is how a whole factory is scheduled end-to-end: an e-heater and a boiler feed an internal ``heat`` node, a steamer converts heat into an internal ``steam`` node, a CHP also feeds steam (while exporting electricity to the grid), and a fixed steam demand closes the balance — priced commodities (electricity, gas) at the grid, unpriced commodities (heat, steam) balanced internally.
+
 We hope this demonstration helped to illustrate multi-commodity scheduling.
 To revisit scheduling several devices that share a single commodity and stock, head back to :ref:`tut_multi_feed_storage`.
 Next, in :ref:`tut_toy_schedule_process`, we'll turn to something different: the optimal timing of processes with fixed energy work and duration.

flexmeasures/data/models/planning/linear_optimization.py

@@ -44,6 +44,7 @@ def device_scheduler(  # noqa C901
     initial_stock: float | list[float] = 0,
     stock_groups: dict[int, list[int]] | None = None,
     coupling_groups: dict[str, list[tuple[int, float]]] | None = None,
+    balance_groups: dict[str, list[int]] | None = None,
     ems_constraint_groups: list[list[int]] | None = None,
 ) -> tuple[list[pd.Series], float, SolverResults, ConcreteModel]:
     """This generic device scheduler is able to handle an EMS with multiple devices,
@@ -92,6 +93,15 @@ def device_scheduler(  # noqa C901
 
                                     coupling_groups={"chp": [(0, 1.0), (1, -0.5), (2, -0.3)]}
 
+    :param balance_groups:      Flow-balance constraints for internal commodity nodes (e.g. a heat or steam network
+                                without a grid connection). Each entry maps a node name to a list of device indices
+                                whose stock-side flows must balance at every time step:
+                                ``sum_d(P_up[d, j] * eff_up[d, j] + P_down[d, j] / eff_down[d, j] + stock_delta[d, j]) == 0``.
+                                In other words, everything produced into the node is consumed from it within the
+                                same time step; the node itself stores nothing. To add storage to a node, include
+                                a storage device in the group (its flow absorbs the imbalance and its stock is
+                                bounded by its own device constraints).
+
     Potentially deprecated arguments:
         commitment_quantities: amounts of flow specified in commitments (both previously ordered and newly requested)
             - e.g. in MW or boxes/h
@@ -205,6 +215,13 @@ def device_scheduler(  # noqa C901
             for d_idx, coeff in members:
                 coupling_device_specs.append((g_idx, d_idx, coeff))
 
+    # Collect the device lists of the balance groups (internal commodity nodes).
+    balance_group_specs: list[list[int]] = []
+    if balance_groups:
+        balance_group_specs = [
+            list(devices) for devices in balance_groups.values() if devices
+        ]
+
     # Move commitments from old structure to new
     if commitments is None:
         commitments = []
@@ -829,6 +846,28 @@ def flow_coupling_rule(m, c, j):
             model.coupling_device_range, model.j, rule=flow_coupling_rule
         )
 
+    if balance_group_specs:
+        model.balance_group_range = RangeSet(0, len(balance_group_specs) - 1)
+
+        def node_balance_rule(m, b, j):
+            """Balance the power flows of an internal commodity node at every time step.
+
+            Everything produced into the node must be consumed from it within the same
+            time step. The balance sums the devices' commodity-side flows (ems_power);
+            derivative efficiencies and stock deltas describe each device's own
+            stock-side conversion (e.g. of a shared buffer) and do not enter the
+            commodity balance.
+            """
+            return (
+                0,
+                sum(m.ems_power[d, j] for d in balance_group_specs[b]),
+                0,
+            )
+
+        model.node_balance_constraints = Constraint(
+            model.balance_group_range, model.j, rule=node_balance_rule
+        )
+
     # Add objective
     def cost_function(m):
         costs = 0

flexmeasures/data/models/planning/storage.py

@@ -175,6 +175,11 @@ def _prepare(self, skip_validation: bool = False) -> tuple:  # noqa: C901
         # with signed coefficients per canonical device index.
         self.coupling_groups = inventory.coupling_groups
 
+        # Balance groups for internal commodity nodes (commodities without energy
+        # prices, i.e. without a grid connection) are derived further below, once
+        # the devices of each commodity are enumerated.
+        self.balance_groups: dict[str, list[int]] = {}
+
         # List the asset(s) and sensor(s) being scheduled
         sensors: list[Sensor | None] = inventory.power_sensors
         assets: list[Asset | None] = inventory.assets
@@ -362,10 +367,33 @@ def device_list_series(
             if production_price is None:
                 production_price = consumption_price
 
-            if consumption_price is None:
-                raise ValueError(
-                    f"Missing consumption price for commodity '{commodity}'."
+            # A context without user-given price fields may still carry smart-defaulted
+            # zero prices (see CommodityFlexContextSchema.fill_grid_connection_defaults),
+            # in which case it is flagged with prices_are_defaulted.
+            has_no_user_given_prices = consumption_price is None or (
+                commodity_context.get("prices_are_defaulted", False)
+                and consumption_price_sensor is None
+                and production_price_sensor is None
+            )
+
+            if has_no_user_given_prices:
+                if commodity == "electricity":
+                    # Electricity is assumed to be grid-connected, so a missing
+                    # price is treated as a configuration error rather than as
+                    # an internal node.
+                    raise ValueError(
+                        f"Missing consumption price for commodity '{commodity}'."
+                    )
+                # A non-electricity commodity without energy prices is treated as an
+                # internal node (e.g. a heat or steam network without a grid
+                # connection): its devices must balance each other at every time
+                # step, and it needs no commitments or EMS-level capacity constraints.
+                current_app.logger.info(
+                    f"Commodity '{commodity}' has no energy prices; treating it as an "
+                    f"internal node whose devices (indices {devices}) balance each other."
                 )
+                self.balance_groups[commodity] = list(devices)
+                continue
 
             # Energy prices for this commodity.
             up_deviation_prices = get_continuous_series_sensor_or_quantity(
@@ -2863,6 +2891,7 @@ def compute(self, skip_validation: bool = False) -> SchedulerOutputType:
             initial_stock=initial_stock,
             stock_groups=self.stock_groups,
             coupling_groups=self.coupling_groups if self.coupling_groups else None,
+            balance_groups=getattr(self, "balance_groups", None) or None,
         )
         if "infeasible" in (tc := scheduler_results.solver.termination_condition):
             raise InfeasibleProblemException(tc)

flexmeasures/data/models/planning/tests/test_commitments.py

@@ -1852,9 +1852,15 @@ def _flow_df(**kwargs) -> pd.DataFrame:
 def _run_factory_scenario(
     gas_price: float,
     elec_price: float,
+    use_balance_groups: bool = False,
 ) -> tuple:
     """Run the simplified factory scenario and return the 7 device schedules.
 
+    With ``use_balance_groups=False``, the heat and steam nodes are balanced via
+    shared stock groups whose first ("reference") device carries min=max=0 stock
+    bounds. With ``use_balance_groups=True``, the same nodes are expressed directly
+    as ``balance_groups``, needing neither stock groups nor reference-device bounds.
+
     Devices
     ~~~~~~~
     d=0  e-heater       electricity   → heat coupling (ems_power ≥ 0, i.e. consumes electricity)
@@ -1905,11 +1911,14 @@ def _df(**kwargs) -> pd.DataFrame:
         defaults.update(kwargs)
         return pd.DataFrame(defaults, index=index)
 
+    # With balance groups, no reference device needs min=max=0 stock bounds.
+    node_bounds = {} if use_balance_groups else {"min": 0.0, "max": 0.0}
+
     device_constraints = [
         # d=0  e-heater: heat-node reference device. The min=max=0 forces the heat
         #       node to balance at every step (zero-capacity flow node), making
         #       the per-step dispatch deterministic despite flat prices.
-        _df(min=0.0, max=0.0, **{"derivative max": HEATER_POWER_MAX}),
+        _df(**node_bounds, **{"derivative max": HEATER_POWER_MAX}),
         # d=1  gas boiler: up to 100 kW gas → 100 kW heat (efficiency 1 for clean maths in test)
         _df(**{"derivative max": BOILER_GAS_MAX, "commodity": "gas"}),
         # d=2  steamer: can only produce steam (negative ems_power).
@@ -1927,8 +1936,7 @@ def _df(**kwargs) -> pd.DataFrame:
         # d=4  CHP heat output: positive ems_power adds heat to the steam node.
         #      The min=max=0 forces the steam node to balance at every step.
         _df(
-            min=0.0,
-            max=0.0,
+            **node_bounds,
             **{
                 "derivative min": -CHP_GAS_MAX * ETA_HEAT,
                 "derivative max": 0.0,
@@ -1952,11 +1960,18 @@ def _df(**kwargs) -> pd.DataFrame:
         index=index,
     )
 
-    # stock group: all heat-buffer devices share the same stock
-    # (key 0 is an arbitrary group id, not a device index)
-    heat_group_id = 0
-    steam_group_id = 1
-    stock_groups = {heat_group_id: [0, 1, 2], steam_group_id: [2, 4, 6]}
+    # Node membership: the steamer (d=2) converts heat to steam, so it belongs
+    # to both nodes (its single flow drains heat and feeds steam).
+    heat_node = [0, 1, 2]
+    steam_node = [2, 4, 6]
+    if use_balance_groups:
+        stock_groups = None
+        balance_groups = {"heat": heat_node, "steam": steam_node}
+    else:
+        # stock group: all heat-buffer devices share the same stock
+        # (keys 0 and 1 are arbitrary group ids, not device indices)
+        stock_groups = {0: heat_node, 1: steam_node}
+        balance_groups = None
 
     # CHP coupling: coefficients are signed efficiency fractions.
     # coeff_heat  = -η_heat  = -0.5 →  P_heat = -0.5 * alpha = -0.5 * P_gas
@@ -1998,6 +2013,7 @@ def _df(**kwargs) -> pd.DataFrame:
         commitments=commitments,
         stock_groups=stock_groups,
         coupling_groups=coupling_groups,
+        balance_groups=balance_groups,
     )
 
     assert results.solver.termination_condition == "optimal", (
@@ -2007,10 +2023,14 @@ def _df(**kwargs) -> pd.DataFrame:
     return tuple(schedules)
 
 
-def test_factory_chp_dispatch():
+@pytest.mark.parametrize("use_balance_groups", [False, True])
+def test_factory_chp_dispatch(use_balance_groups):
     """Factory: CHP + gas boiler + e-heater competing to meet a fixed steam demand.
 
-    The shared heat buffer (modelled via ``stock_groups``) is drained at a
+    The heat and steam nodes are balanced either via shared stock groups with
+    a min=max=0 reference device (``use_balance_groups=False``) or via explicit
+    ``balance_groups`` (``use_balance_groups=True``) — both must yield the same
+    dispatch. The steam node is drained at a
     constant rate of 15 kW by the steam demand device. Two price scenarios
     verify that the optimizer correctly chooses the cheapest heat source.
 
@@ -2061,7 +2081,9 @@ def test_factory_chp_dispatch():
     # Scenario A: gas cheaper — CHP at max, gas boiler fills the rest    #
     # ------------------------------------------------------------------ #
     (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = (
-        _run_factory_scenario(gas_price=20.0, elec_price=50.0)
+        _run_factory_scenario(
+            gas_price=20.0, elec_price=50.0, use_balance_groups=use_balance_groups
+        )
     )
 
     expected_chp_gas = pd.Series(20.0, index=e_heater.index)
@@ -2126,7 +2148,9 @@ def test_factory_chp_dispatch():
     # Scenario B: electricity cheaper — e-heater meets all demand        #
     # ------------------------------------------------------------------ #
     (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = (
-        _run_factory_scenario(gas_price=100.0, elec_price=10.0)
+        _run_factory_scenario(
+            gas_price=100.0, elec_price=10.0, use_balance_groups=use_balance_groups
+        )
     )
 
     expected_eheater_b = pd.Series(15.0, index=e_heater.index)
@@ -2174,7 +2198,9 @@ def test_factory_chp_dispatch():
     # Scenario C: gas slightly cheaper — gas boiler at max, e-heater fills the rest     #
     # --------------------------------------------------------------------------------- #
     (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = (
-        _run_factory_scenario(gas_price=50.0, elec_price=55.0)
+        _run_factory_scenario(
+            gas_price=50.0, elec_price=55.0, use_balance_groups=use_balance_groups
+        )
     )
 
     expected_chp_gas = pd.Series(0.0, index=e_heater.index)