feat: results.aggregate('total') - estimator-owned totals for CS/EDiD/Imputation/TwoStage + MMM no-scale admission

CHANGELOG.md

@@ -8,6 +8,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **`results.aggregate('total')` - the estimator-owned total incremental
+  outcome** on CallawaySantAnna, EfficientDiD, ImputationDiD, and TwoStageDiD,
+  promoted into the library-wide aggregation vocabulary. The single
+  `target="total"` row is an exact relay `C x overall` CONDITIONAL on the
+  realized aggregation mass - `att/se/CI` scaled by `C`, `t`/`p`/`df`
+  inherited - where `C` is each estimator's finite-masked complete-case
+  aggregation support (CS: the kept post-anticipation finite cells' mass,
+  replayed from an immutable fit-time kit snapshot; ImputationDiD: the
+  finite-tau support, fixing the documented raw-support overcount of the MMM
+  `scale="auto"` route for total exports; TwoStageDiD: the post-filter
+  treatment-indicator support; EfficientDiD: the kept cells' integer
+  `n_treated` sum). Scope is PANEL, NON-SURVEY fits: repeated-cross-section
+  routings, fits declaring a `survey_design=` (any weight type), CS
+  bare-`cluster=` fits whose cohort-mass weighting diverges from the
+  complete-case count, and - on CS and ImputationDiD, whose masses need new
+  fit-time kit snapshots - pre-upgrade pickled kits, all raise
+  `NotImplementedError` naming the reason (EfficientDiD and TwoStageDiD
+  compute their masses from state older kits already retain, so their legacy
+  artifacts work as-is) (the survey/RC remainder - with its
+  att*dC mass-uncertainty variance term - and the StackedDiD stage-out are
+  tracked in DEFERRED.md). 'total' is a RELAY level: available on
+  bootstrapped fits with a NaN df column. Both `diff_diff.mmm` exporters
+  admit the container ALONE - any `scale` is rejected as double-counting,
+  and admission is provenance-gated to the four audited adopters with the
+  full producer contract validated (hand-built/StackedDiD/unknown-provenance
+  totals fail closed) -
+  and for overall-total exports this supersedes `scale="auto"` (which remains
+  the per-cohort `group`-container route).
 - **Meridian `roi_calibration_period` mask builder + `to_code()` array support.**
   New `meridian_calibration_mask(media_times=, media_channels=, channel=, window=)`
   builds the boolean `(n_media_times, n_media_channels)` mask Meridian's
@@ -38,7 +66,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `results.aggregate('simple')` (one experiment row) or `'group'` (one row per
   cohort) - together with the new `scale=`, deriving `effect = att * scale` and
   `se = se * scale` per row in `to_dataframe()` order (the order every per-row
-  sequence kwarg aligns to). `scale` is always required with a container: a
+  sequence kwarg aligns to). `scale` is required with a `'simple'`/`'group'` container (a `'total'`
+  container forbids it - see the totals entry below): a
   finite positive number (scalar or per-row), or `"auto"`, which reads the
   container's per-row `n` and is honored ONLY for ImputationDiD/TwoStageDiD
   fits - the audited producers whose `n` is exactly the treated unit-periods
@@ -49,8 +78,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   pinned in the tests). Everything else fails closed with the remedy inline:
   raw results objects and `EventStudyResults` (the exporters never call
   `aggregate()` themselves), `calendar`/`dose`/unknown levels, non-`"att"`
-  target rows (ContinuousDiD's `acrt`, HAD's WAS, dCDH's estimand relays -
-  container rejected whole, no silent row filtering), rows without usable
+  target rows on `'simple'`/`'group'` containers (ContinuousDiD's `acrt`,
+  HAD's WAS, dCDH's estimand relays -
+  container rejected whole, no silent row filtering; `'total'` rows carry
+  `target='total'` by contract), rows without usable
   inference, and `scale="auto"` on any other estimator provenance
   (CallawaySantAnna/EfficientDiD/StackedDiD hints explain their `n` semantics;
   routing keys on the container's `estimator` field, never on `n_kind`, which
@@ -61,8 +92,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `tests/test_doc_snippets.py`), and both LLM guides drop the pre-3.9 "does
   not introspect result objects / deferred to the post-4.0 aggregate() layer"
   framing - that layer shipped in 3.9 and this is its consumer. The
-  estimator-owned-totals remainder for CS/EfficientDiD/StackedDiD stays
-  tracked in DEFERRED.md.
+  estimator-owned totals shipped in this release for
+  CS/EfficientDiD/ImputationDiD/TwoStageDiD on panel non-survey fits (the
+  `aggregate('total')` entry below); the two remainders - StackedDiD totals
+  and survey/repeated-cross-section totals - stay tracked in DEFERRED.md.
 - **Tutorial 28: The Scholarship Illusion** (`docs/tutorials/28_rdd_scholarship_illusion.ipynb`) -
   the regression discontinuity walkthrough: a naive above-vs-below comparison
   overstates a merit scholarship's effect fivefold; `RDPlot` makes the confound

CONTRIBUTING.md

@@ -171,7 +171,7 @@ A new parameter is only complete when it is:
   a `_PARAM_ATTR_ALIASES` entry, and constructor-derived state a
   `_DERIVED_CONFIG_ATTRS` entry — see `diff_diff/_base.py` and the
   inheritance map in `CLAUDE.md`)
-- applied in **every** aggregation mode — `simple`, `event_study`, and `group`
+- applied in **every** parameter-applying aggregation mode — `simple`, `event_study`, and `group`. `total` accepts no independent aggregation parameters, but every parameter that changes the overall keeper set or aggregation mass (e.g. `anticipation`) must also be reflected — and tested — in `total`'s mass
 - applied in the **bootstrap/inference** paths, not just the analytical one
 - reflected on the result object, so `to_dict()`/`summary()` do not misreport it
 - propagated to the estimators that inherit it: `TwoWayFixedEffects` defines

DEFERRED.md

@@ -36,6 +36,7 @@ provenance and AI-review deviation-documentation: a row here (or in
 | `SpilloverDiD(vcov_type="conley", conley_lag_cutoff>0, survey_design=...)` no-effective-PSU serial Bartlett HAC: weights-only / strata-only designs without a cluster fallback raise `NotImplementedError` (each pseudo-PSU appears in one period, so the serial cross-period loop contributes zero). Needs a unit-level serial fallback derivation or routing through `conley_unit` with documented IF-allocator asymmetry. | `spillover.py`, `two_stage.py::_compute_stratified_serial_bartlett_meat` | Wave E.2 tail | Low |
 | `SpilloverDiD` data-driven `d_bar` selection (Butts 2021b / 2023 JUE Insight cross-validation). | `spillover.py` | follow-up | Low |
 | **`LPDiD` non-absorbing exit-event dynamics** (Dube et al. 2025 online Appendix C `eta_h^{g,n}`): the shipped `non_absorbing` modes estimate the **entry-effect** estimands (Eq. 12/13) only; separate dynamic event-studies for treatment switch-*offs* are not implemented. Needs the exit-event clean-sample derivation + estimand contract. | `lpdid.py`, REGISTRY | PR-C follow-up | Low |
+| MMM interop, survey/RC totals for the four `aggregate('total')` adopters (CS/EfficientDiD/ImputationDiD/TwoStageDiD; the shipped v1 is panel non-survey only and fails closed with `NotImplementedError` on repeated-cross-section-routed fits, fits declaring a `survey_design=`, and the CS bare-`cluster=` divergent corner where the cohort-mass weighting disagrees with the complete-case count): needs (a) DESIGN-AWARE raw-mass recovery - analytic pweight/aweight resolved weights are normalized to sum = n and CS retains no raw record (fit-time RAW retention required there; CS accepts pweight only), while EDiD's analytic fweight fits and all replicate-design fits already retain raw scale in the kits and Imputation/TwoStage retain raw `SurveyMetadata.sum_weights` - and (b) the mass-uncertainty `att*dC` variance derivation (the shipped relay conditions on the realized mass and omits it). Never ship a survey/RC total from normalized masses. | `diff_diff/staggered_results.py`, `diff_diff/survey.py` | mmm-interop | Low |
 | **`LPDiD` non-absorbing interior-gap support**: non-absorbing modes require a gap-free panel within each unit's observed span and raise on interior time gaps (the `[t-L, t+h]` window conditions can't be verified across a gap). The absorbing path already reindexes interior gaps to the calendar grid; extending that fail-closed handling (per-window gap masking) to non-absorbing is deferred. | `lpdid.py::_prepare_panel` | PR-C follow-up | Low |
 
 ## Needs external reference (R / Stata / Julia)
@@ -72,7 +73,7 @@ For survey-specific limitations (`NotImplementedError` paths), see the
 | Issue | Location | PR | Priority |
 |-------|----------|----|----------|
 | EfficientDiD/ImputationDiD/TwoStageDiD kit retention has no memory opt-out: EfficientDiD's kit always retains the per-(g,t) EIF dict (M-023), ImputationDiD's references the `_fit_data` panel objects (M-021, pre-existing retention), and TwoStageDiD's retains a column-subset working-frame copy (M-022, the first new panel retention); a `store_kit=False` constructor knob would restore an opt-out at the cost of post-fit aggregation on that result | `diff_diff/efficient_did.py`, `diff_diff/imputation.py`, `diff_diff/two_stage.py` | 2(b) PR-3a | Low |
-| MMM interop: estimator-owned total incremental outcome for CallawaySantAnna/EfficientDiD/StackedDiD aggregations - their `AggregationResult` containers report units/cells (no treated-obs mass), so container-derived scaling (`aggregation_result=` + `scale="auto"`, shipped for ImputationDiD/TwoStageDiD) cannot cover them and their exporter route requires a caller-supplied numeric `scale=`. Closing the gap means each estimator's own aggregation (which owns its weights/balance/survey masses) computing a total effect + SE. | `diff_diff/mmm.py`, `diff_diff/staggered_aggregation.py` | mmm-interop | Low |
+| MMM interop, StackedDiD totals (remainder of the estimator-owned-totals delivery - `aggregate('total')` shipped for CS/EfficientDiD/ImputationDiD/TwoStageDiD on panel non-survey fits): StackedDiD is STAGED OUT because the total estimand is ill-defined under `weighting="population"`/`"sample_share"` (a weighted estimand for which raw treated exposure is not the right mass), and its results class is view-only (a total would be its first computed level). Needs an estimand decision per weighting variant before any implementation. | `diff_diff/mmm.py`, `diff_diff/stacked.py` | mmm-interop | Low |
 | `SyntheticControl` fit-snapshot residency (`_SyntheticControlFitSnapshot`) — **investigated 2026-07-07, parked**: the snapshot ALIASES the fit's own working pivots (zero extra construction cost); the retained residency implements the documented freeze contract (post-fit mutation of estimator inputs must not change `in_space_placebo()` / `leave_one_out()` / conformal output on an already-returned results object, and `__getstate__` already excludes it from pickles). A compact array representation saves only pandas overhead (the float panel dominates); releasing residency needs new API surface (`release`/opt-out flag) or a freeze-contract change. Revisit on user demand for very large donor panels. | `synthetic_control.py`, `synthetic_control_results.py` | follow-up | Low |
 | Stratified survey-PSU multiplier-weight draw-tiling — **investigated 2026-07-07, parked**: the stratified generator (`generate_survey_multiplier_weights_batch`) consumes ONE sequential rng stream stratum-major (`rng.choice(size=(n_bootstrap, n_h))` per stratum, then lonely-PSU pooling), so draw-chunked assembly CANNOT reproduce the stream bit-identically (contra the old row's parenthetical) — it would need per-stratum generator state skipping (PCG64.advance + per-weight-type variate accounting; fragile) or a stream-layout change (MC-level SE changes → baseline/golden recapture + REGISTRY note). Stratified designs have few PSUs, so the full `(n_bootstrap × n_psu)` matrix rarely matters; unstratified (the large-`n_units` case) is already tiled. Revisit only if a large-PSU stratified design hits memory, as a documented stream change. | `diff_diff/bootstrap_chunking.py::iter_survey_multiplier_weight_blocks` | follow-up | Low |
 | ChangesInChanges FULL Melly-Santangelo covariate estimator (monotonized integrated-indicator conditional CDFs, treated-post `F_{X|11}` integration, exchangeable bootstrap with variance-weighted KS bands, tail trimming, pre-period specification test). The simplified qte-`xformla` form of the MS pipeline is already supported (`covariates=` on both estimators, parity-tested vs qte 1.3.1); the FULL estimator has no reference implementation (the MS Stata code is the only one; distinct from Kranker's `cic`) and would need simulation-based validation. Reviewed: `docs/methodology/papers/melly-santangelo-2015-review.md`. | `diff_diff/changes_in_changes.py` | #682 | Low |

diff_diff/aggregation.py

@@ -45,6 +45,13 @@
     "event_study",
     "group",
     "calendar",
+    # "total": the estimator-owned total incremental outcome - an exact relay
+    # C x overall (C = the estimator's finite-masked complete-case aggregation
+    # support), CONDITIONAL on the realized aggregation mass. Shared meaning
+    # across adopters (CallawaySantAnna / ImputationDiD / TwoStageDiD /
+    # EfficientDiD); per-estimator support remains a subset, so non-adopters
+    # raise the same vocabulary-suffixed error as "calendar".
+    "total",
 )
 
 #: Pinned column schema of ``AggregationResult.to_dataframe()`` - identical for
@@ -97,6 +104,77 @@ def resolve_inference_df(results: Any) -> Optional[float]:
     return None
 
 
+def build_total_relay_row(
+    *,
+    mass: float,
+    att: Optional[float],
+    se: Optional[float],
+    t_stat: Optional[float],
+    p_value: Optional[float],
+    conf_int: Tuple[float, float],
+    df: Optional[float],
+    alpha: float,
+    estimator: str,
+) -> "AggregationResult":
+    """One ``level='total'`` row: the exact relay ``mass x overall``.
+
+    The estimand is the total incremental outcome CONDITIONAL ON THE REALIZED
+    AGGREGATION MASS: ``att = mass x overall_att``, ``se = mass x overall_se``,
+    CI scaled by ``mass``, ``t_stat``/``p_value``/``df`` inherited unchanged
+    (every SE branch is homogeneous of degree 1, so the relay is exact under
+    analytical AND percentile-bootstrap inference). Inherited NaNs PASS
+    THROUGH - a degenerate fit whose CI is (nan, nan) beside a finite att
+    keeps its finite ``att``/``n``, mirroring what ``aggregate('simple')``
+    publishes on the same fit (the repo's non-estimable-row convention:
+    att/n are inputs, never blanked by NaN inference fields).
+
+    Two cases blank the WHOLE row (att/se/t/p/CI/n/df all NaN):
+    - ``mass`` is not finite (no aggregable support exists), or
+    - a FINITE overall value became non-finite BY the scaling (true float
+      overflow; surfaced per this code family's finiteness convention rather
+      than an ``errstate`` wrap).
+    """
+
+    def _f(v: Optional[float]) -> float:
+        return float(v) if v is not None else np.nan
+
+    a, s = _f(att), _f(se)
+    t, p = _f(t_stat), _f(p_value)
+    lo, hi = _f(conf_int[0]), _f(conf_int[1])
+    d = _f(df)
+
+    s_att, s_se = mass * a, mass * s
+    s_lo, s_hi = mass * lo, mass * hi
+    n_val = float(mass)
+    overflowed = np.isfinite(mass) and (
+        (np.isfinite(a) and not np.isfinite(s_att))
+        or (np.isfinite(s) and not np.isfinite(s_se))
+        or (np.isfinite(lo) and not np.isfinite(s_lo))
+        or (np.isfinite(hi) and not np.isfinite(s_hi))
+    )
+    if not np.isfinite(mass) or overflowed:
+        s_att = s_se = s_lo = s_hi = t = p = d = np.nan
+        n_val = np.nan
+
+    return AggregationResult(
+        level="total",
+        label=np.array(["total"], dtype=object),
+        target=np.array(["total"], dtype=object),
+        att=np.array([s_att], dtype=float),
+        se=np.array([s_se], dtype=float),
+        t_stat=np.array([t], dtype=float),
+        p_value=np.array([p], dtype=float),
+        conf_int_lower=np.array([s_lo], dtype=float),
+        conf_int_upper=np.array([s_hi], dtype=float),
+        n=np.array([n_val], dtype=float),
+        df=d,
+        alpha=alpha,
+        n_kind="obs",
+        weight=np.array([1.0], dtype=float),
+        estimator=estimator,
+    )
+
+
 def _sortable(labels: np.ndarray) -> bool:
     """Can ``labels`` be ordered without raising?
 
@@ -128,11 +206,14 @@ class AggregationResult(BaseResults):
     label : np.ndarray
         Per-row aggregation key: the cohort for ``"group"``, the calendar
         period for ``"calendar"``, the dose for ``"dose"``. A single
-        ``"overall"`` entry for ``"simple"``.
+        ``"overall"`` entry for ``"simple"``; a single ``"total"`` entry for
+        ``"total"``.
     target : np.ndarray
         Per-row estimand discriminator, so one container can carry two
         aligned estimands over the same labels (ContinuousDiD's ATT(d) and
-        ACRT(d) become 2N rows). ``"att"`` where an estimator has one.
+        ACRT(d) become 2N rows). ``"att"`` where an estimator has one;
+        ``"total"`` on a ``"total"`` row - a total incremental outcome over
+        the estimator's aggregation mass, not an ATT.
     att, se, t_stat, p_value : np.ndarray
         The canonical quintet, per row, carrying WHATEVER inference the fit
         stored - never recomputed. On a bootstrapped fit that usually means
@@ -157,7 +238,8 @@ class AggregationResult(BaseResults):
         ``(level, target)`` group. ``None`` where no per-row mass exists -
         CallawaySantAnna's ``"group"`` aggregation weights ``(g, t)`` cells
         equally WITHIN each cohort and has no cross-cohort mass, so inventing
-        one would be a fabricated number.
+        one would be a fabricated number. Single-row containers
+        (``"simple"``, ``"total"``) carry ``weight=[1.0]``.
     df : np.ndarray
         Per-row inference degrees of freedom, NaN where none governed the
         stored p-value. NaN on percentile-bootstrap rows (no df governs
@@ -454,9 +536,12 @@ class AggregationKit:
     ----------
     bookkeeping : dict
         The aggregation-relevant subset of the estimator's ``precomputed``
-        mapping. O(n_units) on panel fits; several entries are
-        observation-length on repeated cross-sections, where
-        ``all_units = np.arange(n_obs)`` by construction.
+        mapping, plus fit-time DERIVED keys stashed by the kit builder
+        (CallawaySantAnna's ``agg_gt_cells``/``is_survey_fit``,
+        ImputationDiD's ``total_support`` - see each builder). O(n_units) on
+        panel fits, except ``agg_gt_cells`` which is O(n_gt) 4-tuples;
+        several entries are observation-length on repeated cross-sections,
+        where ``all_units = np.arange(n_obs)`` by construction.
     influence : dict
         Per-``(g, t)`` influence-function payload. The DOMINANT retained
         object, roughly O(n_units x n_gt).