diff --git a/CHANGELOG.md b/CHANGELOG.md index fb7128241..56158bc18 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **`results.aggregate('total')` - the estimator-owned total incremental + outcome** on CallawaySantAnna, EfficientDiD, ImputationDiD, and TwoStageDiD, + promoted into the library-wide aggregation vocabulary. The single + `target="total"` row is an exact relay `C x overall` CONDITIONAL on the + realized aggregation mass - `att/se/CI` scaled by `C`, `t`/`p`/`df` + inherited - where `C` is each estimator's finite-masked complete-case + aggregation support (CS: the kept post-anticipation finite cells' mass, + replayed from an immutable fit-time kit snapshot; ImputationDiD: the + finite-tau support, fixing the documented raw-support overcount of the MMM + `scale="auto"` route for total exports; TwoStageDiD: the post-filter + treatment-indicator support; EfficientDiD: the kept cells' integer + `n_treated` sum). Scope is PANEL, NON-SURVEY fits: repeated-cross-section + routings, fits declaring a `survey_design=` (any weight type), CS + bare-`cluster=` fits whose cohort-mass weighting diverges from the + complete-case count, and - on CS and ImputationDiD, whose masses need new + fit-time kit snapshots - pre-upgrade pickled kits, all raise + `NotImplementedError` naming the reason (EfficientDiD and TwoStageDiD + compute their masses from state older kits already retain, so their legacy + artifacts work as-is) (the survey/RC remainder - with its + att*dC mass-uncertainty variance term - and the StackedDiD stage-out are + tracked in DEFERRED.md). 'total' is a RELAY level: available on + bootstrapped fits with a NaN df column. Both `diff_diff.mmm` exporters + admit the container ALONE - any `scale` is rejected as double-counting, + and admission is provenance-gated to the four audited adopters with the + full producer contract validated (hand-built/StackedDiD/unknown-provenance + totals fail closed) - + and for overall-total exports this supersedes `scale="auto"` (which remains + the per-cohort `group`-container route). - **Meridian `roi_calibration_period` mask builder + `to_code()` array support.** New `meridian_calibration_mask(media_times=, media_channels=, channel=, window=)` builds the boolean `(n_media_times, n_media_channels)` mask Meridian's @@ -38,7 +66,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `results.aggregate('simple')` (one experiment row) or `'group'` (one row per cohort) - together with the new `scale=`, deriving `effect = att * scale` and `se = se * scale` per row in `to_dataframe()` order (the order every per-row - sequence kwarg aligns to). `scale` is always required with a container: a + sequence kwarg aligns to). `scale` is required with a `'simple'`/`'group'` container (a `'total'` + container forbids it - see the totals entry below): a finite positive number (scalar or per-row), or `"auto"`, which reads the container's per-row `n` and is honored ONLY for ImputationDiD/TwoStageDiD fits - the audited producers whose `n` is exactly the treated unit-periods @@ -49,8 +78,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 pinned in the tests). Everything else fails closed with the remedy inline: raw results objects and `EventStudyResults` (the exporters never call `aggregate()` themselves), `calendar`/`dose`/unknown levels, non-`"att"` - target rows (ContinuousDiD's `acrt`, HAD's WAS, dCDH's estimand relays - - container rejected whole, no silent row filtering), rows without usable + target rows on `'simple'`/`'group'` containers (ContinuousDiD's `acrt`, + HAD's WAS, dCDH's estimand relays - + container rejected whole, no silent row filtering; `'total'` rows carry + `target='total'` by contract), rows without usable inference, and `scale="auto"` on any other estimator provenance (CallawaySantAnna/EfficientDiD/StackedDiD hints explain their `n` semantics; routing keys on the container's `estimator` field, never on `n_kind`, which @@ -61,8 +92,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `tests/test_doc_snippets.py`), and both LLM guides drop the pre-3.9 "does not introspect result objects / deferred to the post-4.0 aggregate() layer" framing - that layer shipped in 3.9 and this is its consumer. The - estimator-owned-totals remainder for CS/EfficientDiD/StackedDiD stays - tracked in DEFERRED.md. + estimator-owned totals shipped in this release for + CS/EfficientDiD/ImputationDiD/TwoStageDiD on panel non-survey fits (the + `aggregate('total')` entry below); the two remainders - StackedDiD totals + and survey/repeated-cross-section totals - stay tracked in DEFERRED.md. - **Tutorial 28: The Scholarship Illusion** (`docs/tutorials/28_rdd_scholarship_illusion.ipynb`) - the regression discontinuity walkthrough: a naive above-vs-below comparison overstates a merit scholarship's effect fivefold; `RDPlot` makes the confound diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5bb6dcbcf..69da69c8e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -171,7 +171,7 @@ A new parameter is only complete when it is: a `_PARAM_ATTR_ALIASES` entry, and constructor-derived state a `_DERIVED_CONFIG_ATTRS` entry — see `diff_diff/_base.py` and the inheritance map in `CLAUDE.md`) -- applied in **every** aggregation mode — `simple`, `event_study`, and `group` +- applied in **every** parameter-applying aggregation mode — `simple`, `event_study`, and `group`. `total` accepts no independent aggregation parameters, but every parameter that changes the overall keeper set or aggregation mass (e.g. `anticipation`) must also be reflected — and tested — in `total`'s mass - applied in the **bootstrap/inference** paths, not just the analytical one - reflected on the result object, so `to_dict()`/`summary()` do not misreport it - propagated to the estimators that inherit it: `TwoWayFixedEffects` defines diff --git a/DEFERRED.md b/DEFERRED.md index cc6c07565..2af5c93e6 100644 --- a/DEFERRED.md +++ b/DEFERRED.md @@ -36,6 +36,7 @@ provenance and AI-review deviation-documentation: a row here (or in | `SpilloverDiD(vcov_type="conley", conley_lag_cutoff>0, survey_design=...)` no-effective-PSU serial Bartlett HAC: weights-only / strata-only designs without a cluster fallback raise `NotImplementedError` (each pseudo-PSU appears in one period, so the serial cross-period loop contributes zero). Needs a unit-level serial fallback derivation or routing through `conley_unit` with documented IF-allocator asymmetry. | `spillover.py`, `two_stage.py::_compute_stratified_serial_bartlett_meat` | Wave E.2 tail | Low | | `SpilloverDiD` data-driven `d_bar` selection (Butts 2021b / 2023 JUE Insight cross-validation). | `spillover.py` | follow-up | Low | | **`LPDiD` non-absorbing exit-event dynamics** (Dube et al. 2025 online Appendix C `eta_h^{g,n}`): the shipped `non_absorbing` modes estimate the **entry-effect** estimands (Eq. 12/13) only; separate dynamic event-studies for treatment switch-*offs* are not implemented. Needs the exit-event clean-sample derivation + estimand contract. | `lpdid.py`, REGISTRY | PR-C follow-up | Low | +| MMM interop, survey/RC totals for the four `aggregate('total')` adopters (CS/EfficientDiD/ImputationDiD/TwoStageDiD; the shipped v1 is panel non-survey only and fails closed with `NotImplementedError` on repeated-cross-section-routed fits, fits declaring a `survey_design=`, and the CS bare-`cluster=` divergent corner where the cohort-mass weighting disagrees with the complete-case count): needs (a) DESIGN-AWARE raw-mass recovery - analytic pweight/aweight resolved weights are normalized to sum = n and CS retains no raw record (fit-time RAW retention required there; CS accepts pweight only), while EDiD's analytic fweight fits and all replicate-design fits already retain raw scale in the kits and Imputation/TwoStage retain raw `SurveyMetadata.sum_weights` - and (b) the mass-uncertainty `att*dC` variance derivation (the shipped relay conditions on the realized mass and omits it). Never ship a survey/RC total from normalized masses. | `diff_diff/staggered_results.py`, `diff_diff/survey.py` | mmm-interop | Low | | **`LPDiD` non-absorbing interior-gap support**: non-absorbing modes require a gap-free panel within each unit's observed span and raise on interior time gaps (the `[t-L, t+h]` window conditions can't be verified across a gap). The absorbing path already reindexes interior gaps to the calendar grid; extending that fail-closed handling (per-window gap masking) to non-absorbing is deferred. | `lpdid.py::_prepare_panel` | PR-C follow-up | Low | ## Needs external reference (R / Stata / Julia) @@ -72,7 +73,7 @@ For survey-specific limitations (`NotImplementedError` paths), see the | Issue | Location | PR | Priority | |-------|----------|----|----------| | EfficientDiD/ImputationDiD/TwoStageDiD kit retention has no memory opt-out: EfficientDiD's kit always retains the per-(g,t) EIF dict (M-023), ImputationDiD's references the `_fit_data` panel objects (M-021, pre-existing retention), and TwoStageDiD's retains a column-subset working-frame copy (M-022, the first new panel retention); a `store_kit=False` constructor knob would restore an opt-out at the cost of post-fit aggregation on that result | `diff_diff/efficient_did.py`, `diff_diff/imputation.py`, `diff_diff/two_stage.py` | 2(b) PR-3a | Low | -| MMM interop: estimator-owned total incremental outcome for CallawaySantAnna/EfficientDiD/StackedDiD aggregations - their `AggregationResult` containers report units/cells (no treated-obs mass), so container-derived scaling (`aggregation_result=` + `scale="auto"`, shipped for ImputationDiD/TwoStageDiD) cannot cover them and their exporter route requires a caller-supplied numeric `scale=`. Closing the gap means each estimator's own aggregation (which owns its weights/balance/survey masses) computing a total effect + SE. | `diff_diff/mmm.py`, `diff_diff/staggered_aggregation.py` | mmm-interop | Low | +| MMM interop, StackedDiD totals (remainder of the estimator-owned-totals delivery - `aggregate('total')` shipped for CS/EfficientDiD/ImputationDiD/TwoStageDiD on panel non-survey fits): StackedDiD is STAGED OUT because the total estimand is ill-defined under `weighting="population"`/`"sample_share"` (a weighted estimand for which raw treated exposure is not the right mass), and its results class is view-only (a total would be its first computed level). Needs an estimand decision per weighting variant before any implementation. | `diff_diff/mmm.py`, `diff_diff/stacked.py` | mmm-interop | Low | | `SyntheticControl` fit-snapshot residency (`_SyntheticControlFitSnapshot`) — **investigated 2026-07-07, parked**: the snapshot ALIASES the fit's own working pivots (zero extra construction cost); the retained residency implements the documented freeze contract (post-fit mutation of estimator inputs must not change `in_space_placebo()` / `leave_one_out()` / conformal output on an already-returned results object, and `__getstate__` already excludes it from pickles). A compact array representation saves only pandas overhead (the float panel dominates); releasing residency needs new API surface (`release`/opt-out flag) or a freeze-contract change. Revisit on user demand for very large donor panels. | `synthetic_control.py`, `synthetic_control_results.py` | follow-up | Low | | Stratified survey-PSU multiplier-weight draw-tiling — **investigated 2026-07-07, parked**: the stratified generator (`generate_survey_multiplier_weights_batch`) consumes ONE sequential rng stream stratum-major (`rng.choice(size=(n_bootstrap, n_h))` per stratum, then lonely-PSU pooling), so draw-chunked assembly CANNOT reproduce the stream bit-identically (contra the old row's parenthetical) — it would need per-stratum generator state skipping (PCG64.advance + per-weight-type variate accounting; fragile) or a stream-layout change (MC-level SE changes → baseline/golden recapture + REGISTRY note). Stratified designs have few PSUs, so the full `(n_bootstrap × n_psu)` matrix rarely matters; unstratified (the large-`n_units` case) is already tiled. Revisit only if a large-PSU stratified design hits memory, as a documented stream change. | `diff_diff/bootstrap_chunking.py::iter_survey_multiplier_weight_blocks` | follow-up | Low | | ChangesInChanges FULL Melly-Santangelo covariate estimator (monotonized integrated-indicator conditional CDFs, treated-post `F_{X|11}` integration, exchangeable bootstrap with variance-weighted KS bands, tail trimming, pre-period specification test). The simplified qte-`xformla` form of the MS pipeline is already supported (`covariates=` on both estimators, parity-tested vs qte 1.3.1); the FULL estimator has no reference implementation (the MS Stata code is the only one; distinct from Kranker's `cic`) and would need simulation-based validation. Reviewed: `docs/methodology/papers/melly-santangelo-2015-review.md`. | `diff_diff/changes_in_changes.py` | #682 | Low | diff --git a/diff_diff/aggregation.py b/diff_diff/aggregation.py index fc6cf0c61..a26a53a37 100644 --- a/diff_diff/aggregation.py +++ b/diff_diff/aggregation.py @@ -45,6 +45,13 @@ "event_study", "group", "calendar", + # "total": the estimator-owned total incremental outcome - an exact relay + # C x overall (C = the estimator's finite-masked complete-case aggregation + # support), CONDITIONAL on the realized aggregation mass. Shared meaning + # across adopters (CallawaySantAnna / ImputationDiD / TwoStageDiD / + # EfficientDiD); per-estimator support remains a subset, so non-adopters + # raise the same vocabulary-suffixed error as "calendar". + "total", ) #: Pinned column schema of ``AggregationResult.to_dataframe()`` - identical for @@ -97,6 +104,77 @@ def resolve_inference_df(results: Any) -> Optional[float]: return None +def build_total_relay_row( + *, + mass: float, + att: Optional[float], + se: Optional[float], + t_stat: Optional[float], + p_value: Optional[float], + conf_int: Tuple[float, float], + df: Optional[float], + alpha: float, + estimator: str, +) -> "AggregationResult": + """One ``level='total'`` row: the exact relay ``mass x overall``. + + The estimand is the total incremental outcome CONDITIONAL ON THE REALIZED + AGGREGATION MASS: ``att = mass x overall_att``, ``se = mass x overall_se``, + CI scaled by ``mass``, ``t_stat``/``p_value``/``df`` inherited unchanged + (every SE branch is homogeneous of degree 1, so the relay is exact under + analytical AND percentile-bootstrap inference). Inherited NaNs PASS + THROUGH - a degenerate fit whose CI is (nan, nan) beside a finite att + keeps its finite ``att``/``n``, mirroring what ``aggregate('simple')`` + publishes on the same fit (the repo's non-estimable-row convention: + att/n are inputs, never blanked by NaN inference fields). + + Two cases blank the WHOLE row (att/se/t/p/CI/n/df all NaN): + - ``mass`` is not finite (no aggregable support exists), or + - a FINITE overall value became non-finite BY the scaling (true float + overflow; surfaced per this code family's finiteness convention rather + than an ``errstate`` wrap). + """ + + def _f(v: Optional[float]) -> float: + return float(v) if v is not None else np.nan + + a, s = _f(att), _f(se) + t, p = _f(t_stat), _f(p_value) + lo, hi = _f(conf_int[0]), _f(conf_int[1]) + d = _f(df) + + s_att, s_se = mass * a, mass * s + s_lo, s_hi = mass * lo, mass * hi + n_val = float(mass) + overflowed = np.isfinite(mass) and ( + (np.isfinite(a) and not np.isfinite(s_att)) + or (np.isfinite(s) and not np.isfinite(s_se)) + or (np.isfinite(lo) and not np.isfinite(s_lo)) + or (np.isfinite(hi) and not np.isfinite(s_hi)) + ) + if not np.isfinite(mass) or overflowed: + s_att = s_se = s_lo = s_hi = t = p = d = np.nan + n_val = np.nan + + return AggregationResult( + level="total", + label=np.array(["total"], dtype=object), + target=np.array(["total"], dtype=object), + att=np.array([s_att], dtype=float), + se=np.array([s_se], dtype=float), + t_stat=np.array([t], dtype=float), + p_value=np.array([p], dtype=float), + conf_int_lower=np.array([s_lo], dtype=float), + conf_int_upper=np.array([s_hi], dtype=float), + n=np.array([n_val], dtype=float), + df=d, + alpha=alpha, + n_kind="obs", + weight=np.array([1.0], dtype=float), + estimator=estimator, + ) + + def _sortable(labels: np.ndarray) -> bool: """Can ``labels`` be ordered without raising? @@ -128,11 +206,14 @@ class AggregationResult(BaseResults): label : np.ndarray Per-row aggregation key: the cohort for ``"group"``, the calendar period for ``"calendar"``, the dose for ``"dose"``. A single - ``"overall"`` entry for ``"simple"``. + ``"overall"`` entry for ``"simple"``; a single ``"total"`` entry for + ``"total"``. target : np.ndarray Per-row estimand discriminator, so one container can carry two aligned estimands over the same labels (ContinuousDiD's ATT(d) and - ACRT(d) become 2N rows). ``"att"`` where an estimator has one. + ACRT(d) become 2N rows). ``"att"`` where an estimator has one; + ``"total"`` on a ``"total"`` row - a total incremental outcome over + the estimator's aggregation mass, not an ATT. att, se, t_stat, p_value : np.ndarray The canonical quintet, per row, carrying WHATEVER inference the fit stored - never recomputed. On a bootstrapped fit that usually means @@ -157,7 +238,8 @@ class AggregationResult(BaseResults): ``(level, target)`` group. ``None`` where no per-row mass exists - CallawaySantAnna's ``"group"`` aggregation weights ``(g, t)`` cells equally WITHIN each cohort and has no cross-cohort mass, so inventing - one would be a fabricated number. + one would be a fabricated number. Single-row containers + (``"simple"``, ``"total"``) carry ``weight=[1.0]``. df : np.ndarray Per-row inference degrees of freedom, NaN where none governed the stored p-value. NaN on percentile-bootstrap rows (no df governs @@ -454,9 +536,12 @@ class AggregationKit: ---------- bookkeeping : dict The aggregation-relevant subset of the estimator's ``precomputed`` - mapping. O(n_units) on panel fits; several entries are - observation-length on repeated cross-sections, where - ``all_units = np.arange(n_obs)`` by construction. + mapping, plus fit-time DERIVED keys stashed by the kit builder + (CallawaySantAnna's ``agg_gt_cells``/``is_survey_fit``, + ImputationDiD's ``total_support`` - see each builder). O(n_units) on + panel fits, except ``agg_gt_cells`` which is O(n_gt) 4-tuples; + several entries are observation-length on repeated cross-sections, + where ``all_units = np.arange(n_obs)`` by construction. influence : dict Per-``(g, t)`` influence-function payload. The DOMINANT retained object, roughly O(n_units x n_gt). diff --git a/diff_diff/efficient_did.py b/diff_diff/efficient_did.py index 8aab3f720..17161f26b 100644 --- a/diff_diff/efficient_did.py +++ b/diff_diff/efficient_did.py @@ -503,13 +503,15 @@ def fit( ``"event_study"``, ``"group"``, or ``"all"``; unknown strings act like ``None``, unchanged). Aggregate as a post-fit step instead: ``results.aggregate('event_study')`` / - ``.aggregate('group')`` / ``.aggregate('simple')``. On + ``.aggregate('group')`` / ``.aggregate('simple')`` / + ``.aggregate('total')``. On bootstrapped fits (``n_bootstrap > 0``) the post-fit RECOMPUTE levels (``'event_study'``/``'group'``) fail closed — the deprecated fit-time path remains the supported route for those bootstrapped aggregated surfaces — while - ``aggregate('simple')`` relays the stored bootstrap - inference and stays available (the per-level policy + ``aggregate('simple')`` and, where supported, + ``aggregate('total')`` relay the stored bootstrap + inference and stay available (the per-level policy converged with row M-027). balance_e : int, optional DEPRECATED (3.9, removed in 4.0, row M-120): moves onto @@ -559,7 +561,7 @@ def fit( "removed in 4.0. Fit once, then aggregate as a post-fit " "step: results = EfficientDiD().fit(...); " "results.aggregate('event_study') / .aggregate('group') / " - ".aggregate('simple'). balance_e moves onto aggregate() " + ".aggregate('simple') / .aggregate('total'). balance_e moves onto aggregate() " "alongside it: results.aggregate('event_study', " "balance_e=2).", FutureWarning, diff --git a/diff_diff/efficient_did_results.py b/diff_diff/efficient_did_results.py index 1e5bf1902..afa809ae2 100644 --- a/diff_diff/efficient_did_results.py +++ b/diff_diff/efficient_did_results.py @@ -12,7 +12,7 @@ import numpy as np import pandas as pd -from diff_diff.aggregation import AggregationMixin, AggregationResult +from diff_diff.aggregation import AggregationMixin, AggregationResult, build_total_relay_row from diff_diff.efficient_did_aggregation import _EfficientAggregationMixin from diff_diff.results import _format_survey_block, _get_significance_stars from diff_diff.results_base import BaseResults, build_event_study_surface @@ -235,7 +235,7 @@ class EfficientDiDResults(BaseResults, AggregationMixin): # annotation) so they never enter dataclasses.fields; the mixin's # ClassVar-annotated defaults document the contract. balance_e keeps # the mixin default ("event_study",) - CS precedent, do not redeclare. - _AGGREGATE_SUPPORTED = ("simple", "event_study", "group") + _AGGREGATE_SUPPORTED = ("simple", "event_study", "group", "total") # --- Inference-field aliases (balance/external-adapter compatibility) --- @property @@ -305,14 +305,17 @@ def _aggregate_compute( # is honored by the relay's NaN df column.) if level == "simple": return self._aggregate_simple_result(kit) + if level == "total": + return self._aggregate_total_result(kit) if self.bootstrap_results is not None: raise NotImplementedError( f"aggregate({level!r}) is not yet available on a bootstrapped " "fit (n_bootstrap > 0): the per-horizon bootstrap draws are " "not retained, so post-fit re-aggregation cannot replay " "percentile inference and analytical inference would " - "misrepresent the fit. aggregate('simple') relays the stored " - "bootstrap inference and remains available; otherwise re-fit " + "misrepresent the fit. aggregate('simple') and, where " + "supported, aggregate('total') relay the stored " + "bootstrap inference and remain available; otherwise re-fit " "with the aggregation you need, or use n_bootstrap=0." ) bk = dict(kit.bookkeeping) @@ -417,6 +420,57 @@ def _aggregate_simple_result(self, kit: Any) -> AggregationResult: estimator=type(self).__name__.replace("Results", ""), ) + def _aggregate_total_result(self, kit: Any) -> AggregationResult: + """The estimator-owned total incremental outcome as a one-row table. + + Exact relay ``C x overall`` CONDITIONAL on the realized aggregation + mass, with ``C = sum(n_treated)`` over the KEPT (g, t) cells of the + kit's deep-copied ``group_time_effects`` snapshot - keepers are the + post-anticipation, finite-effect cells, mirroring the simple + aggregation's ``cohort_fractions`` support. The integer per-cell + ``n_treated`` sum is used, NEVER the float + ``n_units x sum(cohort_fractions)`` product (routinely non-integral + by roundoff). No keepers -> NaN mass -> all-NaN row. Fails closed on + fits declaring a ``survey_design=`` (see the REGISTRY Note). + """ + bk = kit.bookkeeping + # Survey gate, from the kit's own markers: EDiD never synthesizes an + # internal design, so a non-None marker means the fit DECLARED a + # survey_design (weight-type-agnostic: unweighted psu-only and + # analytic fweight designs populate these too). There is NO + # "survey_weights" key in this kit. + if bk.get("unit_level_weights") is not None or bk.get("resolved_survey_unit") is not None: + raise NotImplementedError( + "aggregate('total') is not available on fits declaring a " + "survey_design: the realized-mass relay omits the survey " + "mass-uncertainty variance term, and design-aware " + "population-scale totals are not implemented (retained " + "weight scale differs by design family) - tracked in " + "DEFERRED.md. For an unweighted clustered fit, cluster= " + "(without survey_design) supports totals." + ) + support = 0.0 + n_keepers = 0 + for (g, t), cell in bk["group_time_effects"].items(): + if t < g - kit.anticipation: + continue + if not np.isfinite(float(cell["effect"])): + continue + support += float(cell["n_treated"]) + n_keepers += 1 + mass = support if n_keepers > 0 else float(np.nan) + return build_total_relay_row( + mass=mass, + att=self.overall_att, + se=self.overall_se, + t_stat=self.overall_t_stat, + p_value=self.overall_p_value, + conf_int=self.overall_conf_int, + df=(np.nan if self.bootstrap_results is not None else bk["df_survey"]), + alpha=kit.alpha, + estimator=type(self).__name__.replace("Results", ""), + ) + def _group_effects_to_aggregation( self, effects: Dict[Any, Dict[str, Any]], kit: Any ) -> AggregationResult: diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index 3bf811e78..104ed5961 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -1646,13 +1646,21 @@ Returned by `CallawaySantAnna.fit()`. **Methods:** `aggregate(type, weights=None, *, balance_e=None)`, `summary()`, `print_summary()`, `to_dataframe(level="event_study"|"group_time"|"group")` -`aggregate("simple"|"event_study"|"group")` re-aggregates POST-FIT with no +`aggregate("simple"|"event_study"|"group"|"total")` re-aggregates POST-FIT +with no refit, returning a NEW object and leaving the result unchanged. `"event_study"` returns `EventStudyResults`; the others return -`AggregationResult`. `balance_e=` applies to `"event_study"` only. Raises on +`AggregationResult`. `"total"` (3.10) is the estimator-owned total +incremental outcome - an exact relay C x overall over the finite-masked +complete-case support (single `target="total"` row; panel non-survey fits +only: repeated-cross-section-routed and declared-`survey_design` fits raise +`NotImplementedError` with the reason, as do bare-`cluster=` fits whose +cohort-mass weighting diverges from the complete-case count). +`balance_e=` applies to `"event_study"` only. Raises on `"calendar"` (CS has no calendar aggregator) and, on a bootstrapped fit, on the recompute levels (`"event_study"`/`"group"`) rather than substituting -analytical inference for percentile-bootstrap statistics - `"simple"` relays +analytical inference for percentile-bootstrap statistics - `"simple"` and, +where supported, `"total"` relay the stored bootstrap inference with a NaN df column (the per-level rule). ### SunAbrahamResults @@ -1832,7 +1840,7 @@ Each event study effect dict contains: `effect`, `se`, `t_stat`, `p_value`, `con | `n_clusters` | `int | None` | Number of effective clusters; `None` under survey designs and `None` under EfficientDiD's default unclustered fit | | `df_convention` | `str | None` | On the knob-carrying containers (`StackedDiDResults`, `ImputationDiDResults`, `WooldridgeDiDResults`, `LPDiDResults`, `SunAbrahamResults`): the configured df convention (3.9 / M-127); `StackedDiDResults` additionally carries `inference_df` (the overall-ATT df actually used) | -**Methods:** `summary()`, `print_summary()`, `to_dataframe()`, `to_dict()` (flat dict of headline aliases + `vcov_type` + conditional `cluster_name`/`n_clusters`/`n_bootstrap`/`inference_method`); `aggregate(type, weights=None, *, balance_e=None)` on the shipped post-fit adopters (`StackedDiDResults` views since 3.9/M-024; `EfficientDiDResults` recomputes from retained EIFs since 3.9/M-023; `ImputationDiDResults` and `TwoStageDiDResults` recompute from their PANEL-BACKED kits since 3.9/M-021/M-022 - on bootstrapped fits their recompute levels raise while `'simple'` relays the stored bootstrap inference with a NaN df column (the per-level rule, converged with M-027); `ContinuousDiDResults` is MIXED since 3.9/M-025 - `'simple'`/`'dose'` are views over stored fields that work on any fit incl. bootstrapped, `'event_study'` recomputes from a pruned per-cell IF kit and raises on bootstrapped fits; `HeterogeneousAdoptionDiDResults` and `HeterogeneousAdoptionDiDEventStudyResults` are pure views since 3.9/M-027 - `'simple'` on the overall class, `'event_study'` on the event-study class, no kit, work on pickles from any release) +**Methods:** `summary()`, `print_summary()`, `to_dataframe()`, `to_dict()` (flat dict of headline aliases + `vcov_type` + conditional `cluster_name`/`n_clusters`/`n_bootstrap`/`inference_method`); `aggregate(type, weights=None, *, balance_e=None)` on the shipped post-fit adopters (`StackedDiDResults` views since 3.9/M-024; `EfficientDiDResults` recomputes from retained EIFs since 3.9/M-023; `ImputationDiDResults` and `TwoStageDiDResults` recompute from their PANEL-BACKED kits since 3.9/M-021/M-022 - on bootstrapped fits their recompute levels raise while `'simple'` and, where supported, `'total'` relay the stored bootstrap inference with a NaN df column (the per-level rule, converged with M-027); `'total'` (3.10) is the estimator-owned total incremental outcome on `CallawaySantAnnaResults`/`EfficientDiDResults`/`ImputationDiDResults`/`TwoStageDiDResults` - panel non-survey fits only, fails closed with the reason elsewhere; `ContinuousDiDResults` is MIXED since 3.9/M-025 - `'simple'`/`'dose'` are views over stored fields that work on any fit incl. bootstrapped, `'event_study'` recomputes from a pruned per-cell IF kit and raises on bootstrapped fits; `HeterogeneousAdoptionDiDResults` and `HeterogeneousAdoptionDiDEventStudyResults` are pure views since 3.9/M-027 - `'simple'` on the overall class, `'event_study'` on the event-study class, no kit, work on pickles from any release) ### ContinuousDiDResults @@ -2889,7 +2897,12 @@ incremental outcome and its SE (read off summary(), aggregated to the population/window one MMM row represents). Alternatively both exporters take aggregation_result= (the AggregationResult from post-fit results.aggregate('simple'|'group')) + scale=, deriving effect = -att * scale and se = se * scale per row in to_dataframe() order; +att * scale and se = se * scale per row in to_dataframe() order, OR a +results.aggregate('total') container ALONE (3.10: its single row is the +estimator-owned total incremental outcome - scale is REJECTED, and for +overall totals this supersedes scale="auto", eliminating the documented +Imputation/TwoStage raw-support overcount for that use; the total remains +meaningful only for additive-level outcomes); scale="auto" (per-row n off the container) is honored ONLY for ImputationDiD/TwoStageDiD fits - it acknowledges an additive-level outcome, an unweighted fit, and fully identified effects - and every @@ -2965,24 +2978,28 @@ print(prior.to_code( # ready-to-paste PriorDistribution + )) # full_model_window=True; sets # media_prior_type accordingly. -# Container route (ImputationDiD/TwoStageDiD): derive the totals from the -# post-fit aggregation instead of hand-scoping. +# Container route: the estimator-owned total needs no scale at all (3.10; +# CS/EfficientDiD/ImputationDiD/TwoStageDiD, panel non-survey fits). res = ImputationDiD().fit(data, outcome="outcome", unit="unit", time="period", first_treat="first_treat") prior2 = to_meridian_roi_prior( - aggregation_result=res.aggregate("simple"), - scale="auto", # att x n (n = treated unit-periods here) + aggregation_result=res.aggregate("total"), # already the total spend=200_000.0, ) +# 'simple'/'group' containers remain available with scale= (or scale="auto" +# on ImputationDiD/TwoStageDiD, e.g. for per-cohort 'group' exports). ``` Key points: - Container mode: aggregation_result= accepts ONLY AggregationResult ('simple' = one experiment row; 'group' = one per cohort, rows in - to_dataframe() order - per-row scale/spend/x/delta_x align to it). - scale= is always required: numeric, or "auto" for the audited + to_dataframe() order - per-row scale/spend/x/delta_x align to it; + 'total' = one already-scaled total row, NO scale accepted). + scale= is required for 'simple'/'group' containers: numeric, or "auto" + for the audited ImputationDiD/TwoStageDiD producers only (CS/EfficientDiD/StackedDiD - n counts units/cells, not treated unit-periods - pass a numeric scale). + n counts units/cells, not treated unit-periods - use + results.aggregate('total') where supported, else a numeric scale). A 'group' container's cohorts come from ONE fit: widen se_widening for Meridian pooling, and note PyMC scores each row as independent evidence. - Lift frame: one row per experiment, columns [channel, *dims, x, delta_x, diff --git a/diff_diff/guides/llms-practitioner.txt b/diff_diff/guides/llms-practitioner.txt index 4015f8874..363018671 100644 --- a/diff_diff/guides/llms-practitioner.txt +++ b/diff_diff/guides/llms-practitioner.txt @@ -362,8 +362,9 @@ estimated pre-periods exist). results.aggregate('event_study'))` - no refit needed (the fit-time `aggregate=` route is deprecated since 3.9). EXCEPTION: on a BOOTSTRAPPED fit (`n_bootstrap > 0`) the post-fit recompute levels - (`'event_study'`/`'group'`) raise, while `aggregate('simple')` - relays the stored bootstrap inference; use the retained fit-time + (`'event_study'`/`'group'`) raise, while `aggregate('simple')` and, + where supported, `aggregate('total')` + relay the stored bootstrap inference; use the retained fit-time `aggregate='event_study'` for a bootstrapped event-study surface. - For dCDH: requires `L_max >= 1` (multi-horizon mode). Bounds use placebo estimates `DID^{pl}_l` as pre-period coefficients rather than standard @@ -441,9 +442,13 @@ print(results.aggregate('group').to_dataframe()) # Per-cohort ATTs print(results.aggregate('event_study').to_dataframe()) # Dynamic effects # balance_e moves onto aggregate() too: print(results.aggregate('event_study', balance_e=2).to_dataframe()) +# NEW in 3.10 - the estimator-owned TOTAL incremental outcome (exact +# relay C x overall; single row; panel non-survey fits only): +print(results.aggregate('total').to_dataframe()) # EXCEPTION: on a BOOTSTRAPPED fit the RECOMPUTE levels # (event_study/group) raise on CS/EfficientDiD/ImputationDiD/TwoStageDiD -# — aggregate('simple') relays the stored bootstrap inference; use the +# — aggregate('simple') and, where supported, aggregate('total') relay +# the stored bootstrap inference; use the # deprecated fit-time aggregation for a bootstrapped ES/group surface: results = cs.fit(data, ..., aggregate='all') ``` diff --git a/diff_diff/guides/llms.txt b/diff_diff/guides/llms.txt index cf89bfae7..fb2c8b78e 100644 --- a/diff_diff/guides/llms.txt +++ b/diff_diff/guides/llms.txt @@ -21,7 +21,7 @@ diagnostic steps produces unreliable results. 4. **Choose estimator** — staggered adoption → CS/SA/BJS (NOT plain TWFE); few treated units → SDiD; factor confounding → TROP; simple 2x2 → DiD. Run `BaconDecomposition` to diagnose TWFE bias. 5. **Estimate** — `estimator.fit(data, ...)`. Always print the cluster count first and choose inference method based on the result (cluster-robust if >= 50 clusters, wild bootstrap if fewer — for DifferenceInDifferences pass `cluster=`; TwoWayFixedEffects auto-clusters at unit level). 6. **Sensitivity analysis** — `compute_honest_did(results)` for bounds under PT violations (MultiPeriodDiD, CS, or dCDH natively; the TwoWayFixedEffects `event_study=True` surface and a StackedDiD `results.aggregate('event_study')` container also admit - Stacked needs `kappa_pre >= 2` so estimated pre-periods exist), `run_all_placebo_tests()` for 2x2 falsification, specification comparisons for staggered designs. -7. **Heterogeneity** — CS: `results.aggregate('group')`/`.aggregate('event_study')` post-fit, no refit (fit-time `aggregate=`/`balance_e=` are deprecated since 3.9, removed in 4.0; `compute_honest_did` / `compute_pretrends_power` / `plot_event_study` all accept the post-fit `results.aggregate('event_study')` container directly; EXCEPTION: on a BOOTSTRAPPED CS fit the recompute levels `'event_study'`/`'group'` raise while `.aggregate('simple')` relays the stored bootstrap inference (NaN df column); use the fit-time aggregation for a bootstrapped event-study surface); dCDH: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (bootstrap fits included — pure views); SA: `results.event_study_effects`/`to_dataframe(level='cohort')`; StackedDiD: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (the surface is ALWAYS computed at fit since 3.9 - row M-024 - and the container admits into `compute_honest_did`/`compute_pretrends_power` with `kappa_pre >= 2`); EDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit, RECOMPUTED from retained EIFs (3.9, row M-023; fit-time `aggregate=`/`balance_e=` deprecated; on bootstrapped EDiD fits the recompute levels raise while `.aggregate('simple')` relays the stored bootstrap inference - use the fit-time aggregation for a bootstrapped ES/group surface; EDiD containers are NOT admitted into honest/pretrends - no joint ES covariance); BJS/TwoStageDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit on ImputationDiD and TwoStageDiD too (3.9, rows M-021/M-022; recomputed from panel-backed kits, `balance_e=` on `aggregate('event_study')`; on bootstrapped fits the recompute levels raise while `.aggregate('simple')` relays the stored bootstrap inference - use the deprecated fit-time aggregation for a bootstrapped ES/group surface; their containers are not admitted into honest/pretrends - Imputation by design, TwoStage deferred pending a normalization derivation); CGBS continuous: ContinuousDiD is a MIXED adopter (3.9, row M-025) - `results.aggregate('dose')` (ATT(d)+ACRT(d) rows) and `.aggregate('simple')` (att+acrt rows) are views over the always-computed curves and work on ANY fit incl. bootstrapped, while `.aggregate('event_study')` recomputes the binarized event study from a pruned per-cell IF kit and raises on bootstrapped fits (use the deprecated fit-time `aggregate='eventstudy'` there until 4.0; its container is not admitted into honest/pretrends - no joint ES covariance and no reference normalization); HAD: `results.aggregate('simple')` (overall two-period fits; the target column carries the WAS estimand label) / `.aggregate('event_study')` (multi-period fits) - pure views, work on any fit (3.9, rows M-027/M-139; fit() selects the mode from the panel shape; HAD containers are not admitted into honest/pretrends - no joint cross-horizon covariance, deferred); subgroup re-estimation. +7. **Heterogeneity** — CS: `results.aggregate('group')`/`.aggregate('event_study')` post-fit, no refit (fit-time `aggregate=`/`balance_e=` are deprecated since 3.9, removed in 4.0; `compute_honest_did` / `compute_pretrends_power` / `plot_event_study` all accept the post-fit `results.aggregate('event_study')` container directly; EXCEPTION: on a BOOTSTRAPPED CS fit the recompute levels `'event_study'`/`'group'` raise while `.aggregate('simple')` and, where supported, `.aggregate('total')` relay the stored bootstrap inference (NaN df column); use the fit-time aggregation for a bootstrapped event-study surface). NEW in 3.10: `.aggregate('total')` on CS/EfficientDiD/ImputationDiD/TwoStageDiD - the estimator-owned total incremental outcome (exact relay C x overall over the finite-masked complete-case support; single target='total' row; bootstrap-safe RELAY; panel non-survey fits only - repeated-cross-section and declared survey_design fits raise NotImplementedError with the reason); dCDH: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (bootstrap fits included — pure views); SA: `results.event_study_effects`/`to_dataframe(level='cohort')`; StackedDiD: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (the surface is ALWAYS computed at fit since 3.9 - row M-024 - and the container admits into `compute_honest_did`/`compute_pretrends_power` with `kappa_pre >= 2`); EDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit, RECOMPUTED from retained EIFs (3.9, row M-023; fit-time `aggregate=`/`balance_e=` deprecated; on bootstrapped EDiD fits the recompute levels raise while `.aggregate('simple')` relays the stored bootstrap inference - use the fit-time aggregation for a bootstrapped ES/group surface; EDiD containers are NOT admitted into honest/pretrends - no joint ES covariance); BJS/TwoStageDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit on ImputationDiD and TwoStageDiD too (3.9, rows M-021/M-022; recomputed from panel-backed kits, `balance_e=` on `aggregate('event_study')`; on bootstrapped fits the recompute levels raise while `.aggregate('simple')` relays the stored bootstrap inference - use the deprecated fit-time aggregation for a bootstrapped ES/group surface; their containers are not admitted into honest/pretrends - Imputation by design, TwoStage deferred pending a normalization derivation); CGBS continuous: ContinuousDiD is a MIXED adopter (3.9, row M-025) - `results.aggregate('dose')` (ATT(d)+ACRT(d) rows) and `.aggregate('simple')` (att+acrt rows) are views over the always-computed curves and work on ANY fit incl. bootstrapped, while `.aggregate('event_study')` recomputes the binarized event study from a pruned per-cell IF kit and raises on bootstrapped fits (use the deprecated fit-time `aggregate='eventstudy'` there until 4.0; its container is not admitted into honest/pretrends - no joint ES covariance and no reference normalization); HAD: `results.aggregate('simple')` (overall two-period fits; the target column carries the WAS estimand label) / `.aggregate('event_study')` (multi-period fits) - pure views, work on any fit (3.9, rows M-027/M-139; fit() selects the mode from the panel shape; HAD containers are not admitted into honest/pretrends - no joint cross-horizon covariance, deferred); subgroup re-estimation. 8. **Robustness** — compare 2-3 estimators (CS vs SA vs BJS), MUST report with and without covariates (shows whether conditioning drives identification), present pre-trends and sensitivity bounds. After estimation, call `practitioner_next_steps(results)` for context-aware @@ -90,7 +90,7 @@ The site is organized into 5 sections, each with a landing page: - [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html): Rambachan & Roth (2023) sensitivity analysis — robust CI under parallel trends violations, breakdown values - [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html): Roth (2022) Section II.A-B no-individually-significant (NIS) box-probability pretest power + minimum detectable violation; `pretest_form='nis'` (default) implements the paper's primary form, `pretest_form='wald'` retained as paper-supported alternative (Propositions 1+3+4 all apply); linear-violation MDV in Roth's γ units when relative-time labels are threaded through `fit()`; full Σ_22 routing on non-bootstrap CallawaySantAnna and SunAbraham adapters and on admitted CS-/StackedDiD-sourced `aggregate('event_study')` containers (StackedDiD persists its ES VCV in every inference mode) - [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html): Analytical and simulation-based power analysis — MDE, sample size, power curves for study design -- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html): Assemble Marketing Mix Model calibration inputs from experiment results. Two input routes: explicit already-scoped numbers, or the pinned `AggregationResult` container from post-fit `results.aggregate('simple'|'group')` via `aggregation_result=` + `scale=` (effect = att x scale per row, in `to_dataframe()` order; `scale="auto"` reads the container's treated-obs count and is honored ONLY for ImputationDiD/TwoStageDiD fits - it acknowledges an additive-level outcome, an unweighted fit, and fully identified effects; other estimators and raw results objects fail closed). `to_pymc_marketing_lift_test(channel, x, delta_x, delta_y=, sigma=, aggregation_result=, scale=, dims=, on_wrong_sign=)` builds the PyMC-Marketing/prophetverse lift-test DataFrame with sign/zero/positivity guards. `to_meridian_roi_prior(incremental_outcome=, incremental_outcome_se=, aggregation_result=, scale=, spend, parameter="roi_m"|"mroi_m", se_widening=)` builds Google Meridian lognormal ROI priors (spend-weighted pooling, lognormal parity with `lognormal_dist_from_mean_std`, channel- and time-scoped `.to_code()` snippet setting `media_prior_type` - time scope via `meridian_calibration_mask(media_times=, media_channels=, channel=, window=)`, which builds the boolean (n_media_times, n_media_channels) mask (window = inclusive (start, end) tuple or explicit-labels list; experiment channel True on the window, every other channel ALL-TRUE per Meridian's documented convention; roi_m priors only - Meridian rejects the mask for mroi_m) and whose array `.to_code()` serializes into the snippet, or via a mask expression string, or full_model_window=True; widen `se_widening` when pooling a group-level container's same-fit cohorts). Pure numpy/pandas; imports no MMM package; never calls `aggregate()` itself. +- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html): Assemble Marketing Mix Model calibration inputs from experiment results. Two input routes: explicit already-scoped numbers, or the pinned `AggregationResult` container from post-fit `results.aggregate('simple'|'group')` via `aggregation_result=` + `scale=`, or a `results.aggregate('total')` container ALONE (its single row is the estimator-owned total; scale is rejected - for overall totals this supersedes scale="auto") (effect = att x scale per row, in `to_dataframe()` order; `scale="auto"` reads the container's treated-obs count and is honored ONLY for ImputationDiD/TwoStageDiD fits - it acknowledges an additive-level outcome, an unweighted fit, and fully identified effects; other estimators and raw results objects fail closed). `to_pymc_marketing_lift_test(channel, x, delta_x, delta_y=, sigma=, aggregation_result=, scale=, dims=, on_wrong_sign=)` builds the PyMC-Marketing/prophetverse lift-test DataFrame with sign/zero/positivity guards. `to_meridian_roi_prior(incremental_outcome=, incremental_outcome_se=, aggregation_result=, scale=, spend, parameter="roi_m"|"mroi_m", se_widening=)` builds Google Meridian lognormal ROI priors (spend-weighted pooling, lognormal parity with `lognormal_dist_from_mean_std`, channel- and time-scoped `.to_code()` snippet setting `media_prior_type` - time scope via `meridian_calibration_mask(media_times=, media_channels=, channel=, window=)`, which builds the boolean (n_media_times, n_media_channels) mask (window = inclusive (start, end) tuple or explicit-labels list; experiment channel True on the window, every other channel ALL-TRUE per Meridian's documented convention; roi_m priors only - Meridian rejects the mask for mroi_m) and whose array `.to_code()` serializes into the snippet, or via a mask expression string, or full_model_window=True; widen `se_widening` when pooling a group-level container's same-fit cohorts). Pure numpy/pandas; imports no MMM package; never calls `aggregate()` itself. - Conley spatial HAC SE (`vcov_type="conley"`) on cross-sectional `LinearRegression` / `compute_robust_vcov` PLUS panel `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` (with `conley_lag_cutoff=` for within-unit Bartlett temporal HAC) — Conley (1999) spatial-correlation-aware SEs with haversine/euclidean/callable distance metric and Bartlett/uniform spatial kernel; panel path uses the R `conleyreg`-form block-decomposed sandwich (within-period spatial + within-unit Bartlett serial, same-time excluded); parity vs R `conleyreg` (Düsterhöft 2021) on cross-sectional AND panel `lag_cutoff > 0` fixtures. Combining with explicit `cluster=` applies the combined spatial + cluster product kernel `K_total[i,j] = K_space · 1{c_i = c_j}` (cluster must be constant within each unit across periods on the panel path; validator-enforced). DiD takes `unit=` as a fit-time kwarg when `vcov_type="conley"` (not on `__init__`). Sparse k-d-tree fast path auto-activates for `n > 5_000` with bartlett kernel + haversine/euclidean metric ## Tutorials diff --git a/diff_diff/imputation.py b/diff_diff/imputation.py index a1db4999f..d71edd2f5 100644 --- a/diff_diff/imputation.py +++ b/diff_diff/imputation.py @@ -288,7 +288,8 @@ def fit( aggregate : str, optional DEPRECATED (3.9, removed in 4.0; row M-021): aggregate as a post-fit step instead — ``results.aggregate('event_study')`` / - ``.aggregate('group')`` / ``.aggregate('simple')``. Supplying + ``.aggregate('group')`` / ``.aggregate('simple')`` / + ``.aggregate('total')``. Supplying ANY value (``None`` included) warns; the deprecated path still works and returns exactly the numbers it always did (fit-time mode: None/"simple" overall only, "event_study", @@ -336,7 +337,7 @@ def fit( "removed in 4.0. Fit once, then aggregate as a post-fit " "step: results = ImputationDiD().fit(...); " "results.aggregate('event_study') / .aggregate('group') / " - ".aggregate('simple'). balance_e moves onto aggregate() " + ".aggregate('simple') / .aggregate('total'). balance_e moves onto aggregate() " "alongside it: results.aggregate('event_study', " "balance_e=2).", FutureWarning, @@ -1351,6 +1352,16 @@ def _build_imputation_aggregation_kit( "treatment_groups": list(treatment_groups), "overall_att": float(overall_att), "n_treated_obs": int(n_treated_obs), + # 'total' mass: the finite-tau complete-case support, SNAPSHOT at kit + # build - the kit's "df" above is a live reference to the _fit_data + # frame, so a post-fit frame edit must not be able to move the total + # (the snapshot-discipline rule; missing on legacy kits -> the total + # route fails closed with the refit message). + "total_support": float( + np.isfinite( + np.asarray(fit_data["df"].loc[fit_data["omega_1_mask"], "_tau_hat"], dtype=float) + ).sum() + ), "uses_replicate": bool(uses_replicate), "horizon_max": horizon_max, "pretrends": pretrends, diff --git a/diff_diff/imputation_results.py b/diff_diff/imputation_results.py index 1743e645a..3881b8279 100644 --- a/diff_diff/imputation_results.py +++ b/diff_diff/imputation_results.py @@ -12,7 +12,7 @@ import numpy as np import pandas as pd -from diff_diff.aggregation import AggregationMixin, AggregationResult +from diff_diff.aggregation import AggregationMixin, AggregationResult, build_total_relay_row from diff_diff.imputation_aggregation import _ImputationAggregationMixin from diff_diff.results import _format_survey_block, _get_significance_stars from diff_diff.results_base import BaseResults, build_event_study_surface @@ -205,7 +205,7 @@ class ImputationDiDResults(BaseResults, AggregationMixin): # Post-fit aggregation vocabulary (M-021). balance_e keeps the mixin # default ("event_study",) - CS precedent, do not redeclare. - _AGGREGATE_SUPPORTED = ("simple", "event_study", "group") + _AGGREGATE_SUPPORTED = ("simple", "event_study", "group", "total") # --- Inference-field aliases (balance/external-adapter compatibility) --- @property @@ -250,14 +250,17 @@ def _aggregate_compute( # honored by the relay's NaN df column.) if level == "simple": return self._aggregate_simple_result(kit) + if level == "total": + return self._aggregate_total_result(kit) if self.bootstrap_results is not None: raise NotImplementedError( f"aggregate({level!r}) is not yet available on a bootstrapped " "fit (n_bootstrap > 0): the per-target bootstrap draws are " "not retained, so post-fit re-aggregation cannot replay " "percentile inference and analytical inference would " - "misrepresent the fit. aggregate('simple') relays the stored " - "bootstrap inference and remains available; otherwise re-fit " + "misrepresent the fit. aggregate('simple') and, where " + "supported, aggregate('total') relay the stored " + "bootstrap inference and remain available; otherwise re-fit " "with the aggregation you need, or use n_bootstrap=0." ) bk = dict(kit.bookkeeping) @@ -420,6 +423,50 @@ def _aggregate_simple_result(self, kit: Any) -> AggregationResult: estimator=type(self).__name__.replace("Results", ""), ) + def _aggregate_total_result(self, kit: Any) -> AggregationResult: + """The estimator-owned total incremental outcome as a one-row table. + + Exact relay ``C x overall`` CONDITIONAL on the realized aggregation + mass, with ``C`` the FINITE-tau complete-case support snapshot + (``total_support``, stashed at kit build) - the finite support is + what the overall averages over, so ``C x overall = sum(tau)`` + exactly, fixing the documented raw-|Omega_1| overcount of the + ``scale="auto"`` route ('simple''s ``n`` stays raw by contract). + Zero support (the all-unidentified fit, where overall is already + NaN) maps to a NaN mass -> all-NaN row. Fails closed on fits + declaring a ``survey_design=`` (see the REGISTRY Note). + """ + bk = kit.bookkeeping + support = bk.get("total_support") + if support is None: + raise NotImplementedError( + "aggregate('total') needs fit-time state this result " + "predates: it was fitted before aggregate('total') existed - " + "refit to use it." + ) + if bk["survey_metadata"] is not None: + raise NotImplementedError( + "aggregate('total') is not available on fits declaring a " + "survey_design: the realized-mass relay omits the survey " + "mass-uncertainty variance term, and design-aware " + "population-scale totals are not implemented (retained " + "weight scale differs by design family) - tracked in " + "DEFERRED.md. For an unweighted clustered fit, cluster= " + "(without survey_design) supports totals." + ) + mass = float(support) if support > 0 else float(np.nan) + return build_total_relay_row( + mass=mass, + att=self.overall_att, + se=self.overall_se, + t_stat=self.overall_t_stat, + p_value=self.overall_p_value, + conf_int=self.overall_conf_int, + df=(np.nan if self.bootstrap_results is not None else bk["survey_df_final"]), + alpha=kit.alpha, + estimator=type(self).__name__.replace("Results", ""), + ) + def _group_effects_to_aggregation( self, effects: Dict[Any, Dict[str, Any]], kit: Any ) -> AggregationResult: diff --git a/diff_diff/mmm.py b/diff_diff/mmm.py index 079d24219..d4739c9b1 100644 --- a/diff_diff/mmm.py +++ b/diff_diff/mmm.py @@ -29,8 +29,13 @@ result's ``summary()``, aggregated to the population and window their MMM row represents). Alternatively, both exporters accept ``aggregation_result=`` - the pinned :class:`~diff_diff.aggregation.AggregationResult` container returned by -``results.aggregate('simple')`` / ``results.aggregate('group')`` - together with -``scale=``, and derive ``effect = att * scale`` / ``se_out = se * scale`` per row. +``results.aggregate('simple')`` / ``results.aggregate('group')`` (with ``scale=``, +deriving ``effect = att * scale`` / ``se_out = se * scale`` per row) or by +``results.aggregate('total')`` (whose single row is the estimator-owned total +incremental outcome and takes NO scale - for overall-total exports this route +supersedes ``scale="auto"``: its finite-masked support eliminates the documented +Imputation/TwoStage raw-support overcount for that use; the total is meaningful +only for outcomes additive in levels, the same caveat every scale route carries). The derivation is fail-closed: raw results objects are rejected (this module never calls ``aggregate()`` itself), and ``scale="auto"`` (reading the row count off the container) is honored only for the audited producers whose ``n`` matches the ATT's @@ -80,12 +85,18 @@ # Container-mode routing (audited against each producer's aggregation source): # scale="auto" reads per-row n off the container ONLY for these producers, whose n # matches the treated observations (unit-periods) the ATT averages over on -# unweighted, fully identified fits, at both -# supported levels. Routing keys on AggregationResult.estimator - NEVER on n_kind +# unweighted, fully identified fits, at the 'simple' and 'group' levels +# ('total' containers arrive already scaled and take neither scale nor "auto"). +# Routing keys on AggregationResult.estimator - NEVER on n_kind # alone: CallawaySantAnna repeated-cross-section fits report n_kind="obs" with # n = treated+control observations, so n_kind is only a drift sanity guard here. _SCALE_AUTO_ESTIMATORS = frozenset({"ImputationDiD", "TwoStageDiD"}) -_CONTAINER_LEVELS = ("simple", "group") +_CONTAINER_LEVELS = ("simple", "group", "total") +# The audited aggregate('total') producers: only their total containers are +# admitted (the "already scaled" claim is provenance-gated like "auto" - +# StackedDiD is staged out and unknown provenance could be any hand-built +# container). A new adopter extends this set in the PR that ships its total. +_TOTAL_ESTIMATORS = frozenset({"CallawaySantAnna", "EfficientDiD", "ImputationDiD", "TwoStageDiD"}) # Why scale="auto" is refused, per audited non-allowlisted producer. The generic # closing prescription in the error is an EXAMPLE for the unweighted additive case, @@ -96,11 +107,22 @@ "container reports treated+control units (treated+control OBSERVATIONS on " "repeated-cross-section fits, where n_kind='obs' still does not mean " "treated unit-periods), and the 'group' container reports contributing " - "(g, t) cells (n_kind='cells')." + "(g, t) cells (n_kind='cells'). Where supported, use " + "results.aggregate('total') instead - its single row is the " + "estimator-owned total incremental outcome and needs no scale; on fits " + "it does not support it raises with the reason (repeated-cross-section, " + "declared survey_design, cluster-mass fits with incomplete treated " + "cells, or pre-upgrade results) - there, pass a numeric scale (a " + "caller-defined estimand, not the estimator-owned complete-case total)." ), "EfficientDiD": ( "EfficientDiD's n counts disjoint treated+control units ('simple') or " - "contributing (g, t) cells ('group'), not treated unit-periods." + "contributing (g, t) cells ('group'), not treated unit-periods. Where " + "supported, use results.aggregate('total') instead - its single row is " + "the estimator-owned total incremental outcome and needs no scale; its " + "only unsupported routing is a fit declaring a survey_design, which " + "raises with the reason - there, pass a numeric scale (a caller-defined " + "estimand, not the estimator-owned complete-case total)." ), "StackedDiD": ( "StackedDiD's n is a deduplicated distinct-treated-unit count - units, not " @@ -254,7 +276,8 @@ def _extract_aggregation_rows( if isinstance(aggregation_result, BaseResults): raise TypeError( f"aggregation_result must be an AggregationResult - the container " - f"returned by res.aggregate('simple') or res.aggregate('group') on " + f"returned by res.aggregate('simple'), res.aggregate('group'), or " + f"res.aggregate('total') on " f"estimators that produce one; got " f"{type(aggregation_result).__name__}. EventStudyResults and " f"estimators whose aggregate() does not return that container " @@ -265,14 +288,16 @@ def _extract_aggregation_rows( ) raise TypeError( f"aggregation_result must be an AggregationResult (the return value of " - f"res.aggregate('simple') or res.aggregate('group')); got " + f"res.aggregate('simple'), res.aggregate('group'), or " + f"res.aggregate('total')); got " f"{type(aggregation_result).__name__}. Only AggregationResult is " f"supported." ) level = aggregation_result.level if level not in _CONTAINER_LEVELS: raise ValueError( - f"aggregation_result.level must be 'simple' or 'group'; got {level!r}. " + f"aggregation_result.level must be 'simple', 'group', or 'total'; got " + f"{level!r}. " f"Other levels ('calendar', 'dose', estimator-specific extras) have no " f"defined MMM-experiment mapping - re-aggregate at a supported level." ) @@ -282,6 +307,72 @@ def _extract_aggregation_rows( "aggregation_result has no rows (the aggregation selected no cells); " "nothing to export" ) + if level == "total": + # A total container's row is ALREADY the total incremental outcome + # (the estimator applied its own finite-masked aggregation mass), so + # the simple/group target+scale machinery below does not apply: the + # row relays as-is and any scale would double-count. Because "already + # scaled" is a PRODUCER claim, admission is provenance-gated exactly + # like scale="auto": only the audited total adopters are trusted + # (StackedDiD is staged out - its total estimand is undefined under + # weighting= variants - and unknown/missing provenance could be any + # hand-built container whose att is NOT a total). + estimator = aggregation_result.estimator + if estimator not in _TOTAL_ESTIMATORS: + raise ValueError( + f"a level='total' container is accepted only from the audited " + f"total adopters {sorted(_TOTAL_ESTIMATORS)}; got estimator " + f"provenance {estimator!r}. Their aggregate('total') is the " + f"only producer whose single row is verifiably the " + f"estimator-owned total (StackedDiD totals are staged out - " + f"see DEFERRED.md); for other sources pass the explicit " + f"{effect_name}/{se_name} numbers, or a 'simple'/'group' " + f"container with a numeric scale." + ) + if len(frame) != 1: + raise ValueError( + f"aggregation_result has level='total' but {len(frame)} rows; " + f"producers emit a single 'total' row, so this container is " + f"out of contract - re-aggregate with results.aggregate('total')" + ) + total_target = frame["target"].tolist()[0] + if total_target != "total": + raise ValueError( + f"a 'total' container's single row must carry target='total'; " + f"got {total_target!r}. The container is rejected whole - " + f"re-aggregate with results.aggregate('total')" + ) + total_label = frame["label"].tolist()[0] + n_kind = aggregation_result.n_kind + weight_arr = aggregation_result.weight + weight_ok = weight_arr is not None and len(weight_arr) == 1 and float(weight_arr[0]) == 1.0 + if total_label != "total" or n_kind != "obs" or not weight_ok: + raise ValueError( + f"a 'total' container must carry the producer contract " + f"label='total', n_kind='obs', weight=[1.0]; got " + f"label={total_label!r}, n_kind={n_kind!r}, " + f"weight={None if weight_arr is None else list(weight_arr)!r}. " + f"The container schema has drifted from the audited contract - " + f"re-aggregate with results.aggregate('total')" + ) + labels = frame["label"].tolist() + atts = [float(v) for v in frame["att"].tolist()] + ses = [float(v) for v in frame["se"].tolist()] + for i, (att_i, se_i) in enumerate(zip(atts, ses)): + if not (math.isfinite(att_i) and math.isfinite(se_i) and se_i > 0): + raise ValueError( + f"aggregation_result row [{i}] (label {labels[i]!r}) has " + f"att={att_i!r}, se={se_i!r}; the fit carries no usable point " + f"estimate/SE for this row, so it cannot calibrate an MMM" + ) + if scale is not None: + raise ValueError( + "scale is not accepted with a level='total' container: this " + "container's rows are already totals; scale would double-count " + "- pass the container alone (aggregate('total') already " + "applied the estimator's own finite-masked aggregation mass)" + ) + return atts, ses, labels if level == "simple": target_list = list(frame["target"].tolist()) offending = sorted({t for t in target_list if target_list.count(t) > 1}) @@ -374,7 +465,9 @@ def _extract_aggregation_rows( f"unweighted additive fit> (scalar or one value per row in " f"to_dataframe() order), or scale='auto' (ImputationDiD/TwoStageDiD " f"fits only - see the {effect_name} docstring for the assumptions " - f"'auto' acknowledges)" + f"'auto' acknowledges). For an overall total with no scale at all, " + f"pass results.aggregate('total') where the estimator supports it - " + f"it supersedes scale='auto' for total-report use" ) effects = [att_i * s_i for att_i, s_i in zip(atts, scales)] ses_out = [se_i * s_i for se_i, s_i in zip(ses, scales)] @@ -404,10 +497,11 @@ def to_pymc_marketing_lift_test( **Two input routes.** Either the caller supplies the scoped effect explicitly - ``delta_y`` and ``sigma``, the measured incremental outcome and its standard error already aggregated to the population and time window ONE target-MMM row - represents - or passes ``aggregation_result=`` (the pinned container returned by - ``results.aggregate('simple')`` / ``results.aggregate('group')``) together with - ``scale=``, and this function derives ``delta_y = att * scale`` and - ``sigma = se * scale`` per container row. Rescaling is performed ONLY under that + represents - or passes ``aggregation_result=`` - the pinned container returned by + ``results.aggregate('simple')`` / ``results.aggregate('group')`` (together with + ``scale=``, this function deriving ``delta_y = att * scale`` and + ``sigma = se * scale`` per container row) or by ``results.aggregate('total')`` + (a single already-scaled total row; NO scale accepted). Rescaling is performed ONLY under that explicit contract - reconciliation context the container cannot carry (the MMM's row granularity, the outcome's scale) remains the caller's acknowledgement, see ``scale``. Either way, PyMC-Marketing scores one row's ``delta_y`` against @@ -434,14 +528,17 @@ def to_pymc_marketing_lift_test( ``delta_y`` unless ``aggregation_result`` is given. aggregation_result : AggregationResult, optional The pinned container returned by ``results.aggregate('simple')`` (one - experiment row) or ``results.aggregate('group')`` (one row per cohort). - Mutually exclusive with ``delta_y``/``sigma``; requires ``scale``. Rows are + experiment row), ``results.aggregate('group')`` (one row per cohort), or + ``results.aggregate('total')`` (one already-scaled total row). + Mutually exclusive with ``delta_y``/``sigma``; requires ``scale`` for + 'simple'/'group' containers and FORBIDS it for 'total'. Rows are consumed in ``aggregation_result.to_dataframe()`` order - the order ``summary()`` prints - and every per-row sequence kwarg (``scale``, ``x``, ``delta_x``, ``dims``, ...) aligns to that order. Raw results objects and ``EventStudyResults`` are rejected; this function never calls ``aggregate()`` itself. On a bootstrapped fit, ``aggregate('simple')`` - relays the stored percentile SE, which is used as-is. Group-container + and, where supported, ``aggregate('total')`` relay the stored percentile + SE, which is used as-is. Group-container cautions: (1) cohort rows come from ONE fit sharing controls and windows, yet each emitted lift row is scored by PyMC-Marketing as an independent observation with only its marginal ``sigma`` - the omitted cross-cohort @@ -457,7 +554,9 @@ def to_pymc_marketing_lift_test( scale : float, sequence of float, or "auto", optional Converts each container row's per-observation ATT to the row's total incremental outcome: ``delta_y = att * scale``, ``sigma = se * scale``. - Required with ``aggregation_result`` (finite, positive; scalar or one value + Required with a 'simple'/'group' ``aggregation_result``; forbidden with a + 'total' container, whose row is already the estimator-owned total + (finite, positive; scalar or one value per row in ``to_dataframe()`` order; e.g. treated units x treated periods for an unweighted additive fit - an example, not a universal formula). ``scale="auto"`` derives ``scale`` from the container's per-row ``n`` and is @@ -515,7 +614,8 @@ def to_pymc_marketing_lift_test( raise ValueError( "pass either aggregation_result= or delta_y=/sigma=, not both; " "aggregation mode derives delta_y and sigma from the container " - "(delta_y = att * scale, sigma = se * scale)" + "(delta_y = att * scale, sigma = se * scale for 'simple'/'group' " + "containers; a 'total' container's row relays as-is)" ) delta_y, sigma, _ = _extract_aggregation_rows( aggregation_result, scale, effect_name="delta_y", se_name="sigma" @@ -535,9 +635,11 @@ def to_pymc_marketing_lift_test( ) raise ValueError( f"{missing} required when aggregation_result is not given; pass " - f"both delta_y and sigma, or pass " + f"both delta_y and sigma, pass " f"aggregation_result=res.aggregate('simple'|'group') with scale= " - f"to derive them from a fitted result" + f"to derive them from a fitted result, or pass " + f"aggregation_result=res.aggregate('total') (no scale) where the " + f"estimator supports it" ) n = _seq_len(channel, x, delta_x, delta_y, sigma, dims) channels = _broadcast("channel", channel, n) @@ -980,9 +1082,12 @@ def to_meridian_roi_prior( outcome attributable to the channel's spend against a zero-spend counterfactual (a full holdout), divided here by ``spend`` = the channel's total spend over the window; for ``parameter="mroi_m"`` it is the marginal outcome of the spend - change, divided by that spend change - or passes ``aggregation_result=`` with - ``scale=``, and this function derives ``incremental_outcome = att * scale`` and - ``incremental_outcome_se = se * scale`` per container row. Rescaling happens ONLY + change, divided by that spend change - or passes ``aggregation_result=``: + with ``scale=`` for a 'simple'/'group' container (this function deriving + ``incremental_outcome = att * scale`` and + ``incremental_outcome_se = se * scale`` per container row), or a + ``results.aggregate('total')`` container alone (a single already-scaled + total row; NO scale accepted). Rescaling happens ONLY under that explicit contract; sign, estimand match, and population remain the caller's responsibility. @@ -1000,9 +1105,11 @@ def to_meridian_roi_prior( ``aggregation_result`` is given. aggregation_result : AggregationResult, optional The pinned container returned by ``results.aggregate('simple')`` (one - experiment) or ``results.aggregate('group')`` (one experiment per cohort, - feeding the spend-weighted pooling below). Mutually exclusive with - ``incremental_outcome``/``incremental_outcome_se``; requires ``scale``. Rows + experiment), ``results.aggregate('group')`` (one experiment per cohort, + feeding the spend-weighted pooling below), or ``results.aggregate('total')`` + (one already-scaled total experiment). Mutually exclusive with + ``incremental_outcome``/``incremental_outcome_se``; requires ``scale`` for + 'simple'/'group' containers and FORBIDS it for 'total'. Rows are consumed in ``aggregation_result.to_dataframe()`` order - the order ``summary()`` prints - and per-row sequence kwargs (``scale``, ``spend``) align to that order. Raw results objects and ``EventStudyResults`` are @@ -1023,8 +1130,10 @@ def to_meridian_roi_prior( scale : float, sequence of float, or "auto", optional Converts each container row's per-observation ATT to that experiment's total incremental outcome: ``incremental_outcome = att * scale``, - ``incremental_outcome_se = se * scale``. Required with - ``aggregation_result`` (finite, positive; scalar or one value per row in + ``incremental_outcome_se = se * scale``. Required with a + 'simple'/'group' ``aggregation_result``; forbidden with a 'total' + container, whose row is already the estimator-owned total + (finite, positive; scalar or one value per row in ``to_dataframe()`` order; e.g. treated units x treated periods for an unweighted additive fit - an example, not a universal formula). ``scale="auto"`` derives ``scale`` from the container's per-row ``n`` and is @@ -1088,7 +1197,8 @@ def to_meridian_roi_prior( "incremental_outcome=/incremental_outcome_se=, not both; " "aggregation mode derives them from the container " "(incremental_outcome = att * scale, " - "incremental_outcome_se = se * scale)" + "incremental_outcome_se = se * scale for 'simple'/'group' " + "containers; a 'total' container's row relays as-is)" ) incremental_outcome, incremental_outcome_se, _ = _extract_aggregation_rows( aggregation_result, @@ -1116,8 +1226,10 @@ def to_meridian_roi_prior( ) raise ValueError( f"{missing} required when aggregation_result is not given; pass " - f"both, or pass aggregation_result=res.aggregate('simple'|'group') " - f"with scale= to derive them from a fitted result" + f"both, pass aggregation_result=res.aggregate('simple'|'group') " + f"with scale= to derive them from a fitted result, or pass " + f"aggregation_result=res.aggregate('total') (no scale) where the " + f"estimator supports it" ) n = _seq_len(incremental_outcome, incremental_outcome_se, spend) outcomes = _broadcast("incremental_outcome", incremental_outcome, n) diff --git a/diff_diff/practitioner.py b/diff_diff/practitioner.py index 4c97a2c85..36b12ad61 100644 --- a/diff_diff/practitioner.py +++ b/diff_diff/practitioner.py @@ -398,7 +398,7 @@ def _handle_multi_period(results: Any): def _handle_cs(results: Any): # The post-fit RECOMPUTE levels raise on a bootstrapped fit # (percentile statistics are not retained for re-aggregation; - # 'simple' relays the stored quintet and stays available), so the + # 'simple'/'total' relay the stored quintet and stay available), so the # event-study guidance must route those fits through the retained # fit-time aggregation instead of advice that cannot run. is_bootstrap = getattr(results, "bootstrap_results", None) is not None @@ -407,7 +407,8 @@ def _handle_cs(results: Any): "Bounds the treatment effect under plausible violations of " "parallel trends. This fit is BOOTSTRAPPED, and the post-fit " "event-study/group recompute levels raise on bootstrap fits " - "(aggregate('simple') still relays the stored inference) - " + "(aggregate('simple') and, where supported, aggregate('total') " + "still relay the stored inference) - " "refit with the fit-time aggregation to populate the " "event-study surface." ) @@ -529,7 +530,8 @@ def _handle_imputation(results: Any): if getattr(results, "bootstrap_results", None) is None else "This fit is BOOTSTRAPPED: the post-fit event-study/group " "recompute levels raise on bootstrap fits, while " - "aggregate('simple') relays the stored inference - " + "aggregate('simple') and, where supported, " + "aggregate('total') relay the stored inference - " "refit with the deprecated fit-time aggregation (or " "n_bootstrap=0) to obtain the recomputed surfaces." ), @@ -587,7 +589,8 @@ def _handle_two_stage(results: Any): if getattr(results, "bootstrap_results", None) is None else "This fit is BOOTSTRAPPED: the post-fit event-study/group " "recompute levels raise on bootstrap fits, while " - "aggregate('simple') relays the stored inference - " + "aggregate('simple') and, where supported, " + "aggregate('total') relay the stored inference - " "refit with the deprecated fit-time aggregation (or " "n_bootstrap=0) to obtain the recomputed surfaces." ), @@ -979,7 +982,8 @@ def _handle_efficient(results: Any): if getattr(results, "bootstrap_results", None) is None else "This fit is BOOTSTRAPPED: the post-fit event-study/group " "recompute levels raise on bootstrap fits, while " - "aggregate('simple') relays the stored inference - " + "aggregate('simple') and, where supported, " + "aggregate('total') relay the stored inference - " "refit with the deprecated fit-time aggregation (or " "n_bootstrap=0) to obtain the recomputed surfaces." ), diff --git a/diff_diff/staggered.py b/diff_diff/staggered.py index bc9ffbb68..3233fdfd2 100644 --- a/diff_diff/staggered.py +++ b/diff_diff/staggered.py @@ -1873,8 +1873,9 @@ def fit( fits (``n_bootstrap > 0``), where the post-fit RECOMPUTE levels (``'event_study'``/``'group'``) raise (percentile inference cannot be reproduced from the retained analytical - state; ``aggregate('simple')`` relays the stored bootstrap - inference and stays available) and fit-time aggregation + state; ``aggregate('simple')`` and, where supported, + ``aggregate('total')`` relay the stored bootstrap + inference and stay available) and fit-time aggregation remains the supported route for those levels. Otherwise it remains only as the deprecated compatibility path through 3.9. @@ -1920,7 +1921,7 @@ def fit( "in 4.0. Fit once, then aggregate as a post-fit step: " "results = CallawaySantAnna().fit(...); " "results.aggregate('event_study') / .aggregate('group') / " - ".aggregate('simple'). balance_e moves onto aggregate() alongside " + ".aggregate('simple') / .aggregate('total'). balance_e moves onto aggregate() alongside " "it: results.aggregate('event_study', balance_e=2).", FutureWarning, stacklevel=2, @@ -3028,10 +3029,16 @@ def fit( # Attach the post-fit aggregation kit (spec section 6, rows M-020/M-117). # Built HERE because neither `precomputed` nor `influence_func_info` # survives fit() - they are locals - so the kit cannot be reconstructed - # later. It deliberately excludes the data matrices: re-aggregation reads - # only unit-level bookkeeping, so the source panel is never retained. + # later. It deliberately excludes the data matrices: re-aggregation + # reads unit-level bookkeeping plus the fit-time derived snapshots + # (`agg_gt_cells` is per-(g, t) CELL data for the 'total' replay), so + # the source panel is never retained. self.results_._aggregation_kit = _build_aggregation_kit( - self, precomputed, influence_func_info, group_time_effects + self, + precomputed, + influence_func_info, + group_time_effects, + is_survey_fit=survey_metadata is not None, ) self.is_fitted_ = True @@ -5120,6 +5127,8 @@ def _build_aggregation_kit( precomputed: Optional[Dict[str, Any]], influence_func_info: Optional[Dict[str, Any]], group_time_effects: Optional[Dict[Any, Any]], + *, + is_survey_fit: bool = False, ) -> Optional["AggregationKit"]: """Distil the fit-time state post-fit re-aggregation needs. @@ -5135,6 +5144,21 @@ def _build_aggregation_kit( if key in precomputed: bookkeeping[key] = precomputed[key] + # Fit-time DERIVED keys (never present in ``precomputed``, so they are + # direct assignments rather than whitelist entries). ``agg_gt_cells`` + # snapshots the (g, t) cells 'total' replays over - an immutable tuple, so + # a post-fit edit of the public ``group_time_effects`` cannot move the + # total mass (the EDiD snapshot-discipline rule). ``is_survey_fit`` + # records whether the ORIGINAL fit declared a ``survey_design=`` (the + # bare-``cluster=`` synthesize path keeps ``survey_metadata`` None, and + # kit ``survey_weights``/``resolved_survey_unit`` are populated on that + # admitted routing too, so neither can serve as the survey marker). + bookkeeping["agg_gt_cells"] = tuple( + (g, t, float(data["effect"]), float(data["n_treated"])) + for (g, t), data in group_time_effects.items() + ) + bookkeeping["is_survey_fit"] = bool(is_survey_fit) + # Data minimization: the results object is picklable and users share # result artifacts, so the kit must not turn it into a carrier for raw # unit identifiers (which are routinely names, emails or administrative @@ -5161,10 +5185,10 @@ def _build_aggregation_kit( # Bootstrap replay is not wired: a bootstrapped fit's percentile # inference cannot be reproduced from analytical state, so the # RECOMPUTE levels (event_study/group) fail closed on one rather than - # silently substituting analytical numbers ('simple' relays the - # stored bootstrap inference and stays available - the per-level - # policy converged with row M-027). BootstrapReplaySpec - # (diff_diff/aggregation.py) is the verified mechanism for the - # follow-up. + # silently substituting analytical numbers ('simple' and, where + # supported, 'total' relay the stored bootstrap inference and stay + # available - the per-level policy converged with row M-027). + # BootstrapReplaySpec (diff_diff/aggregation.py) is the verified + # mechanism for the follow-up. bootstrap=None, ) diff --git a/diff_diff/staggered_results.py b/diff_diff/staggered_results.py index 39f2407ff..10c813772 100644 --- a/diff_diff/staggered_results.py +++ b/diff_diff/staggered_results.py @@ -11,10 +11,18 @@ import numpy as np import pandas as pd -from diff_diff.aggregation import AggregationMixin, AggregationResult, resolve_inference_df +from diff_diff.aggregation import ( + AggregationMixin, + AggregationResult, + build_total_relay_row, + resolve_inference_df, +) from diff_diff.results import _format_survey_block, _get_significance_stars from diff_diff.results_base import BaseResults, build_event_study_surface -from diff_diff.staggered_aggregation import CallawaySantAnnaAggregationMixin +from diff_diff.staggered_aggregation import ( + CallawaySantAnnaAggregationMixin, + fixed_cohort_agg_weights, +) if TYPE_CHECKING: from diff_diff.staggered_bootstrap import CSBootstrapResults @@ -294,10 +302,10 @@ def coef_var(self) -> float: # Post-fit aggregation (spec section 6; ledger rows M-020 / M-117) # ------------------------------------------------------------------ # - #: CS implements simple / event-study / group. ``"calendar"`` is part of - #: the library-wide vocabulary but CS has no calendar aggregator (the - #: DEFERRED "Calendar-time aggregation" row), so asking for it raises. - _AGGREGATE_SUPPORTED = ("simple", "event_study", "group") + #: CS implements simple / event-study / group / total. ``"calendar"`` is + #: part of the library-wide vocabulary but CS has no calendar aggregator + #: (the DEFERRED "Calendar-time aggregation" row), so asking for it raises. + _AGGREGATE_SUPPORTED = ("simple", "event_study", "group", "total") def _aggregate_compute( self, level: str, *, weights: Optional[str], balance_e: Optional[int] @@ -310,12 +318,15 @@ def _aggregate_compute( "result unpickled from an older release will not have one." ) # Per-level bootstrap policy (v4-design section 6, converged with row - # M-027): 'simple' is a bit-exact RELAY of the stored overall - # inference - faithful under any inference regime, bootstrap included - # - so it dispatches BEFORE the bootstrap gate. Only the RECOMPUTE - # levels below fail closed on bootstrapped fits. + # M-027): 'simple' and 'total' are bit-exact RELAYS of the stored + # overall inference - faithful under any inference regime, bootstrap + # included (every SE branch is homogeneous of degree 1 in the x C + # scaling) - so they dispatch BEFORE the bootstrap gate. Only the + # RECOMPUTE levels below fail closed on bootstrapped fits. if level == "simple": return self._aggregate_simple_result(kit) + if level == "total": + return self._aggregate_total_result(kit) if self.bootstrap_results is not None: # Fail closed rather than silently handing back analytical numbers: # a bootstrapped fit's se/p/CI are percentile statistics, and @@ -324,8 +335,9 @@ def _aggregate_compute( f"aggregate({level!r}) is not yet available on a bootstrapped " "fit (n_bootstrap > 0): its inference is percentile-bootstrap " "based and cannot be reproduced from the analytical state " - "retained here. aggregate('simple') relays the stored " - "bootstrap inference and remains available; otherwise re-fit " + "retained here. aggregate('simple') and, where supported, " + "aggregate('total') relay the stored " + "bootstrap inference and remain available; otherwise re-fit " "with the aggregation you need, or use n_bootstrap=0 for " "analytical inference." ) @@ -420,6 +432,131 @@ def _aggregate_simple_result(self, kit: Any) -> AggregationResult: estimator=type(self).__name__.replace("Results", ""), ) + def _aggregate_total_result(self, kit: Any) -> AggregationResult: + """The estimator-owned total incremental outcome as a one-row table. + + Exact relay ``C x overall`` CONDITIONAL on the realized aggregation + mass, where ``C`` is the complete-case treated-observation support + the fit's own simple aggregation used (``_cs_total_mass``). Fails + closed - with the reason - on the routings where that mass is not a + complete-case count: repeated-cross-section routing, declared + ``survey_design=`` fits, and bare-``cluster=`` fits whose cohort-mass + weighting diverges from the complete-case count (incomplete treated + cells). See the REGISTRY CS post-fit ``aggregate('total')`` Note. + """ + bk = kit.bookkeeping + # Gate 1 - RC routing. Checked FIRST: a fit that is both + # survey-declared AND RC-routed must get this (more informative) + # message deterministically. + if not bk.get("is_panel", True) or bk.get("agg_cohort_masses") is not None: + raise NotImplementedError( + "aggregate('total') is not available on " + "repeated-cross-section-routed fits: the cohort mass counts " + "units, not treated observations, so a complete-case total " + "is not recoverable from the retained fit state - tracked " + "in DEFERRED.md." + ) + # Panel-routed legacy kits (pre-upgrade pickles) fail closed: the + # fit-time snapshots below are the only mutation-proof carriers, and + # falling back to the mutable public fields would let a post-fit + # edit silently bypass the survey gate or move the mass. + cells = bk.get("agg_gt_cells") + is_survey_fit = bk.get("is_survey_fit") + if cells is None or is_survey_fit is None: + raise NotImplementedError( + "aggregate('total') needs fit-time state this result " + "predates: it was fitted before aggregate('total') existed - " + "refit to use it." + ) + # Gate 2 - declared survey design (immutable fit-time provenance; + # bare-cluster fits synthesize an internal design but keep this + # False, so they are ADMITTED). + if is_survey_fit: + raise NotImplementedError( + "aggregate('total') is not available on fits declaring a " + "survey_design: the realized-mass relay omits the survey " + "mass-uncertainty variance term, and design-aware " + "population-scale totals are not implemented (retained " + "weight scale differs by design family) - tracked in " + "DEFERRED.md. For an unweighted clustered fit, cluster= " + "(without survey_design) supports totals." + ) + ci = self.overall_conf_int or (np.nan, np.nan) + return build_total_relay_row( + mass=self._cs_total_mass(kit), + att=self.overall_att, + se=self.overall_se, + t_stat=self.overall_t_stat, + p_value=self.overall_p_value, + conf_int=ci, + df=(np.nan if self.bootstrap_results is not None else resolve_inference_df(self)), + alpha=self.alpha, + estimator=type(self).__name__.replace("Results", ""), + ) + + def _cs_total_mass(self, kit: Any) -> float: + """The realized simple-aggregation mass ``C``, from kit snapshots. + + Replays ``_aggregate_simple``'s cell selection verbatim + (staggered_aggregation.py: anticipation filter first, finite-effect + mask second, weight = cohort mass where ``fixed_cohort_agg_weights`` + provides one else per-cell complete-case ``n_treated``, pairwise + ``np.sum``) over the immutable ``agg_gt_cells`` fit-time snapshot - + never the mutable public ``group_time_effects``. The ``agg_weight`` + fallback of the source is deliberately absent: that field exists + only on RC-routed cells, which the RC gate excludes, and the 4-tuple + snapshot does not carry it. + + On admitted fits the cohort-mass branch (bare-``cluster=``: the + synthesized all-ones design makes each cohort mass the integer + cohort size) must COINCIDE with the complete-case count - when kept + cells have incomplete treated support the two disagree (the same + overcount that keeps RC routings gated) and this raises rather than + publishing an ambiguous mass. Empty post set or all-NaN effects + return NaN WITHOUT re-emitting the fit-time UserWarnings (the + no-post-fit-re-warn convention; the NaN row is the signal). + """ + bk = kit.bookkeeping + # TRAP: never bk["agg_total_weight"] - that is the RC all-units WIF + # mass, not the kept-cell simple mass. + masses = fixed_cohort_agg_weights(bk) + effects_list: List[float] = [] + weights_list: List[float] = [] + complete_case: List[float] = [] + for g, t, effect, n_treated in bk["agg_gt_cells"]: + if t < g - kit.anticipation: + continue + effects_list.append(effect) + if masses is not None and g in masses: + weights_list.append(masses[g]) + else: + weights_list.append(n_treated) + complete_case.append(n_treated) + if not effects_list: + return float(np.nan) + effects = np.array(effects_list, dtype=float) + weights = np.array(weights_list, dtype=float) + cc = np.array(complete_case, dtype=float) + finite_mask = np.isfinite(effects) + if not np.all(finite_mask): + weights = weights[finite_mask] + cc = cc[finite_mask] + if len(weights) == 0: + return float(np.nan) + total_weight = float(np.sum(weights)) + # Coincidence guard - only meaningful when a finite mass exists and + # the cohort-mass branch was taken. + if masses is not None and np.isfinite(total_weight) and total_weight != float(np.sum(cc)): + raise NotImplementedError( + "aggregate('total') is not available on this fit: the " + "cohort-mass weighting (from cluster=) counts full cohorts " + "per period, but some kept cells have incomplete treated " + "support, so the realized aggregation mass and the " + "complete-case treated-observation count disagree - tracked " + "in DEFERRED.md." + ) + return total_weight + def _group_effects_to_aggregation( self, effects: Dict[Any, Dict[str, Any]], kit: Any ) -> AggregationResult: diff --git a/diff_diff/two_stage.py b/diff_diff/two_stage.py index b60e179cf..7ddb85625 100644 --- a/diff_diff/two_stage.py +++ b/diff_diff/two_stage.py @@ -1387,7 +1387,8 @@ def fit( aggregate : str, optional DEPRECATED (3.9, removed in 4.0; row M-022): aggregate as a post-fit step instead — ``results.aggregate('event_study')`` / - ``.aggregate('group')`` / ``.aggregate('simple')``. Supplying + ``.aggregate('group')`` / ``.aggregate('simple')`` / + ``.aggregate('total')``. Supplying ANY value (``None`` included) warns; the deprecated path still works and returns exactly the numbers it always did (fit-time mode: None/"simple" overall only, "event_study", @@ -1432,7 +1433,7 @@ def fit( "removed in 4.0. Fit once, then aggregate as a post-fit " "step: results = TwoStageDiD().fit(...); " "results.aggregate('event_study') / .aggregate('group') / " - ".aggregate('simple'). balance_e moves onto aggregate() " + ".aggregate('simple') / .aggregate('total'). balance_e moves onto aggregate() " "alongside it: results.aggregate('event_study', " "balance_e=2).", FutureWarning, diff --git a/diff_diff/two_stage_results.py b/diff_diff/two_stage_results.py index 1d1ac824c..39c24d7f2 100644 --- a/diff_diff/two_stage_results.py +++ b/diff_diff/two_stage_results.py @@ -12,7 +12,7 @@ import numpy as np import pandas as pd -from diff_diff.aggregation import AggregationMixin, AggregationResult +from diff_diff.aggregation import AggregationMixin, AggregationResult, build_total_relay_row from diff_diff.results import _format_survey_block, _get_significance_stars from diff_diff.results_base import BaseResults, build_event_study_surface from diff_diff.two_stage_aggregation import _TwoStageAggregationMixin @@ -206,7 +206,7 @@ class TwoStageDiDResults(BaseResults, AggregationMixin): # Post-fit aggregation vocabulary (M-022). balance_e keeps the mixin # default ("event_study",) - CS precedent, do not redeclare. - _AGGREGATE_SUPPORTED = ("simple", "event_study", "group") + _AGGREGATE_SUPPORTED = ("simple", "event_study", "group", "total") # --- Inference-field aliases (balance/external-adapter compatibility) --- @property @@ -251,14 +251,17 @@ def _aggregate_compute( # honored by the relay's NaN df column.) if level == "simple": return self._aggregate_simple_result(kit) + if level == "total": + return self._aggregate_total_result(kit) if self.bootstrap_results is not None: raise NotImplementedError( f"aggregate({level!r}) is not yet available on a bootstrapped " "fit (n_bootstrap > 0): the per-target bootstrap draws are " "not retained, so post-fit re-aggregation cannot replay " "percentile inference and analytical inference would " - "misrepresent the fit. aggregate('simple') relays the stored " - "bootstrap inference and remains available; otherwise re-fit " + "misrepresent the fit. aggregate('simple') and, where " + "supported, aggregate('total') relay the stored " + "bootstrap inference and remain available; otherwise re-fit " "with the aggregation you need, or use n_bootstrap=0." ) bk = dict(kit.bookkeeping) @@ -425,6 +428,48 @@ def _aggregate_simple_result(self, kit: Any) -> AggregationResult: estimator=type(self).__name__.replace("Results", ""), ) + def _aggregate_total_result(self, kit: Any) -> AggregationResult: + """The estimator-owned total incremental outcome as a one-row table. + + Exact relay ``C x overall`` CONDITIONAL on the realized aggregation + mass, with ``C`` the post-filter treatment-indicator support the + second stage actually regressed on: ``D = omega_1_mask`` with + non-finite ``_y_tilde`` rows zeroed (the fit-time masker's inline + mirror - never re-called post-fit, so its warnings are not + re-emitted; the kit frame is a private column-subset COPY, so this + is snapshot-grade). ``score_pad_mask`` is irrelevant here (it pads + scores/variance only; ``omega_1_mask`` governs D). Degenerate + ``D.sum() == 0`` maps to a NaN mass -> all-NaN row (mirrors the + fit's (nan, nan)). Fails closed on fits declaring a + ``survey_design=`` (see the REGISTRY Note). + """ + bk = kit.bookkeeping + if bk["survey_metadata"] is not None: + raise NotImplementedError( + "aggregate('total') is not available on fits declaring a " + "survey_design: the realized-mass relay omits the survey " + "mass-uncertainty variance term, and design-aware " + "population-scale totals are not implemented (retained " + "weight scale differs by design family) - tracked in " + "DEFERRED.md. For an unweighted clustered fit, cluster= " + "(without survey_design) supports totals." + ) + d_mask = np.asarray(bk["omega_1_mask"], dtype=bool) + y_tilde = np.asarray(bk["df"]["_y_tilde"], dtype=float) + support = float(np.sum(d_mask & np.isfinite(y_tilde))) + mass = support if support > 0 else float(np.nan) + return build_total_relay_row( + mass=mass, + att=self.overall_att, + se=self.overall_se, + t_stat=self.overall_t_stat, + p_value=self.overall_p_value, + conf_int=self.overall_conf_int, + df=(np.nan if self.bootstrap_results is not None else bk["survey_df_final"]), + alpha=kit.alpha, + estimator=type(self).__name__.replace("Results", ""), + ) + def _group_effects_to_aggregation( self, effects: Dict[Any, Dict[str, Any]], kit: Any, *, group_df: Optional[int] ) -> AggregationResult: diff --git a/docs/api/efficient_did.rst b/docs/api/efficient_did.rst index 09bb6484f..67cf67da7 100644 --- a/docs/api/efficient_did.rst +++ b/docs/api/efficient_did.rst @@ -122,6 +122,7 @@ Basic usage:: # Aggregate post-fit - recomputed from retained EIFs, no refit: print(results.aggregate('event_study').to_dataframe()) print(results.aggregate('group').to_dataframe()) + print(results.aggregate('total').to_dataframe()) # estimator-owned total (3.10) PT-Post mode (matches CS for post-treatment ATT):: diff --git a/docs/api/imputation.rst b/docs/api/imputation.rst index f7b797e39..a26cbd9cf 100644 --- a/docs/api/imputation.rst +++ b/docs/api/imputation.rst @@ -108,6 +108,7 @@ Event study with visualization:: results = est.fit(data, outcome='outcome', unit='unit', time='period', first_treat='first_treat') es = results.aggregate('event_study') # post-fit (M-021) + total = results.aggregate('total') # estimator-owned total (3.10) plot_event_study(es) Pre-trend test:: diff --git a/docs/api/mmm.rst b/docs/api/mmm.rst index c3646356f..3d3063a5a 100644 --- a/docs/api/mmm.rst +++ b/docs/api/mmm.rst @@ -30,8 +30,9 @@ default route stays fully explicit: the caller supplies the already-scoped incremental outcome and its standard error (the numbers read off a fitted result's ``summary()``, aggregated to the population and window one MMM row represents). Alternatively both exporters accept ``aggregation_result=`` - the pinned -:class:`~diff_diff.AggregationResult` returned by ``results.aggregate('simple')`` -or ``results.aggregate('group')`` - with ``scale=``, deriving +:class:`~diff_diff.AggregationResult` returned by ``results.aggregate('simple')``, +``results.aggregate('group')`` (each with ``scale=``), or +``results.aggregate('total')`` (already the total; no scale) - deriving ``effect = att * scale`` and ``se = se * scale`` per row; ``scale="auto"`` (reading the container's own treated-observation count) is honored only for ImputationDiD and TwoStageDiD fits, and acknowledges assumptions the container cannot verify @@ -95,11 +96,15 @@ Example Deriving totals from a fitted aggregation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -For ImputationDiD and TwoStageDiD fits, the post-fit aggregation container can -supply the effect numbers directly - ``scale="auto"`` multiplies the per-observation -ATT by the container's treated-observation count (an explicit acknowledgement that -the outcome is in additive levels, the fit is unweighted, and every treated -observation's effect is identified; see the docstring): +The primary route (3.10) is ``results.aggregate('total')`` - the +estimator-owned total incremental outcome on CallawaySantAnna, EfficientDiD, +ImputationDiD, and TwoStageDiD (panel non-survey fits): its single row is +already ``C x overall`` over the estimator's finite-masked complete-case +support, so the exporter takes the container alone and rejects any ``scale``. +For overall-total exports this supersedes ``scale="auto"``; ``"auto"`` remains +the route for ImputationDiD/TwoStageDiD ``group``-level (per-cohort) +containers. Either way the outcome must be in additive levels (see the +docstrings): .. code-block:: python @@ -114,10 +119,13 @@ observation's effect is identified; see the docstring): channel='tv', x=50_000.0, delta_x=20_000.0, - aggregation_result=res.aggregate('simple'), # one experiment row - scale='auto', # ImputationDiD's n IS the treated unit-periods + aggregation_result=res.aggregate('total'), # already the total; no scale ) - # delta_y == att * n, sigma == se * n, then the usual guards apply. + # delta_y == the total row's att, sigma == its se; the usual guards apply. + # Scaled-container alternatives: aggregation_result=res.aggregate('simple') + # (one overall row) or res.aggregate('group') (per-cohort rows) with + # scale='auto' - ImputationDiD's n IS the treated unit-periods there, at + # the cost of the documented raw-support caveat. to_meridian_roi_prior --------------------- @@ -200,8 +208,11 @@ narrower period: Deriving totals from a fitted aggregation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Estimators outside the ``scale="auto"`` allowlist still hand the exporter their -effect and SE via the container - the caller supplies the numeric scale, because +Estimators outside the ``scale="auto"`` allowlist reach the exporter through +``results.aggregate('total')`` too - where supported, that container needs no +scale at all. On the routings totals do not support (repeated-cross-section, +declared ``survey_design=``, divergent bare-``cluster=`` masses), the caller +supplies a numeric scale instead - a caller-defined estimand, needed because those containers' ``n`` does not count treated unit-periods (CallawaySantAnna's ``simple`` container, for example, counts treated *and* control units): diff --git a/docs/api/staggered.rst b/docs/api/staggered.rst index ebfe17313..771d56e11 100644 --- a/docs/api/staggered.rst +++ b/docs/api/staggered.rst @@ -52,6 +52,11 @@ Results container for Callaway-Sant'Anna estimation. ~CallawaySantAnnaResults.summary ~CallawaySantAnnaResults.to_dataframe + ``aggregate()`` levels: ``'simple'``, ``'event_study'``, ``'group'``, and + ``'total'`` (3.10) - the estimator-owned total incremental outcome + (``C x overall`` over the finite-masked complete-case support; panel + non-survey fits only, consumed by :mod:`diff_diff.mmm` with no scale). + GroupTimeEffect --------------- diff --git a/docs/api/two_stage.rst b/docs/api/two_stage.rst index 501bd4e4e..a6a2072d2 100644 --- a/docs/api/two_stage.rst +++ b/docs/api/two_stage.rst @@ -111,6 +111,7 @@ Event study with visualization:: results = est.fit(data, outcome='outcome', unit='unit', time='period', first_treat='first_treat') es = results.aggregate('event_study') # post-fit (M-022) + total = results.aggregate('total') # estimator-owned total (3.10) plot_event_study(es) Comparison with other estimators:: diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 70b225b8d..b8dac6976 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -1053,6 +1053,8 @@ The multiplier bootstrap uses random weights w_i with E[w]=0 and Var(w)=1: - **Note:** Non-survey DR path also includes nuisance IF corrections (PS + OR), matching the survey path structure (Phase 7a). Previously used plug-in IF only. As of v3.7 the non-survey reg and ipw paths carry their corrections too (OR estimation-effect / PS score), so the nuisance-IF treatment is method-uniform. - **Note (post-fit aggregate() - rows M-020/M-117):** `fit(aggregate=)` is deprecated (3.9; removed 4.0) in favor of post-fit `results.aggregate(type, balance_e=)` on the fit-retained aggregation kit ('simple' relays the stored overall inference; 'event_study'/'group' recompute from the kit and fail closed on bootstrapped fits per the M-027 per-level rule). `DiagnosticReport` now derives the event-study container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends`, `pretrends_power`, and `sensitivity` checks run without the deprecated fit-time kwarg — CS containers are M-093-admitted into `compute_pretrends_power` / `HonestDiD.sensitivity_analysis` with pinned raw-route parity, and the raw field, when present (the requested-but-empty `{}` included), always takes precedence. `heterogeneity` is unaffected (it reads `group_time_effects` on plain fits). Bootstrapped fits surface the fail-closed `NotImplementedError` as an explicit per-check skip reason. +- **Note (post-fit `aggregate('total')` - the estimator-owned total incremental outcome, 3.10):** (a) **Estimand:** `total = C x overall_att` with `se = C x overall_se`, CI scaled by `C`, `t`/`p`/`df` inherited unchanged - an exact relay CONDITIONAL ON THE REALIZED AGGREGATION MASS (never call it unqualified "exact"): `C` is the design-fixed complete-case treated-observation count on every admitted routing, the inherited SE prices the overall's full inference (WIF/estimated-share terms included) but treats `C` as fixed at its realized value, and the random-mass `att*dC` variance term is deliberately out of scope (the DEFERRED survey/RC remainder). Every SE branch is homogeneous of degree 1 in the scaling, so the relay is exact under analytical AND percentile-bootstrap inference. (b) **Fail-closed routing gates**, in order: (1) repeated-cross-section-routed fits (`panel=False`, or genuinely-unbalanced `allow_unbalanced_panel=True` fits - a balanced panel with the flag stays panel-routed and admitted) raise `NotImplementedError` - the RC cohort mass counts UNITS and weights every kept post period by it, a ~T-fold overcount of treated observations on a T-wave RCS (execution-verified); (2) fits declaring a `survey_design=` raise - the realized-mass relay omits the survey mass-uncertainty term and design-aware population-scale totals are not implemented (retained weight scale differs by design family: analytic pweight resolves normalized to sum = n and CS retains no raw record - CS accepts pweight only - while replicate designs retain raw scale); the gate reads the fit-time `is_survey_fit` KIT snapshot, so bare-`cluster=` fits are ADMITTED (their synthesized all-ones internal design is not a survey) and a post-fit edit of the public `survey_metadata` cannot bypass it; the DECLARATION BOUNDARY is deliberate - an explicit unweighted `SurveyDesign(psu=...)` fit is numerically identical to bare `cluster=` yet fails closed by declaration (declaring survey_design opts into the survey contract; conservative, never wrong numbers; the message points at the `cluster=` route); a combined survey+RC fit deterministically gets the RC message; (3) the coincidence check: on bare-`cluster=` fits `_aggregate_simple` weights kept cells by the synthesized cohort masses (full cohort size per kept cell), and when kept cells have INCOMPLETE treated support that realized mass diverges from the complete-case count - the same overcount class as (1) - so the fit raises rather than publishing an ambiguous mass; clean bare-`cluster=` fits (the normal case) are admitted with `C` equal to both counts. Panel-routed pre-upgrade (legacy) kits missing the fit-time snapshots raise the refit message - never a mutable-field or public-replay fallback. (c) **Filter-replay contract:** `C` replays `_aggregate_simple`'s cell selection verbatim over the immutable `agg_gt_cells` fit-time kit snapshot (EDiD snapshot discipline; the kit builder stashes `(g, t, effect, n_treated)` 4-tuples in dict order) - anticipation filter FIRST (cells with `g - anticipation <= t < g` COUNT as post, the business-facing subtlety; universal-base reference cells are excluded by this filter alone, their base period being `< g - anticipation` by construction), finite-effect mask SECOND, weight = cohort mass where `fixed_cohort_agg_weights` provides one else per-cell complete-case `n_treated` (the source's `agg_weight` fallback is deliberately absent - RC-only, gated out), pairwise `np.sum`; the overall scalars come from the same stored fields the `'simple'` relay reads, so a post-fit mutation of `group_time_effects` moves NEITHER factor and `total == C x aggregate('simple').att` cannot desynchronize (pinned by a mutation test). NEVER use `agg_total_weight` (the RC all-units WIF mass). (d) **NaN rules:** empty post set / all-NaN effects give `C = NaN` and an all-NaN row WITHOUT re-emitting the fit-time UserWarnings (the no-post-fit-re-warn convention; the NaN row is the signal); with finite `C` the relay is VERBATIM - inherited NaNs (a degenerate fit's `(nan, nan)` CI beside finite att/se = 0) pass through, mirroring `aggregate('simple')` per the repo's non-estimable-row convention (att/n are never blanked by NaN inference fields); the ONLY additional blanking is true float overflow (a finite overall value made non-finite BY the `x C` scaling - unreachable in practice, guarded per this code family's finiteness convention). (e) **Bootstrap:** 'total' is a RELAY level - available on bootstrapped fits with the df column NaN (the M-027 per-level policy). (f) **Container conventions:** single row, `level="total"`, `label=["total"]`, `target=["total"]` (a total incremental outcome, NOT an ATT - the single-non-att neutral "estimate" rendering applies), `n = [C]` with `n_kind="obs"`, `weight=[1.0]`, per-class df carrier (CS: `resolve_inference_df` - finite on admitted bare-`cluster=` fits, NaN on plain analytical panel fits). Per-level `n` semantics are deliberately DIFFERENT: 'total''s `n` is the treated observations entering the support, while 'simple''s `n` stays treated+control units by contract - `n_kind` is per-container and each is documented, satisfying the never-conflate rule. (g) **MMM admission:** both `diff_diff.mmm` exporters accept the total container ALONE - `scale` (numeric or `"auto"`) is rejected as double-counting; for overall-total exports this route supersedes `scale="auto"` (see the MMM section). The total remains meaningful only for outcomes additive in levels - the same unverifiable caveat every scale route carries. + **Reference implementation(s):** - R: `did::att_gt()` (Callaway & Sant'Anna's official package) - Stata: `csdid` @@ -1060,7 +1062,7 @@ The multiplier bootstrap uses random weights w_i with E[w]=0 and Var(w)=1: **Requirements checklist:** - [ ] Requires never-treated units when `control_group="never_treated"` (default); not required for `"not_yet_treated"` - [ ] Bootstrap weights support Rademacher, Mammen, Webb distributions -- [ ] Aggregations: simple, event_study, group all implemented +- [ ] Aggregations: simple, event_study, group, total all implemented - [ ] Doubly robust estimation when covariates provided - [ ] Multiplier bootstrap preserves panel structure - [x] Repeated cross-sections (`panel=False`) for non-panel surveys (Phase 7b) @@ -1632,7 +1634,7 @@ where `q_{g,e} = pi_g / sum_{g' in G_{trt,e}} pi_{g'}`. - **Note:** `vcov_type` is permanently narrow to `{"hc1"}` per the Chen-Sant'Anna-Xie (2025) EIF-based variance achieving the semiparametric efficiency bound. Analytical-sandwich families `{classical, hc2, hc2_bm}` are rejected at `__init__` — the per-unit EIF aggregation has no equivalent single design matrix on which hat-matrix leverage or Bell-McCaffrey Satterthwaite DOF can be defined. `cluster=` invokes Liang-Zeger CR1 on cluster-aggregated EIF (`_compute_se_from_eif` with `cluster_indices`); `survey_design=` invokes TSL on the combined IF (`_compute_survey_eif_se`); both live in `diff_diff/efficient_did_aggregation.py` since the M-023 post-fit aggregate() extraction. `vcov_type='conley'` deferred to the EfficientDiD Conley follow-up row in DEFERRED.md. - **Note:** Default `cluster=None` (no survey design) renders summary label "HC1 heteroskedasticity-robust" because the per-unit EIF SE `sqrt(mean(EIF²)/n)` is methodologically HC1-style (no Liang-Zeger G/(G-1) finite-sample correction). `EfficientDiDResults.cluster_name` and `n_clusters` stay None under unclustered fits. This diverges from `ImputationDiD` which auto-clusters at unit per Borusyak-Jaravel-Spiess (2024) Theorem 3 — there the default summary renders the CR1 unit-clustered label. - **Note:** `set_params(vcov_type=bad)` raises immediately on EVERY estimator: since the shared `BaseEstimator` mixin (`diff_diff/_base.py`, v4 2(c)-i), `set_params` validates transactionally by constructor probe re-init, so it enforces exactly `__init__`'s validation, eagerly, library-wide. The former split — EfficientDiD eager vs `ImputationDiD`/`TripleDifference`/`CallawaySantAnna` (and six more: SunAbraham, StackedDiD, StaggeredTripleDifference, SpilloverDiD, TROP, PreTrendsPower) accepting constructor-rejected values until `fit()` — is retired; the fit-time re-validation layers remain as a second check against DIRECT attribute mutation (`est.vcov_type = ...`), which no setter can see. -- **Note (post-fit aggregate() - rows M-023/M-120):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work) in favor of post-fit `EfficientDiDResults.aggregate(type, balance_e=)` - a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a StackedDiD/dCDH view relay): `fit()` computes nothing extra, the results object retains an `AggregationKit`, and `aggregate('event_study'/'group', balance_e=)` re-runs the extracted `_EfficientAggregationMixin` aggregators on a throwaway host while `aggregate('simple')` relays the stored overall row bit-exact. (a) RETAINED BUFFERS (memory contract; phrased as maxima - optional design fields stay None when unsupplied): the per-(g,t) EIF dict, O(n_units x n_gt), the dominant payload - retained on EVERY fit regardless of `store_eif`, which since 3.9 governs only the public `influence_functions` field; `unit_cohorts` (cohort labels), `unit_level_weights`, factorized cluster codes - O(n_units) each; on ordinary (TSL) survey fits the unit-level `ResolvedSurveyDesign` adds `weights` plus, where supplied, `strata`/`psu`/`fpc` (factorized int codes / float values, never raw labels) - up to four O(n_units) arrays; on replicate designs it adds the O(n_units x n_replicates) replicate matrix plus, where supplied, `replicate_strata`/`replicate_rscales` (O(n_replicates)); per-row dict SNAPSHOTS of `group_time_effects` plus copies of the `groups`/`time_periods` lists and the scalar `pt_assumption`/`n_treated+n_control` provenance (aggregate() recomputes exclusively from these private snapshots, never from the mutable public result fields - a user edit of the public rows cannot mix altered point estimates with the retained EIF variance); scalars `n_units`, `cohort_fractions`, and the POST-OVERALL `df_survey` snapshot (captured after the overall inference and before the ES/group gates: the group pass can degenerate the working df to None on replicate designs with `n_valid <= 1`, and every fit-time aggregation seeds from the post-overall value, so recompute replays the exact seed). The data-minimization guarantee is scoped to unit identifiers - no unit-label container is retained. (b) `balance_e` uses the ANCHOR-HORIZON rule (keep cohorts with a finite effect at `e == balance_e`, then retain all their horizons) - the SAME rule CallawaySantAnna uses, divergent only from ImputationDiD/TwoStageDiD's balanced-window rule; an anchor no cohort reaches warns and yields a legal zero-row container. (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column, while the RECOMPUTE levels (ES/group) fail closed - per-horizon draws are not retained (exact-replay wiring is a TODO row). The prior uniform-conservatism BY-DECISION rule was superseded 2026-08-05 with the M-027 per-level convergence; its rationale - no level publishes analytical-provenance fields beside percentile inference - is honored by the relay's NaN df column. The fit-time bootstrap override still clears the group rows' `df_used` key. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is REJECTED BY DESIGN (both terminal TypeErrors state it): the primary ground is the absent joint event-study covariance (container `vcov=None`, all-NaN per-row df - the scalar `df_survey` channel is the container's only df provenance; the per-row hole is the tracked M-092-completion TODO row). Reference semantics are regime-dependent: under `pt_assumption="all"` there is NO reference row (universal first-period baseline; e=-1 is a genuine estimate); under `"post"` the per-cohort baseline cell is materialized as a mechanical zero anchor at `e = -1 - anticipation` whenever it is not the panel's first period, and the MEMBERSHIP-GATED `reference_period` property (the SunAbraham rule - never synthesized when the anchor cell was not estimated) marks it `is_reference` in the container and corrects `plot_event_study`'s inferred reference (previously the `-1` fallback) on PT-Post `anticipation>0` fits. (e) 'simple' relay conventions: `target="att"`, `n = n_treated_units + n_control_units` with `n_kind="units"` (DISJOINT by construction - `last_cohort` trimming reassigns before the counts, so a true total exists, unlike StackedDiD's overlapping sets), `df` = the post-overall snapshot (provenance-exact where `survey_metadata.df_survey` can diverge in the degenerate replicate state); 'group' relay: `n_kind="cells"`, `weight=None` (equal within-cohort weights, no cross-cohort mass), per-row `df_used` array captured at each row's `safe_inference` call (exact by construction; a stated divergence from CS's conservative-min scalar broadcast); 'event_study' rides the shared `_from_relative_dict` builder via a carrier whose `survey_metadata` copy carries the snapshot `df_survey`. +- **Note (post-fit aggregate() - rows M-023/M-120):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work) in favor of post-fit `EfficientDiDResults.aggregate(type, balance_e=)` - a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a StackedDiD/dCDH view relay): `fit()` computes nothing extra, the results object retains an `AggregationKit`, and `aggregate('event_study'/'group', balance_e=)` re-runs the extracted `_EfficientAggregationMixin` aggregators on a throwaway host while `aggregate('simple')` relays the stored overall row bit-exact. (a) RETAINED BUFFERS (memory contract; phrased as maxima - optional design fields stay None when unsupplied): the per-(g,t) EIF dict, O(n_units x n_gt), the dominant payload - retained on EVERY fit regardless of `store_eif`, which since 3.9 governs only the public `influence_functions` field; `unit_cohorts` (cohort labels), `unit_level_weights`, factorized cluster codes - O(n_units) each; on ordinary (TSL) survey fits the unit-level `ResolvedSurveyDesign` adds `weights` plus, where supplied, `strata`/`psu`/`fpc` (factorized int codes / float values, never raw labels) - up to four O(n_units) arrays; on replicate designs it adds the O(n_units x n_replicates) replicate matrix plus, where supplied, `replicate_strata`/`replicate_rscales` (O(n_replicates)); per-row dict SNAPSHOTS of `group_time_effects` plus copies of the `groups`/`time_periods` lists and the scalar `pt_assumption`/`n_treated+n_control` provenance (aggregate() recomputes exclusively from these private snapshots, never from the mutable public result fields - a user edit of the public rows cannot mix altered point estimates with the retained EIF variance); scalars `n_units`, `cohort_fractions`, and the POST-OVERALL `df_survey` snapshot (captured after the overall inference and before the ES/group gates: the group pass can degenerate the working df to None on replicate designs with `n_valid <= 1`, and every fit-time aggregation seeds from the post-overall value, so recompute replays the exact seed). The data-minimization guarantee is scoped to unit identifiers - no unit-label container is retained. (b) `balance_e` uses the ANCHOR-HORIZON rule (keep cohorts with a finite effect at `e == balance_e`, then retain all their horizons) - the SAME rule CallawaySantAnna uses, divergent only from ImputationDiD/TwoStageDiD's balanced-window rule; an anchor no cohort reaches warns and yields a legal zero-row container. (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column, while the RECOMPUTE levels (ES/group) fail closed - per-horizon draws are not retained (exact-replay wiring is a TODO row). The prior uniform-conservatism BY-DECISION rule was superseded 2026-08-05 with the M-027 per-level convergence; its rationale - no level publishes analytical-provenance fields beside percentile inference - is honored by the relay's NaN df column. The fit-time bootstrap override still clears the group rows' `df_used` key. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is REJECTED BY DESIGN (both terminal TypeErrors state it): the primary ground is the absent joint event-study covariance (container `vcov=None`, all-NaN per-row df - the scalar `df_survey` channel is the container's only df provenance; the per-row hole is the tracked M-092-completion TODO row). Reference semantics are regime-dependent: under `pt_assumption="all"` there is NO reference row (universal first-period baseline; e=-1 is a genuine estimate); under `"post"` the per-cohort baseline cell is materialized as a mechanical zero anchor at `e = -1 - anticipation` whenever it is not the panel's first period, and the MEMBERSHIP-GATED `reference_period` property (the SunAbraham rule - never synthesized when the anchor cell was not estimated) marks it `is_reference` in the container and corrects `plot_event_study`'s inferred reference (previously the `-1` fallback) on PT-Post `anticipation>0` fits. (e) 'simple' relay conventions: `target="att"`, `n = n_treated_units + n_control_units` with `n_kind="units"` (DISJOINT by construction - `last_cohort` trimming reassigns before the counts, so a true total exists, unlike StackedDiD's overlapping sets), `df` = the post-overall snapshot (provenance-exact where `survey_metadata.df_survey` can diverge in the degenerate replicate state); 'group' relay: `n_kind="cells"`, `weight=None` (equal within-cohort weights, no cross-cohort mass), per-row `df_used` array captured at each row's `safe_inference` call (exact by construction; a stated divergence from CS's conservative-min scalar broadcast); 'event_study' rides the shared `_from_relative_dict` builder via a carrier whose `survey_metadata` copy carries the snapshot `df_survey`. (f) `aggregate('total')` (3.10): the estimator-owned total incremental outcome - an exact relay `C x overall` CONDITIONAL on the realized aggregation mass, with `C = sum(n_treated)` over the kept post-anticipation FINITE-effect cells of the kit's deep-copied `group_time_effects` snapshot (an exact integer sum; NEVER the routinely-non-integral `n_units x sum(cohort_fractions)` float product). Unweighted fits only: ANY declared `survey_design=` fails closed via the kit's `unit_level_weights`/`resolved_survey_unit` markers (weight-type-agnostic - unweighted psu-only and analytic fweight designs, whose resolved weights stay RAW, gate identically; EDiD never synthesizes an internal design, so the markers are exact declaration provenance and post-fit mutation of the public `survey_metadata` cannot bypass them). No keepers -> NaN mass -> all-NaN row (no re-warn); with finite `C` the relay is verbatim (inherited NaNs pass through; att/n never blanked). Bootstrap fits relay with a NaN df column (M-027); df carrier = the post-overall `df_survey` snapshot; container: single `target='total'` row, `n_kind='obs'`, the per-level n-semantics note in the CS total Note applies. MMM: the container is admitted with NO scale (see the MMM section); the survey/RC totals remainder and its att*dC variance term are the DEFERRED row. --- @@ -1977,7 +1979,7 @@ where `W_it(h) = 1[K_it = h]` are lead indicators, estimated on `Omega_0` only. - [x] Supports balanced and unbalanced panels (iterative Gauss-Seidel demeaning for exact FE) - [x] Event study and group aggregation -- **Note (post-fit aggregate() - rows M-021/M-118):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work; the `imputation_did` wrapper forwards the shared sentinel so plain wrapper calls never fire the aggregate warning; since 3.9 the wrapper itself warns per M-070) in favor of post-fit `ImputationDiDResults.aggregate(type, balance_e=)` - a PANEL-BACKED lazy recompute kit (not an EIF-payload kit: ES/group aggregation is a target-specific Theorem-3 recompute - each `balance_e` re-masks which treated observations enter every horizon and re-solves the untreated projection - so no compact influence payload can replace the frame). (a) RETAINED BUFFERS (memory contract): the kit's bookkeeping holds REFERENCES to the SAME per-fit objects `_fit_data` already retains for `pretrend_test()` - the working panel copy (all user columns plus `_tau_hat`/`_rel_time`/`_never_treated`), the Omega masks, `unit_fe`/`time_fe`/`grand_mean`/`delta_hat`/`kept_cov_mask`, the resolved survey design, and `survey_weights` - ZERO marginal memory, and pickles are unchanged via memoization (`_estimator_ref` already ships these objects); plus value SNAPSHOTS for isolation (a `treatment_groups` copy, config scalars, a `dataclasses.replace` copy of `survey_metadata`, `overall_att`, `n_treated_obs`) and TWO df-provenance scalars (`survey_df_seed`, what the analytical aggregators received; `survey_df_final`, what the stored overall inference received). Each `aggregate()` call runs on a fresh throwaway host with a call-local projection cache (the fit-local factorizations are unpicklable and never retained). (b) `balance_e` uses the BALANCED-WINDOW rule: a cohort is retained iff its observed relative-time set - checked against the FULL panel via `_build_cohort_rel_times()` - covers the contiguous window `[-balance_e, max_h]`; the SAME rule TwoStageDiD uses, divergent from CS/EfficientDiD's anchor-horizon rule. A window no cohort satisfies warns and yields the reference-marker-only dict (a legal near-empty container). (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall quintet verbatim (finite safe_inference t included) with a NaN df column, while the RECOMPUTE levels fail closed (the per-target psi machinery makes exact replay tractable - a TODO row); the prior uniform fail-closed rule was superseded 2026-08-05 with the M-027 per-level convergence. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is REJECTED BY DESIGN (both terminal TypeErrors state it): the surface carries no joint event-study covariance - per-horizon conservative SEs only (container `vcov=None`; the scalar `df_survey` channel is its only df provenance, the per-row hole being the tracked M-092-completion TODO row). (e) RELAY CONVENTIONS: 'simple' relays the stored overall quintet bit-exact with `n = n_treated_obs`, `n_kind="obs"` (the treated/control UNIT sets overlap - a treated unit with pre-periods counts in both - so the CS/EDiD disjoint-units convention cannot apply; |Omega_1| is the population the ATT averages over, of which only finite-tau-hat observations enter the average - `n` reports the raw count, so on partially unidentified fits `n` exceeds the averaged support) and `df = survey_df_final`; 'group' rows carry per-row `df_used` captured at each row's `safe_inference` (the replicate override rewrites it, the bootstrap override clears it, the all-NaN cohort branch writes no key - consumers read via `.get`); 'event_study' rides the shared `_from_relative_dict` builder via a carrier whose metadata is a copy-on-use of the KIT's fit-final metadata copy. REPLICATE-WEIGHT fits replay the extracted `_replicate_override_aggregates` with a LEVEL-MATCHED stack: `compute_replicate_refit_variance` validates replicates JOINTLY (all-finite rows), so `aggregate(L)` reproduces `fit(aggregate=L)` exactly, a `fit(aggregate='all')` surface is NOT the equivalence target when a replicate NaNs on exactly one family's targets, and - the documented migration delta - moving a replicate fit from `fit(aggregate=)` to plain fit + post-fit `aggregate()` can change the public OVERALL row's se/CI/df on such degenerate designs (each surface self-consistent; pinned in the contract tests). `pretrends=True` + replicate: post-fit `aggregate('event_study')` raises the same NotImplementedError the fit-time gate raises (per-replicate lead refits unimplemented); 'group'/'simple' still work. Recompute re-emits the fit-time warnings (LSMR, Prop-5, empty-window) with fit-tuned stacklevels - post-fit attribution lands on a library frame, an accepted verbatim-move trade-off. `DiagnosticReport` now derives this container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends` (`pretrends=True` fits) and `heterogeneity` checks run without the deprecated fit-time kwarg; the re-emitted recompute warnings are captured and re-published on the consuming report section (record-and-republish), and derivation failures (bootstrap, missing kit, the replicate gate above) surface as explicit per-check skip reasons. +- **Note (post-fit aggregate() - rows M-021/M-118):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work; the `imputation_did` wrapper forwards the shared sentinel so plain wrapper calls never fire the aggregate warning; since 3.9 the wrapper itself warns per M-070) in favor of post-fit `ImputationDiDResults.aggregate(type, balance_e=)` - a PANEL-BACKED lazy recompute kit (not an EIF-payload kit: ES/group aggregation is a target-specific Theorem-3 recompute - each `balance_e` re-masks which treated observations enter every horizon and re-solves the untreated projection - so no compact influence payload can replace the frame). (a) RETAINED BUFFERS (memory contract): the kit's bookkeeping holds REFERENCES to the SAME per-fit objects `_fit_data` already retains for `pretrend_test()` - the working panel copy (all user columns plus `_tau_hat`/`_rel_time`/`_never_treated`), the Omega masks, `unit_fe`/`time_fe`/`grand_mean`/`delta_hat`/`kept_cov_mask`, the resolved survey design, and `survey_weights` - ZERO marginal memory, and pickles are unchanged via memoization (`_estimator_ref` already ships these objects); plus value SNAPSHOTS for isolation (a `treatment_groups` copy, config scalars, a `dataclasses.replace` copy of `survey_metadata`, `overall_att`, `n_treated_obs`) and TWO df-provenance scalars (`survey_df_seed`, what the analytical aggregators received; `survey_df_final`, what the stored overall inference received). Each `aggregate()` call runs on a fresh throwaway host with a call-local projection cache (the fit-local factorizations are unpicklable and never retained). (b) `balance_e` uses the BALANCED-WINDOW rule: a cohort is retained iff its observed relative-time set - checked against the FULL panel via `_build_cohort_rel_times()` - covers the contiguous window `[-balance_e, max_h]`; the SAME rule TwoStageDiD uses, divergent from CS/EfficientDiD's anchor-horizon rule. A window no cohort satisfies warns and yields the reference-marker-only dict (a legal near-empty container). (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall quintet verbatim (finite safe_inference t included) with a NaN df column, while the RECOMPUTE levels fail closed (the per-target psi machinery makes exact replay tractable - a TODO row); the prior uniform fail-closed rule was superseded 2026-08-05 with the M-027 per-level convergence. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is REJECTED BY DESIGN (both terminal TypeErrors state it): the surface carries no joint event-study covariance - per-horizon conservative SEs only (container `vcov=None`; the scalar `df_survey` channel is its only df provenance, the per-row hole being the tracked M-092-completion TODO row). (e) RELAY CONVENTIONS: 'simple' relays the stored overall quintet bit-exact with `n = n_treated_obs`, `n_kind="obs"` (the treated/control UNIT sets overlap - a treated unit with pre-periods counts in both - so the CS/EDiD disjoint-units convention cannot apply; |Omega_1| is the population the ATT averages over, of which only finite-tau-hat observations enter the average - `n` reports the raw count, so on partially unidentified fits `n` exceeds the averaged support) and `df = survey_df_final`; 'group' rows carry per-row `df_used` captured at each row's `safe_inference` (the replicate override rewrites it, the bootstrap override clears it, the all-NaN cohort branch writes no key - consumers read via `.get`); 'event_study' rides the shared `_from_relative_dict` builder via a carrier whose metadata is a copy-on-use of the KIT's fit-final metadata copy. REPLICATE-WEIGHT fits replay the extracted `_replicate_override_aggregates` with a LEVEL-MATCHED stack: `compute_replicate_refit_variance` validates replicates JOINTLY (all-finite rows), so `aggregate(L)` reproduces `fit(aggregate=L)` exactly, a `fit(aggregate='all')` surface is NOT the equivalence target when a replicate NaNs on exactly one family's targets, and - the documented migration delta - moving a replicate fit from `fit(aggregate=)` to plain fit + post-fit `aggregate()` can change the public OVERALL row's se/CI/df on such degenerate designs (each surface self-consistent; pinned in the contract tests). `pretrends=True` + replicate: post-fit `aggregate('event_study')` raises the same NotImplementedError the fit-time gate raises (per-replicate lead refits unimplemented); 'group'/'simple' still work. Recompute re-emits the fit-time warnings (LSMR, Prop-5, empty-window) with fit-tuned stacklevels - post-fit attribution lands on a library frame, an accepted verbatim-move trade-off. `DiagnosticReport` now derives this container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends` (`pretrends=True` fits) and `heterogeneity` checks run without the deprecated fit-time kwarg; the re-emitted recompute warnings are captured and re-published on the consuming report section (record-and-republish), and derivation failures (bootstrap, missing kit, the replicate gate above) surface as explicit per-check skip reasons. (f) `aggregate('total')` (3.10): the estimator-owned total incremental outcome - an exact relay `C x overall` CONDITIONAL on the realized aggregation mass, with `C` the FINITE-tau complete-case support (`total_support`, a float SNAPSHOT stashed at kit build precisely because the kit's `df` is a live `_fit_data` reference - a post-fit frame edit must not move the total; pre-upgrade kits missing the stash raise the refit message). Since the overall IS the finite-support mean, `C x overall = sum(tau)` exactly - this total uses the FINITE support and thereby fixes the documented raw-|Omega_1| overcount of the MMM `scale=\"auto\"` route for total exports ('simple''s `n` stays the raw count by contract; different levels, different documented n semantics). Zero support (the all-unidentified fit, overall already NaN) -> NaN mass -> all-NaN row (no re-warn); finite `C` relays verbatim - the constant-outcome degenerate fit keeps finite att/se = 0 beside its inherited `(nan, nan)` CI, mirroring 'simple'. Unweighted fits only: declared `survey_design=` fails closed via the kit's `survey_metadata` snapshot (mutation-proof). Bootstrap fits relay with a NaN df column (M-027); df carrier = `survey_df_final`; container: single `target='total'` row, `n_kind='obs'`. MMM: admitted with NO scale (see the MMM section). --- @@ -2066,7 +2068,7 @@ Our implementation uses multiplier bootstrap on the GMM influence function: clus - [x] Multiplier bootstrap on GMM influence function - [x] Event study and overall ATT aggregation -- **Note (post-fit aggregate() - rows M-022/M-119):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work; the `two_stage_did` wrapper forwards the shared sentinel so plain wrapper calls never fire the aggregate warning; since 3.9 the wrapper itself warns per M-071) in favor of post-fit `TwoStageDiDResults.aggregate(type, balance_e=)` - a PANEL-BACKED lazy recompute kit: each level is a fresh Stage-2 OLS + joint Gardner-GMM sandwich on a level-specific design, so no compact influence payload exists. (a) RETAINED BUFFERS (memory contract - the FIRST panel retention on TwoStageDiD results, a deliberate break from the CS/EDiD identifier-minimization guarantee, with a `store_kit` opt-out tracked in DEFERRED.md): a COLUMN-SUBSET COPY of the working frame - `unit`/`time`/`outcome`/`first_treat` + covariates + the cluster column (deduplicated: `cluster=` may legally name a core column) + `_never_treated`/`_rel_time`/`_y_tilde` - O(n_obs) on every results object and pickle; the Stage-1 FE model (`unit_fe`/`time_fe`/`grand_mean`/`delta_hat`/`kept_cov_mask`), the Omega masks, the full-domain `keep_mask`, the Wave-E.3-GATED `score_pad_mask`/`cluster_ids_full` values fit actually passed (None unless the always-treated pad was active), `survey_weights`, and the resolved survey design - on replicate designs that adds the O(n_obs x R) replicate matrix; plus value snapshots (`treatment_groups` copy, `ref_period`, `overall_att`, `n_treated_obs`, a `dataclasses.replace` copy of `survey_metadata`) and TWO df scalars (`survey_df_stage2`, the recompute seed; `survey_df_final`, what the stored overall inference received). (b) `balance_e` uses the BALANCED-WINDOW rule (`[-balance_e, max_h]` coverage against the full panel - the ImputationDiD rule, divergent from CS/EfficientDiD's anchor-horizon rule); zero qualifying cohorts warns and yields the reference-row-only dict with `vcov=None`. (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall quintet verbatim (finite safe_inference t included) with a NaN df column, while the RECOMPUTE levels fail closed (per-level GMM scores are function-locals; replay is a TODO row) - the prior uniform rule superseded 2026-08-05 with the M-027 per-level convergence; a fit whose bootstrap FAILED (`bootstrap_results=None`, analytical inference retained) aggregates normally. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is DEFERRED, not by-design (both terminal TypeErrors state it): analytical surfaces DO carry the real joint Gardner-GMM covariance (M-092), but the pre-period coefficients are stage-1 residual MEANS - the reference horizon is dropped from the no-intercept Stage-2 design and the zero anchor row is appended mechanically - not contrasts against the advertised reference, while HonestDiD's Delta^RM/Delta^SD arithmetic hard-codes the `delta_0 = 0` normalization into its boundary/bridge constraints; admission awaits a normalization derivation (either re-estimating Stage 2 with the reference horizon in the design or deriving the residual-to-reference mapping) - the DEFERRED.md paper-gated row. (e) RELAY CONVENTIONS: 'simple' relays the stored overall quintet bit-exact with `n = n_treated_obs`, `n_kind="obs"` (overlapping unit sets - the StackedDiD carve-out class; the pre-filter |Omega_1| count, while the ATT's Stage-2 support excludes rows whose `y_tilde` is non-finite - on such degenerate fits `n` exceeds the averaged support) and `df = survey_df_final` (on replicate fits that value came from the `[overall]`-only joint stack - snapshotted, never re-derived); 'group' relays a SCALAR df broadcast (deliberate divergence from ImputationDiD's per-row `df_used`: `_stage2_group` passes one immutable `survey_df` to every row's `safe_inference`, so the scalar is provenance-exact by construction and the moved method stays verbatim); 'event_study' reproduces the M-092 container contract exactly - analytical fits thread the recomputed joint vcov + `vcov_index` + the finite-and->0 df scalar through the carrier, replicate fits thread `vcov=None`/`index=None` with the REPLAYED level-matched df, and the carrier's metadata is a copy-on-use of the KIT's fit-final metadata copy. REPLICATE-WEIGHT fits replay the extracted `_replay_replicate_inference` with a LEVEL-MATCHED stack (the ImputationDiD semantics: `aggregate(L)` reproduces `fit(aggregate=L)`; `fit(aggregate='all')` is not the equivalence target on degenerate designs; the OVERALL-row migration delta on such designs is documented and pinned). Recompute re-emits fit-time warnings with fit-tuned stacklevels - an accepted verbatim-move trade-off. `DiagnosticReport` now derives this container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends` (`pretrends=True` fits; the recomputed joint vcov drives the joint-Wald path) and `heterogeneity` checks run without the deprecated fit-time kwarg; re-emitted recompute warnings are captured and re-published on the consuming report section, and derivation failures surface as explicit per-check skip reasons. +- **Note (post-fit aggregate() - rows M-022/M-119):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work; the `two_stage_did` wrapper forwards the shared sentinel so plain wrapper calls never fire the aggregate warning; since 3.9 the wrapper itself warns per M-071) in favor of post-fit `TwoStageDiDResults.aggregate(type, balance_e=)` - a PANEL-BACKED lazy recompute kit: each level is a fresh Stage-2 OLS + joint Gardner-GMM sandwich on a level-specific design, so no compact influence payload exists. (a) RETAINED BUFFERS (memory contract - the FIRST panel retention on TwoStageDiD results, a deliberate break from the CS/EDiD identifier-minimization guarantee, with a `store_kit` opt-out tracked in DEFERRED.md): a COLUMN-SUBSET COPY of the working frame - `unit`/`time`/`outcome`/`first_treat` + covariates + the cluster column (deduplicated: `cluster=` may legally name a core column) + `_never_treated`/`_rel_time`/`_y_tilde` - O(n_obs) on every results object and pickle; the Stage-1 FE model (`unit_fe`/`time_fe`/`grand_mean`/`delta_hat`/`kept_cov_mask`), the Omega masks, the full-domain `keep_mask`, the Wave-E.3-GATED `score_pad_mask`/`cluster_ids_full` values fit actually passed (None unless the always-treated pad was active), `survey_weights`, and the resolved survey design - on replicate designs that adds the O(n_obs x R) replicate matrix; plus value snapshots (`treatment_groups` copy, `ref_period`, `overall_att`, `n_treated_obs`, a `dataclasses.replace` copy of `survey_metadata`) and TWO df scalars (`survey_df_stage2`, the recompute seed; `survey_df_final`, what the stored overall inference received). (b) `balance_e` uses the BALANCED-WINDOW rule (`[-balance_e, max_h]` coverage against the full panel - the ImputationDiD rule, divergent from CS/EfficientDiD's anchor-horizon rule); zero qualifying cohorts warns and yields the reference-row-only dict with `vcov=None`. (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall quintet verbatim (finite safe_inference t included) with a NaN df column, while the RECOMPUTE levels fail closed (per-level GMM scores are function-locals; replay is a TODO row) - the prior uniform rule superseded 2026-08-05 with the M-027 per-level convergence; a fit whose bootstrap FAILED (`bootstrap_results=None`, analytical inference retained) aggregates normally. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is DEFERRED, not by-design (both terminal TypeErrors state it): analytical surfaces DO carry the real joint Gardner-GMM covariance (M-092), but the pre-period coefficients are stage-1 residual MEANS - the reference horizon is dropped from the no-intercept Stage-2 design and the zero anchor row is appended mechanically - not contrasts against the advertised reference, while HonestDiD's Delta^RM/Delta^SD arithmetic hard-codes the `delta_0 = 0` normalization into its boundary/bridge constraints; admission awaits a normalization derivation (either re-estimating Stage 2 with the reference horizon in the design or deriving the residual-to-reference mapping) - the DEFERRED.md paper-gated row. (e) RELAY CONVENTIONS: 'simple' relays the stored overall quintet bit-exact with `n = n_treated_obs`, `n_kind="obs"` (overlapping unit sets - the StackedDiD carve-out class; the pre-filter |Omega_1| count, while the ATT's Stage-2 support excludes rows whose `y_tilde` is non-finite - on such degenerate fits `n` exceeds the averaged support) and `df = survey_df_final` (on replicate fits that value came from the `[overall]`-only joint stack - snapshotted, never re-derived); 'group' relays a SCALAR df broadcast (deliberate divergence from ImputationDiD's per-row `df_used`: `_stage2_group` passes one immutable `survey_df` to every row's `safe_inference`, so the scalar is provenance-exact by construction and the moved method stays verbatim); 'event_study' reproduces the M-092 container contract exactly - analytical fits thread the recomputed joint vcov + `vcov_index` + the finite-and->0 df scalar through the carrier, replicate fits thread `vcov=None`/`index=None` with the REPLAYED level-matched df, and the carrier's metadata is a copy-on-use of the KIT's fit-final metadata copy. REPLICATE-WEIGHT fits replay the extracted `_replay_replicate_inference` with a LEVEL-MATCHED stack (the ImputationDiD semantics: `aggregate(L)` reproduces `fit(aggregate=L)`; `fit(aggregate='all')` is not the equivalence target on degenerate designs; the OVERALL-row migration delta on such designs is documented and pinned). Recompute re-emits fit-time warnings with fit-tuned stacklevels - an accepted verbatim-move trade-off. `DiagnosticReport` now derives this container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends` (`pretrends=True` fits; the recomputed joint vcov drives the joint-Wald path) and `heterogeneity` checks run without the deprecated fit-time kwarg; re-emitted recompute warnings are captured and re-published on the consuming report section, and derivation failures surface as explicit per-check skip reasons. (f) `aggregate('total')` (3.10): the estimator-owned total incremental outcome - an exact relay `C x overall` CONDITIONAL on the realized aggregation mass, with `C` the POST-FILTER treatment-indicator support the second stage actually regressed on (`omega_1_mask` minus non-finite `_y_tilde` rows, computed inline from the kit's private column-subset frame COPY - snapshot-grade, no fit-time stash needed, and the fit-time masker is never re-called so its warnings are not re-emitted; `score_pad_mask` is irrelevant here - it pads scores/variance only). This is the post-filter count 'simple''s raw pre-filter `n` documents as exceeding on degenerate fits. `D.sum() == 0` -> NaN mass -> all-NaN row (mirrors the fit's `(nan, nan)`); finite `C` relays verbatim. Unweighted fits only: declared `survey_design=` fails closed via the kit's `survey_metadata` snapshot (mutation-proof). Bootstrap fits relay with a NaN df column (M-027); df carrier = `survey_df_final`; container: single `target='total'` row, `n_kind='obs'`. MMM: admitted with NO scale (see the MMM section). --- @@ -6329,10 +6331,11 @@ estimator-focused: **Key implementation requirements:** *Container mode (`aggregation_result=` + `scale=`, both exporters):* -- **Note:** Scale contract and row order. Derivation is `effect_i = att_i * scale_i`, `se_out_i = se_i * scale_i` per container row, with rows consumed in `to_dataframe()` order (sorted by label when sortable, producer order otherwise - the order `summary()` prints); every per-row sequence kwarg (`scale`, `spend`, `x`, `delta_x`, `dims`) aligns to that order, and the helper reads all per-row values from one `to_dataframe()` call. `scale` is always required with a container: a number (finite, positive; scalar or per-row) or the literal `"auto"`. Accepted levels are `simple` (one row; a multi-row simple container is out-of-contract and rejected) and `group` (one experiment per cohort); every row's `target` must be `"att"` (ContinuousDiD's `acrt`, HAD's WAS estimands, and dCDH's estimand relays reject the container WHOLE - no silent row filtering), and every row needs finite `att` and finite positive `se`. -- **Note:** `scale="auto"` allowlist and routing. Auto-derivation (`scale_i = n_i`) is honored ONLY for container `estimator` provenance in `{ImputationDiD, TwoStageDiD}`, the audited producers whose `n` is the treated observations (unit-periods) the ATT averages over at both supported levels. Routing keys on the `estimator` field, never on `n_kind` alone - CallawaySantAnna repeated-cross-section fits report `n_kind="obs"` with `n` = treated+control observations - and `n_kind == "obs"` serves only as a schema-drift sanity guard on the allowlist path. CallawaySantAnna (`simple`: treated+control units/obs; `group`: contributing (g, t) cells) and EfficientDiD (units/cells) are refused with hints naming their `n` semantics; StackedDiD is refused because its `n` is a deduplicated distinct-treated-UNIT count (no per-observation mass) and its `weighting="population"`/`"sample_share"` estimands make raw treated exposure the wrong multiplier. -- **Note:** `scale="auto"` is an explicit caller acknowledgement of three assumptions the container cannot verify: the outcome is in additive levels (no outcome-scale provenance exists on the container); the fit is unweighted (survey-weighted fits pair a weighted `att` with a raw-count `n` and carry no survey marker - pass a numeric `scale` there); and every treated observation's effect is identified. On the identification point, ImputationDiD averages finite tau-hat only while `n` stays the raw treated-obs count at BOTH levels, and TwoStageDiD's `simple` ATT support excludes rows whose first-stage residual is non-finite while `n` reports the pre-filter count (its `group` `n` IS post-filter) - both estimators warn at fit time on the degenerate branch, and `"auto"` on such a fit overcounts (behaviorally pinned in `tests/test_mmm.py::TestAggregationWorkflow`). +- **Note:** Scale contract and row order. Derivation is `effect_i = att_i * scale_i`, `se_out_i = se_i * scale_i` per container row, with rows consumed in `to_dataframe()` order (sorted by label when sortable, producer order otherwise - the order `summary()` prints); every per-row sequence kwarg (`scale`, `spend`, `x`, `delta_x`, `dims`) aligns to that order, and the helper reads all per-row values from one `to_dataframe()` call. `scale` is required with a `simple`/`group` container - a number (finite, positive; scalar or per-row) or the literal `"auto"` - and FORBIDDEN with a `total` container (see the total-admission Note below). Accepted levels are `simple` (one row; a multi-row simple container is out-of-contract and rejected), `group` (one experiment per cohort), and `total` (one already-scaled row); every `simple`/`group` row's `target` must be `"att"` (ContinuousDiD's `acrt`, HAD's WAS estimands, and dCDH's estimand relays reject the container WHOLE - no silent row filtering), and every row needs finite `att` and finite positive `se`. +- **Note:** `scale="auto"` allowlist and routing. Auto-derivation (`scale_i = n_i`) is honored ONLY for container `estimator` provenance in `{ImputationDiD, TwoStageDiD}`, the audited producers whose `n` is the treated observations (unit-periods) the ATT averages over at both supported levels. Routing keys on the `estimator` field, never on `n_kind` alone - CallawaySantAnna repeated-cross-section fits report `n_kind="obs"` with `n` = treated+control observations - and `n_kind == "obs"` serves only as a schema-drift sanity guard on the allowlist path. CallawaySantAnna (`simple`: treated+control units/obs; `group`: contributing (g, t) cells) and EfficientDiD (units/cells) are refused with hints naming their `n` semantics AND the successor route - where supported, `results.aggregate('total')` needs no scale at all; StackedDiD is refused because its `n` is a deduplicated distinct-treated-UNIT count (no per-observation mass) and its `weighting="population"`/`"sample_share"` estimands make raw treated exposure the wrong multiplier. +- **Note:** `scale="auto"` is an explicit caller acknowledgement of three assumptions the container cannot verify: the outcome is in additive levels (no outcome-scale provenance exists on the container); the fit is unweighted (survey-weighted fits pair a weighted `att` with a raw-count `n` and carry no survey marker - pass a numeric `scale` there); and every treated observation's effect is identified. On the identification point, ImputationDiD averages finite tau-hat only while `n` stays the raw treated-obs count at BOTH levels, and TwoStageDiD's `simple` ATT support excludes rows whose first-stage residual is non-finite while `n` reports the pre-filter count (its `group` `n` IS post-filter) - both estimators warn at fit time on the degenerate branch, and `"auto"` on such a fit overcounts (behaviorally pinned in `tests/test_mmm.py::TestAggregationWorkflow`). For OVERALL-TOTAL exports, `results.aggregate('total')` supersedes `scale="auto"` - its finite-masked support eliminates this overcount for that use; `group`-level containers remain a supported `"auto"` route (a 'total' emits ONE row and cannot substitute per-cohort exports, and ImputationDiD's `group` `n` keeps its documented raw-support caveat there). - **Note:** `group` containers carry correlated cohorts, both exporters: all cohort rows come from ONE fit sharing controls/windows, and the container stores only per-row SEs (joint-covariance off-diagonals are discarded upstream). Meridian pooling treats rows as independent (see the pooling Note below) and PyMC-Marketing scores each lift row independently with its marginal `sigma`; the omitted cross-cohort covariance can misstate the joint/pooled uncertainty - anti-conservatively when the net weighted covariance is positive (shared controls typically induce positive correlation) - and its direction cannot be determined from marginal SEs alone. `se_widening > 1` is a conservative heuristic, not an exact correction (docstring-noted on both exporters). +- **Note (`total` container admission, 3.10):** a `results.aggregate('total')` container - the estimator-owned total incremental outcome on CallawaySantAnna/EfficientDiD/ImputationDiD/TwoStageDiD (panel non-survey fits; see each estimator's post-fit total Note) - is admitted ALONE: its single `target="total"` row is already `C x overall`, so ANY `scale` (numeric or `"auto"`) is rejected as double-counting. Admission is PROVENANCE-GATED like `scale="auto"` (`_TOTAL_ESTIMATORS`, exactly the four adopters): "already scaled" is a producer claim, so a hand-built or altered container with unknown/missing/StackedDiD provenance is rejected rather than silently exported unscaled; the full producer contract is validated too (`label="total"`, `n_kind="obs"`, `weight=[1.0]` - schema drift fails closed), and the multi-row / wrong-target / non-finite guards reject the container whole. The additive-levels assumption is NOT dissolved by this route: the total is meaningful only for outcomes additive in levels (log/rate/share outcomes are not additive) - the same caveat the numeric and `"auto"` routes carry, stated in the exporters' docstrings; the container likewise carries no outcome-scale provenance, so it remains the caller's acknowledgement. On the routings totals do not support (repeated-cross-section, declared survey_design, divergent bare-cluster masses, pre-upgrade kits) `aggregate('total')` raises with the reason and the caller falls back to a numeric `scale` - a CALLER-DEFINED estimand, not the estimator-owned complete-case total (a single scalar cannot reweight an already-mass-averaged ATT). *Lift-test frame (`to_pymc_marketing_lift_test`):* - Assembles one row per experiment with the exact schema `[channel, *dims, x, delta_x, delta_y, sigma]` from caller-supplied values. `delta_y`/`sigma` are the scoped incremental outcome and its SE; all four numeric columns must describe the SAME calibration observation (same channel, population, period span, additive-level outcome) - stated in the docstring, not machine-checkable, since spend and scope are user input. diff --git a/docs/migration-4.0.md b/docs/migration-4.0.md index bd0094a1e..ae40acfdf 100644 --- a/docs/migration-4.0.md +++ b/docs/migration-4.0.md @@ -159,7 +159,8 @@ checks read `placebo_event_study` directly. `EfficientDiD` and `ContinuousDiD`, the post-fit recompute levels raise `NotImplementedError` when the fit used `n_bootstrap > 0`, while the fit-time keyword they replace is removed at 4.0. If you bootstrap *and* aggregate, keep the fit-time call for now and track the open -`TODO.md` draw-retention rows. `aggregate("simple")` does relay, and `StackedDiD`, +`TODO.md` draw-retention rows. `aggregate("simple")` — and, on its four adopters, +`aggregate("total")` (3.10) — does relay, and `StackedDiD`, `ChaisemartinDHaultfoeuille` and `HeterogeneousAdoptionDiD` are unaffected — their `aggregate()` is a pure view over stored fields. ``` diff --git a/docs/r_comparison.rst b/docs/r_comparison.rst index fc1eba783..cc582807a 100644 --- a/docs/r_comparison.rst +++ b/docs/r_comparison.rst @@ -212,6 +212,10 @@ staggered DiD. Here's how to translate common operations: agg_simple = results.aggregate('simple') agg_dynamic = results.aggregate('event_study') agg_group = results.aggregate('group') + # 'total' (3.10) has no aggte() counterpart - a library-first extension: + # the estimator-owned total incremental outcome (C x overall) on panel + # non-survey fits, consumed by diff_diff.mmm with no scale. + agg_total = results.aggregate('total') R ``HonestDiD`` Package → diff-diff ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/survey-roadmap.md b/docs/survey-roadmap.md index 056c4833a..a8d652006 100644 --- a/docs/survey-roadmap.md +++ b/docs/survey-roadmap.md @@ -280,6 +280,7 @@ the limitation and suggested alternative. | WooldridgeDiD | Unsupported-period filtering + `survey_design` | Restrict the frame to the supported periods explicitly and re-fit. Deleting rows in-place is naive subsetting: it removes their PSUs and strata from the TSL meat and from `df_survey = n_PSU - n_strata`. Exact only if every PSU and stratum survives the restriction (true on a balanced panel; NOT in general — an unbalanced frame can hold a PSU observed only at unsupported periods). Verify before relying on it. | | WooldridgeDiD | Unidentified-cohort exclusion + `survey_design` | Same reason (ledger `M-123`). Drop the cohort from the frame yourself, or supply a panel where every cohort has a pre-treatment period. | | All bootstrap estimators | Bootstrap + replicate weights | These are alternative variance methods; pick one | +| CS, EfficientDiD, ImputationDiD, TwoStageDiD | `aggregate('total')` on fits declaring a `survey_design` | The estimator-owned total (3.10) is panel non-survey only: the realized-mass relay omits the survey mass-uncertainty (att*dC) variance term and design-aware population-scale totals are not implemented (retained weight scale differs by design family). Pass a caller-derived numeric `scale=` to the MMM exporters instead, or use `cluster=` (without `survey_design`) for an unweighted clustered fit. Tracked in DEFERRED.md (Paper-gated). | **Warning/fallback (no error):** MultiPeriodDiD with `wild_bootstrap` + `survey_design` warns and falls back to analytical inference. diff --git a/docs/troubleshooting.rst b/docs/troubleshooting.rst index 4745622fe..8ad64bd41 100644 --- a/docs/troubleshooting.rst +++ b/docs/troubleshooting.rst @@ -212,8 +212,9 @@ Staggered Adoption Issues # Use bootstrap for better inference. On a BOOTSTRAPPED fit, post-fit # results.aggregate('event_study') raises - the percentile draws are not # retained - so the deprecated fit-time aggregate= remains the documented - # route for this case until 4.0. (results.aggregate('simple') relays the - # stored bootstrap inference and works on any fit.) + # route for this case until 4.0. (results.aggregate('simple') - and, + # where supported, results.aggregate('total') - relays the stored + # bootstrap inference.) cs = CallawaySantAnna(n_bootstrap=999) results = cs.fit(data, outcome='y', unit='unit_id', time='period', first_treat='first_treat', diff --git a/docs/tutorials/02_staggered_did.ipynb b/docs/tutorials/02_staggered_did.ipynb index ee410524d..051d558b0 100644 --- a/docs/tutorials/02_staggered_did.ipynb +++ b/docs/tutorials/02_staggered_did.ipynb @@ -1089,7 +1089,8 @@ "**Note (3.9):** on a *bootstrapped* fit, fit-time `aggregate=` remains the\n", "documented route until 4.0 - post-fit `results.aggregate('event_study')`\n", "raises there because the percentile draws are not retained\n", - "(`aggregate('simple')` relays the stored bootstrap inference). The\n", + "(`aggregate('simple')` and, where supported, `aggregate('total')` relay\n", + "the stored bootstrap inference). The\n", "`FutureWarning` in the next cell's output is expected." ] }, @@ -1129,7 +1130,7 @@ "name": "stderr", "output_type": "stream", "text": [ - ".py:9: FutureWarning: CallawaySantAnna.fit(aggregate=) is deprecated and will be removed in 4.0. Fit once, then aggregate as a post-fit step: results = CallawaySantAnna().fit(...); results.aggregate('event_study') / .aggregate('group') / .aggregate('simple'). balance_e moves onto aggregate() alongside it: results.aggregate('event_study', balance_e=2).\n", + ".py:9: FutureWarning: CallawaySantAnna.fit(aggregate=) is deprecated and will be removed in 4.0. Fit once, then aggregate as a post-fit step: results = CallawaySantAnna().fit(...); results.aggregate('event_study') / .aggregate('group') / .aggregate('simple') / .aggregate('total'). balance_e moves onto aggregate() alongside it: results.aggregate('event_study', balance_e=2).\n", " results_boot = cs_boot.fit(\n" ] } @@ -1151,8 +1152,9 @@ " first_treat=\"first_treat\", # Column with first treatment period\n", " # Fit-time aggregation is the documented exception for BOOTSTRAPPED fits\n", " # until 4.0: post-fit results.aggregate('event_study') raises here because\n", - " # the percentile draws are not retained (aggregate('simple') relays the\n", - " # stored bootstrap inference and works on any fit). Expect a FutureWarning.\n", + " # the percentile draws are not retained (aggregate('simple') and, where\n", + " # supported, aggregate('total') relay the stored bootstrap inference).\n", + " # Expect a FutureWarning.\n", " aggregate=\"event_study\"\n", ")\n", "\n", diff --git a/docs/tutorials/15_efficient_did.ipynb b/docs/tutorials/15_efficient_did.ipynb index 9733b150b..6f6010127 100644 --- a/docs/tutorials/15_efficient_did.ipynb +++ b/docs/tutorials/15_efficient_did.ipynb @@ -565,12 +565,12 @@ "2. Under **PT-All**, EDiD exploits overidentification for tighter SEs than CS\n", "3. Under **PT-Post**, EDiD matches CS for post-treatment ATT(g,t); pre-treatment diagnostics use a fixed baseline and may differ from CS's default varying baseline\n", "4. The efficiency gain comes from optimally weighting across all valid (comparison group, baseline) pairs\n", - "5. **Event study** and **group** aggregations work just like CS\n", + "5. **Event study**, **group**, and (3.10) **total** aggregations work just like CS - `results.aggregate('total')` is the estimator-owned total incremental outcome on panel non-survey fits\n", "6. **Multiplier bootstrap** provides robust inference with Rademacher, Mammen, or Webb weights\n", "7. **Condition numbers** flag potentially unstable weight matrices\n", "8. **Anticipation** shifts the effective treatment boundary for pre-treatment effects\n", "9. **Covariates are supported** via the doubly robust path (sieve outcome regressions + propensity ratios) --- pass `covariates=[...]` to `fit()`\n", - "10. **Aggregate post-fit** (3.9): `results.aggregate('event_study'/'group'/'simple', balance_e=)` recomputes from retained EIFs --- no refit; fit-time `aggregate=`/`balance_e=` are deprecated, and on bootstrapped fits the recompute levels keep the fit-time route (`aggregate('simple')` relays the stored bootstrap inference post-fit)\n", + "10. **Aggregate post-fit** (3.9): `results.aggregate('event_study'/'group', balance_e=)` recomputes from retained EIFs --- no refit; `aggregate('simple')` and (3.10) `aggregate('total')` are RELAYS of the stored inference (no `balance_e`; both stay available on bootstrapped fits, where the recompute levels keep the deprecated fit-time route)\n", "11. When in doubt, run both EDiD and CS --- if ATTs agree, report EDiD for tighter CIs\n", "\n", "**Parameter reference:**\n", diff --git a/docs/v4-deprecations.yaml b/docs/v4-deprecations.yaml index 0a8bbea0c..90044cda7 100644 --- a/docs/v4-deprecations.yaml +++ b/docs/v4-deprecations.yaml @@ -229,7 +229,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/staggered.py, diff_diff/staggered_results.py, diff_diff/aggregation.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt, diff_diff/diagnostic_report.py] - notes: "Shimmed in 3.9: fit(aggregate=) warns via a sentinel default (so a plain fit() never warns) and still returns the fully populated legacy surface; results.aggregate(type=) is the successor. balance_e moves alongside it as its own row [M-117] - it was previously tracked only as prose here, which nothing asserted. VOCABULARY: the closed set is library-wide (simple|event_study|group|calendar); CallawaySantAnna's SUPPORTED SUBSET is simple|event_study|group - it has no calendar aggregator (the DEFERRED 'Calendar-time aggregation' row), and aggregate('calendar') raises naming what is supported. BOOTSTRAP fits: 'simple' RELAYS the stored overall quintet verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column, while the recompute levels (event_study/group) fail closed pending draw retention (BootstrapReplaySpec is the TODO row) - the per-level policy converged with [M-027] (previously ALL levels failed closed; the relay never publishes an analytical df beside percentile inference). DiagnosticReport now derives the event-study surface via post-fit aggregate('event_study') when the raw field is absent, so its ES-gated checks run on plain fits (derivation failures surface as explicit skip reasons)." + notes: "Shimmed in 3.9: fit(aggregate=) warns via a sentinel default (so a plain fit() never warns) and still returns the fully populated legacy surface; results.aggregate(type=) is the successor. balance_e moves alongside it as its own row [M-117] - it was previously tracked only as prose here, which nothing asserted. VOCABULARY: the closed set is library-wide (simple|event_study|group|calendar|total - 'total' promoted 2026-08-16 as the estimator-owned total incremental outcome, an exact relay C x overall over the finite-masked complete-case support); CallawaySantAnna's SUPPORTED SUBSET is simple|event_study|group|total - it has no calendar aggregator (the DEFERRED 'Calendar-time aggregation' row), and aggregate('calendar') raises naming what is supported; 'total' is panel/non-survey only (RC-routed, declared-survey_design, divergent bare-cluster, and pre-upgrade-kit fits raise NotImplementedError naming the reason; the mass replays from the fit-time agg_gt_cells/is_survey_fit kit snapshots). BOOTSTRAP fits: 'simple' and, where supported, 'total' RELAY the stored overall quintet verbatim (percentile se/p/CI beside the finite safe_inference t; 'total' scales att/se/CI by C) with a NaN df column, while the recompute levels (event_study/group) fail closed pending draw retention (BootstrapReplaySpec is the TODO row) - the per-level policy converged with [M-027] (previously ALL levels failed closed; the relay never publishes an analytical df beside percentile inference). DiagnosticReport now derives the event-study surface via post-fit aggregate('event_study') when the raw field is absent, so its ES-gated checks run on plain fits (derivation failures surface as explicit skip reasons)." - id: M-021 kind: param group: aggregate-postfit @@ -243,7 +243,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/imputation.py, diff_diff/imputation_aggregation.py, diff_diff/imputation_results.py, diff_diff/imputation_bootstrap.py, diff_diff/aggregation.py, diff_diff/results_base.py, diff_diff/honest_did.py, diff_diff/pretrends.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt, diff_diff/diagnostic_report.py] - notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-118], warn-and-still-work; the imputation_did wrapper forwards the sentinel so a plain wrapper call never fires the aggregate warning; since 3.9 the wrapper itself warns per [M-070]). NO fit-time value validation existed and none is added (unknown strings silently act like None; the post-fit successor fails closed via the mixin vocabulary - a behavior improvement). The successor is a PANEL-BACKED lazy recompute kit: ES/group aggregation is a target-specific Theorem-3 recompute from the working panel + untreated FE model (no compact influence payload can honor a different balance_e), so the kit's bookkeeping holds REFERENCES to the SAME per-fit objects self._fit_data already retains for pretrend_test() - ZERO marginal memory and unchanged pickles via memoization (the _estimator_ref field already ships the panel; enumeration in the REGISTRY ImputationDiD Note). Value snapshots (treatment_groups copy, config scalars, a dataclasses.replace copy of survey_metadata, and the survey_df_seed/survey_df_final df channels) isolate recompute and the ES carrier from public-field mutation; aggregate() reads NOTHING mutable off the results object except the deliberate overall-quintet relay and the bootstrap_results gate. SUPPORTED SUBSET simple|event_study|group; calendar/'all' fail closed via the mixin; weights= rejected. Bootstrap fits: 'simple' RELAYS the stored percentile quintet verbatim (finite safe_inference t included) with a NaN df column, while the recompute levels fail closed (the per-target psi machinery makes replay tractable - a TODO row). The prior fail-closed-for-ALL-levels uniform-parity decision was superseded 2026-08-05 with the [M-027] per-level convergence; its rationale - never publish analytical provenance beside percentile inference - is honored by the NaN df column. Replicate-weight fits REPLAY the extracted _replicate_override_aggregates with a LEVEL-MATCHED stack ([overall, ES] or [overall, groups]): compute_replicate_refit_variance validates replicates jointly, so aggregate(L) reproduces fit(aggregate=L) exactly and a fit(aggregate='all') surface is NOT the equivalence target when a replicate NaNs on one family's targets; the same joint-stack coupling means migrating a replicate fit from fit(aggregate=) to plain fit changes the OVERALL row's se/CI/df on degenerate designs (documented migration delta, CHANGELOG + REGISTRY note (e) + a contract-test pin). pretrends=True + replicate: post-fit aggregate('event_study') raises the same NotImplementedError the fit-time gate raises (per-replicate lead refits unimplemented); group/simple still work. Simple relay: n = n_treated_obs with n_kind='obs' (the treated/control unit sets OVERLAP, so the CS/EDiD disjoint-units convention cannot apply - the StackedDiD carve-out class); df = the survey_df_final snapshot (what the stored overall inference received). Group rows record per-row df_used at each safe_inference call (additive row-dict key; the replicate override rewrites it, the bootstrap override clears it, the all-NaN cohort branch writes no key - consumers read via .get). The M-127 df_convention inert-config warning predicate is REVISED to reachability (pretrends AND not-replicate AND (deprecated fit-time ES/all supplied OR n_bootstrap <= 0)) because post-fit aggregate() made the old aggregate-keyed claim false; reachability-BASED, not exact - a fit whose bootstrap later fails (bootstrap_results=None) can still aggregate post-fit, so that corner warns spuriously (recorded on M-127 too). Container admission NOT widened: ImputationDiD is rejected BY DESIGN (no joint ES covariance - per-horizon conservative SEs only; see M-093). balance_e moves as its own row [M-118]. DiagnosticReport now derives the event-study surface via post-fit aggregate('event_study') when the raw field is absent, so its ES-gated checks run on plain fits (derivation failures surface as explicit skip reasons)." + notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-118], warn-and-still-work; the imputation_did wrapper forwards the sentinel so a plain wrapper call never fires the aggregate warning; since 3.9 the wrapper itself warns per [M-070]). NO fit-time value validation existed and none is added (unknown strings silently act like None; the post-fit successor fails closed via the mixin vocabulary - a behavior improvement). The successor is a PANEL-BACKED lazy recompute kit: ES/group aggregation is a target-specific Theorem-3 recompute from the working panel + untreated FE model (no compact influence payload can honor a different balance_e), so the kit's bookkeeping holds REFERENCES to the SAME per-fit objects self._fit_data already retains for pretrend_test() - ZERO marginal memory and unchanged pickles via memoization (the _estimator_ref field already ships the panel; enumeration in the REGISTRY ImputationDiD Note). Value snapshots (treatment_groups copy, config scalars, a dataclasses.replace copy of survey_metadata, and the survey_df_seed/survey_df_final df channels) isolate recompute and the ES carrier from public-field mutation; aggregate() reads NOTHING mutable off the results object except the deliberate overall-quintet relay and the bootstrap_results gate. SUPPORTED SUBSET simple|event_study|group|total ('total' since 2026-08-16: exact relay C x overall with C = the FINITE-tau complete-case support snapshot 'total_support', stashed at kit build alongside the panel-backed refs BECAUSE the kit's df is a live _fit_data reference - the one non-ref value snapshot this row's zero-marginal-memory claim gains; pickles still unchanged via memoization since the stash is a float; declared-survey_design fits and pre-upgrade kits missing the stash raise NotImplementedError); calendar/'all' fail closed via the mixin; weights= rejected. Bootstrap fits: 'simple' and 'total' RELAY the stored percentile quintet verbatim (finite safe_inference t included) with a NaN df column, while the recompute levels fail closed (the per-target psi machinery makes replay tractable - a TODO row). The prior fail-closed-for-ALL-levels uniform-parity decision was superseded 2026-08-05 with the [M-027] per-level convergence; its rationale - never publish analytical provenance beside percentile inference - is honored by the NaN df column. Replicate-weight fits REPLAY the extracted _replicate_override_aggregates with a LEVEL-MATCHED stack ([overall, ES] or [overall, groups]): compute_replicate_refit_variance validates replicates jointly, so aggregate(L) reproduces fit(aggregate=L) exactly and a fit(aggregate='all') surface is NOT the equivalence target when a replicate NaNs on one family's targets; the same joint-stack coupling means migrating a replicate fit from fit(aggregate=) to plain fit changes the OVERALL row's se/CI/df on degenerate designs (documented migration delta, CHANGELOG + REGISTRY note (e) + a contract-test pin). pretrends=True + replicate: post-fit aggregate('event_study') raises the same NotImplementedError the fit-time gate raises (per-replicate lead refits unimplemented); group/simple still work. Simple relay: n = n_treated_obs with n_kind='obs' (the treated/control unit sets OVERLAP, so the CS/EDiD disjoint-units convention cannot apply - the StackedDiD carve-out class); df = the survey_df_final snapshot (what the stored overall inference received). Group rows record per-row df_used at each safe_inference call (additive row-dict key; the replicate override rewrites it, the bootstrap override clears it, the all-NaN cohort branch writes no key - consumers read via .get). The M-127 df_convention inert-config warning predicate is REVISED to reachability (pretrends AND not-replicate AND (deprecated fit-time ES/all supplied OR n_bootstrap <= 0)) because post-fit aggregate() made the old aggregate-keyed claim false; reachability-BASED, not exact - a fit whose bootstrap later fails (bootstrap_results=None) can still aggregate post-fit, so that corner warns spuriously (recorded on M-127 too). Container admission NOT widened: ImputationDiD is rejected BY DESIGN (no joint ES covariance - per-horizon conservative SEs only; see M-093). balance_e moves as its own row [M-118]. DiagnosticReport now derives the event-study surface via post-fit aggregate('event_study') when the raw field is absent, so its ES-gated checks run on plain fits (derivation failures surface as explicit skip reasons)." - id: M-022 kind: param group: aggregate-postfit @@ -257,7 +257,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/two_stage.py, diff_diff/two_stage_aggregation.py, diff_diff/two_stage_results.py, diff_diff/two_stage_bootstrap.py, diff_diff/aggregation.py, diff_diff/results_base.py, diff_diff/honest_did.py, diff_diff/pretrends.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt, diff_diff/diagnostic_report.py] - notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-119], warn-and-still-work; the two_stage_did wrapper forwards the sentinel so a plain wrapper call never fires the aggregate warning; since 3.9 the wrapper itself warns per [M-071]). NO fit-time value validation existed and none is added (unknown strings silently act like None; the post-fit successor fails closed via the mixin vocabulary). The successor is a PANEL-BACKED lazy recompute kit: each level is a fresh Stage-2 OLS + joint Gardner-GMM sandwich, so the kit retains a COLUMN-SUBSET COPY of the working frame (only the columns the moved methods read by name, deduplicated - cluster= may legally name the unit/time/first_treat column) plus the Stage-1 FE model, masks, and survey objects. MEMORY CONTRACT: this is the FIRST panel retention on TwoStageDiD results - O(n_obs) incl. unit/time/cluster identifier columns on every results object and pickle, and replicate designs additionally retain the (n_obs x R) replicate matrix via resolved_survey; the CS/EDiD identifier-minimization guarantee deliberately does NOT hold (a store_kit opt-out is a DEFERRED row). score_pad_mask/cluster_ids_full are stored as the Wave-E.3-GATED values fit actually passed. Value snapshots (treatment_groups copy, overall_att, survey_df_stage2/survey_df_final, a dataclasses.replace copy of survey_metadata) isolate recompute and the ES carrier from public-field mutation. SUPPORTED SUBSET simple|event_study|group; calendar/'all' fail closed; weights= rejected. Bootstrap fits: 'simple' RELAYS the stored percentile quintet verbatim (finite safe_inference t included) with a NaN df column, while the recompute levels fail closed (per-level GMM scores are function-locals; replay is a TODO row) - the prior uniform fail-closed decision superseded 2026-08-05 with the [M-027] per-level convergence; a fit whose bootstrap FAILED (bootstrap_results=None, analytical inference retained) aggregates normally. Replicate-weight fits REPLAY the extracted _replay_replicate_inference with a LEVEL-MATCHED stack - aggregate(L) reproduces fit(aggregate=L) exactly; fit(aggregate='all') is NOT the equivalence target on degenerate designs, and the same joint-stack coupling makes the migration to plain fit change the OVERALL row's se/CI/df there (documented migration delta). Post-fit aggregate('event_study') reproduces the M-092 container contract exactly: analytical fits thread the recomputed joint vcov + vcov_index + the finite-and->0 df scalar through the carrier; replicate fits thread vcov=None/index=None with the REPLAYED level-matched df. Simple relay: n = n_treated_obs with n_kind='obs' (overlapping unit sets - StackedDiD carve-out class); df = the survey_df_final snapshot (on replicate fits that value came from the [overall]-only stack - snapshotted, never re-derived). Group relay df is a SCALAR broadcast (deliberate divergence from ImputationDiD's per-row df_used: _stage2_group passes one immutable survey_df to every row, so the scalar is provenance-exact by construction and the moved method stays verbatim). Container admission NOT widened: DEFERRED pending a normalization derivation - analytical surfaces carry the real joint Gardner-GMM covariance (M-092), but pre-period coefficients are stage-1 residual means, not reference-normalized contrasts, while HonestDiD's Delta arithmetic hard-codes delta_0=0 (see M-093 + the DEFERRED.md paper-gated row). balance_e moves as its own row [M-119]. DiagnosticReport now derives the event-study surface via post-fit aggregate('event_study') when the raw field is absent, so its ES-gated checks run on plain fits (derivation failures surface as explicit skip reasons)." + notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-119], warn-and-still-work; the two_stage_did wrapper forwards the sentinel so a plain wrapper call never fires the aggregate warning; since 3.9 the wrapper itself warns per [M-071]). NO fit-time value validation existed and none is added (unknown strings silently act like None; the post-fit successor fails closed via the mixin vocabulary). The successor is a PANEL-BACKED lazy recompute kit: each level is a fresh Stage-2 OLS + joint Gardner-GMM sandwich, so the kit retains a COLUMN-SUBSET COPY of the working frame (only the columns the moved methods read by name, deduplicated - cluster= may legally name the unit/time/first_treat column) plus the Stage-1 FE model, masks, and survey objects. MEMORY CONTRACT: this is the FIRST panel retention on TwoStageDiD results - O(n_obs) incl. unit/time/cluster identifier columns on every results object and pickle, and replicate designs additionally retain the (n_obs x R) replicate matrix via resolved_survey; the CS/EDiD identifier-minimization guarantee deliberately does NOT hold (a store_kit opt-out is a DEFERRED row). score_pad_mask/cluster_ids_full are stored as the Wave-E.3-GATED values fit actually passed. Value snapshots (treatment_groups copy, overall_att, survey_df_stage2/survey_df_final, a dataclasses.replace copy of survey_metadata) isolate recompute and the ES carrier from public-field mutation. SUPPORTED SUBSET simple|event_study|group|total ('total' since 2026-08-16: exact relay C x overall with C = the post-filter treatment-indicator support - omega_1_mask minus non-finite _y_tilde rows, read from the kit's private column-subset frame COPY, so no fit-time stash is needed; declared-survey_design fits raise NotImplementedError); calendar/'all' fail closed; weights= rejected. Bootstrap fits: 'simple' and 'total' RELAY the stored percentile quintet verbatim (finite safe_inference t included) with a NaN df column, while the recompute levels fail closed (per-level GMM scores are function-locals; replay is a TODO row) - the prior uniform fail-closed decision superseded 2026-08-05 with the [M-027] per-level convergence; a fit whose bootstrap FAILED (bootstrap_results=None, analytical inference retained) aggregates normally. Replicate-weight fits REPLAY the extracted _replay_replicate_inference with a LEVEL-MATCHED stack - aggregate(L) reproduces fit(aggregate=L) exactly; fit(aggregate='all') is NOT the equivalence target on degenerate designs, and the same joint-stack coupling makes the migration to plain fit change the OVERALL row's se/CI/df there (documented migration delta). Post-fit aggregate('event_study') reproduces the M-092 container contract exactly: analytical fits thread the recomputed joint vcov + vcov_index + the finite-and->0 df scalar through the carrier; replicate fits thread vcov=None/index=None with the REPLAYED level-matched df. Simple relay: n = n_treated_obs with n_kind='obs' (overlapping unit sets - StackedDiD carve-out class); df = the survey_df_final snapshot (on replicate fits that value came from the [overall]-only stack - snapshotted, never re-derived). Group relay df is a SCALAR broadcast (deliberate divergence from ImputationDiD's per-row df_used: _stage2_group passes one immutable survey_df to every row, so the scalar is provenance-exact by construction and the moved method stays verbatim). Container admission NOT widened: DEFERRED pending a normalization derivation - analytical surfaces carry the real joint Gardner-GMM covariance (M-092), but pre-period coefficients are stage-1 residual means, not reference-normalized contrasts, while HonestDiD's Delta arithmetic hard-codes delta_0=0 (see M-093 + the DEFERRED.md paper-gated row). balance_e moves as its own row [M-119]. DiagnosticReport now derives the event-study surface via post-fit aggregate('event_study') when the raw field is absent, so its ES-gated checks run on plain fits (derivation failures surface as explicit skip reasons)." - id: M-023 kind: param group: aggregate-postfit @@ -271,7 +271,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/efficient_did.py, diff_diff/efficient_did_aggregation.py, diff_diff/efficient_did_results.py, diff_diff/aggregation.py, diff_diff/results_base.py, diff_diff/honest_did.py, diff_diff/pretrends.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt] - notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-120], warn-and-still-work since the params genuinely worked). NO fit-time value validation existed and none is added: unknown strings silently act like None on the deprecated path, unchanged; the post-fit successor fails closed on unknown types via the mixin vocabulary - a behavior improvement. The successor is a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a view relay): fit() computes nothing extra, the results object retains an AggregationKit referencing the per-(g,t) EIF dict (O(n_units x n_gt) dominant payload; full buffer enumeration in the REGISTRY EfficientDiD Note) plus O(n_units) bookkeeping, PRIVATE per-row snapshots of group_time_effects/groups/time_periods plus the pt_assumption/alpha/anticipation/n-total provenance (recompute and the ES carrier never read the mutable public fields), and the POST-OVERALL df_survey snapshot (captured before the ES/group gates - the group pass can degenerate the working df on replicate designs, and every aggregation seeds from the post-overall value), and aggregate('event_study'/'group', balance_e=) recomputes on demand while 'simple' relays the stored overall row bit-exact (n = treated+control units - disjoint by construction; df = the snapshot, provenance-exact where survey_metadata.df_survey can diverge in the degenerate n_valid<=1 replicate state). SUPPORTED SUBSET simple|event_study|group; calendar and 'all' fail closed via the mixin; weights= rejected. Bootstrap fits: 'simple' RELAYS the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column, while the recompute levels (ES/group) fail closed - per-horizon draws are not retained (exact-replay wiring is a TODO row). The prior uniform-conservatism BY-DECISION rule (no level publishes analytical-provenance fields beside percentile inference) was superseded 2026-08-05 with the [M-027] per-level convergence; its rationale is honored by the relay's NaN df column. Group rows record per-row df_used at each safe_inference call (additive public row-dict key; the fit-time bootstrap override clears it - CS precedent); the post-fit group relay publishes the per-row array, a stated divergence from CS's conservative-min scalar broadcast. PT-Post reference provenance: the membership-gated reference_period property (SunAbraham rule - never synthesized when the anchor cell was not estimated) marks the materialized mechanical zero anchor is_reference in the container, and plot_event_study's inferred reference correctly shifts to -1-anticipation on PT-Post anticipation>0 fits. store_eif now governs only the public influence_functions field - the kit ALWAYS retains the EIF dict (memory-contract change; a store_kit opt-out is a DEFERRED row). Container admission NOT widened to EfficientDiD (see M-093); balance_e moves as its own row [M-120]." + notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-120], warn-and-still-work since the params genuinely worked). NO fit-time value validation existed and none is added: unknown strings silently act like None on the deprecated path, unchanged; the post-fit successor fails closed on unknown types via the mixin vocabulary - a behavior improvement. The successor is a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a view relay): fit() computes nothing extra, the results object retains an AggregationKit referencing the per-(g,t) EIF dict (O(n_units x n_gt) dominant payload; full buffer enumeration in the REGISTRY EfficientDiD Note) plus O(n_units) bookkeeping, PRIVATE per-row snapshots of group_time_effects/groups/time_periods plus the pt_assumption/alpha/anticipation/n-total provenance (recompute and the ES carrier never read the mutable public fields), and the POST-OVERALL df_survey snapshot (captured before the ES/group gates - the group pass can degenerate the working df on replicate designs, and every aggregation seeds from the post-overall value), and aggregate('event_study'/'group', balance_e=) recomputes on demand while 'simple' relays the stored overall row bit-exact (n = treated+control units - disjoint by construction; df = the snapshot, provenance-exact where survey_metadata.df_survey can diverge in the degenerate n_valid<=1 replicate state). SUPPORTED SUBSET simple|event_study|group|total ('total' since 2026-08-16: exact relay C x overall with C = the integer sum of kept post-anticipation finite cells' n_treated from the kit's deep-copied group_time_effects snapshot - never the non-integral n_units x sum(cohort_fractions) float product; fits declaring ANY survey_design - the kit's unit_level_weights/resolved_survey_unit markers, weight-type-agnostic - raise NotImplementedError); calendar and 'all' fail closed via the mixin; weights= rejected. Bootstrap fits: 'simple' and 'total' RELAY the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column, while the recompute levels (ES/group) fail closed - per-horizon draws are not retained (exact-replay wiring is a TODO row). The prior uniform-conservatism BY-DECISION rule (no level publishes analytical-provenance fields beside percentile inference) was superseded 2026-08-05 with the [M-027] per-level convergence; its rationale is honored by the relay's NaN df column. Group rows record per-row df_used at each safe_inference call (additive public row-dict key; the fit-time bootstrap override clears it - CS precedent); the post-fit group relay publishes the per-row array, a stated divergence from CS's conservative-min scalar broadcast. PT-Post reference provenance: the membership-gated reference_period property (SunAbraham rule - never synthesized when the anchor cell was not estimated) marks the materialized mechanical zero anchor is_reference in the container, and plot_event_study's inferred reference correctly shifts to -1-anticipation on PT-Post anticipation>0 fits. store_eif now governs only the public influence_functions field - the kit ALWAYS retains the EIF dict (memory-contract change; a store_kit opt-out is a DEFERRED row). Container admission NOT widened to EfficientDiD (see M-093); balance_e moves as its own row [M-120]." - id: M-024 kind: param group: aggregate-postfit @@ -285,7 +285,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/stacked_did.py, diff_diff/stacked_did_results.py, diff_diff/aggregation.py, diff_diff/results_base.py, diff_diff/honest_did.py, diff_diff/pretrends.py, diff_diff/diagnostic_report.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt] - notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns; supplying ANY value, None included, warns - CS-style warn-and-still-work, since the param genuinely worked, unlike dCDH's raise; group/all + unknown values keep their ValueErrors AFTER the warning). The event-study surface is now ALWAYS computed at fit - the pooled regression always included the interactions, and the extraction + per-event Bell-McCaffrey contrast DOFs (hc2_bm) are no longer gated on fit-time aggregate, so the param is behaviorally inert; consequences: plain-fit summary() prints the ES table, to_dataframe('event_study') is total on 3.9+ fits, DiagnosticReport's heterogeneity and (kappa_pre>=2) parallel_trends checks now run on plain fits, and the hc2_bm overall contrast shares the m>1 batch-relative noise floor (fail-closed + warned; REGISTRY Note). Both aggregate() levels are pure VIEWS: 'event_study' -> build_event_study_surface(self); 'simple' relays overall_att/se/t/p/CI bit-exact with target='att' (the scalar is the post-average, NOT the per-event trimmed aggregate ATT), n=n_treated_units/'units' (treated-only scope: control units overlap treated across sub-experiments, so no disjoint total exists), df=inference_df. SUPPORTED SUBSET simple|event_study - group/calendar fail closed via the mixin message; balance_e applies to no level (empty vocabulary); weights= rejected. Every inference mode relays (survey TSL, replicate refit, hc2_bm fail-close). honest/pretrends container admission widened to Stacked-sourced containers (see M-093): base_period='universal' ClassVar + singleton reference_event_times property thread the provenance; analytical fits carry df_survey=None (the resolver reads df_inference, a name Stacked does not use - deliberate: normal-theory FLCI, see the REGISTRY Note and the TODO adapter-naming row's cross-reference)." + notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns; supplying ANY value, None included, warns - CS-style warn-and-still-work, since the param genuinely worked, unlike dCDH's raise; group/all + unknown values keep their ValueErrors AFTER the warning). The event-study surface is now ALWAYS computed at fit - the pooled regression always included the interactions, and the extraction + per-event Bell-McCaffrey contrast DOFs (hc2_bm) are no longer gated on fit-time aggregate, so the param is behaviorally inert; consequences: plain-fit summary() prints the ES table, to_dataframe('event_study') is total on 3.9+ fits, DiagnosticReport's heterogeneity and (kappa_pre>=2) parallel_trends checks now run on plain fits, and the hc2_bm overall contrast shares the m>1 batch-relative noise floor (fail-closed + warned; REGISTRY Note). Both aggregate() levels are pure VIEWS: 'event_study' -> build_event_study_surface(self); 'simple' relays overall_att/se/t/p/CI bit-exact with target='att' (the scalar is the post-average, NOT the per-event trimmed aggregate ATT), n=n_treated_units/'units' (treated-only scope: control units overlap treated across sub-experiments, so no disjoint total exists), df=inference_df. SUPPORTED SUBSET simple|event_study - group/calendar/total fail closed via the mixin message ('total' joined the vocabulary 2026-08-16; StackedDiD is STAGED OUT of totals - estimand ill-defined under weighting= variants, the DEFERRED remainder); balance_e applies to no level (empty vocabulary); weights= rejected. Every inference mode relays (survey TSL, replicate refit, hc2_bm fail-close). honest/pretrends container admission widened to Stacked-sourced containers (see M-093): base_period='universal' ClassVar + singleton reference_event_times property thread the provenance; analytical fits carry df_survey=None (the resolver reads df_inference, a name Stacked does not use - deliberate: normal-theory FLCI, see the REGISTRY Note and the TODO adapter-naming row's cross-reference)." - id: M-025 kind: param group: aggregate-postfit @@ -313,7 +313,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/chaisemartin_dhaultfoeuille.py, diff_diff/chaisemartin_dhaultfoeuille_results.py, diff_diff/aggregation.py, diff_diff/results_base.py] - notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns). The param NEVER computed aggregations on dCDH - every non-None value raised 'reserved for Phase 3' NotImplementedError since introduction - so there is no legacy surface to keep populating: a non-None value now raises ValueError pointing at the post-fit route, and the kwargs-splitting chaisemartin_dhaultfoeuille() wrapper forwards the shim unchanged. The successor is a pure VIEW, not a kit recompute: aggregate('event_study') returns build_event_study_surface(self) (Phase-1 L_max=None fits return the 2-row l=1 view - NOT an error; L_max>=1 the multi-horizon l1_first_switch surface), aggregate('simple') a one-row AggregationResult relaying overall_att/se/t/p/CI bit-exact with target = _estimand_label() (DID_M / DID_1 / delta / the trends-linear first-difference label whose overall row is all-NaN by design) and estimand-aware n/n_kind (N_S switcher_cells at L_max=None; N_1 groups at L_max==1; NaN/None for the delta, which averages horizon-specific N_l). SUPPORTED SUBSET simple|event_study - group/calendar fail closed via the mixin message; balance_e applies to no level here (empty _AGGREGATE_BALANCE_E_TYPES); weights= rejected. BOOTSTRAP FITS ARE PERMITTED (the library-wide per-level relay rule, since [M-027] converged the recompute adopters' 'simple' relays onto it too) because nothing is recomputed - each row relays the STORED inference: percentile-bootstrap DID_M/DID_1 rows carry NaN df, while the L_max>=2 delta keeps analytical safe_inference with a possibly-finite survey df even under bootstrap (REGISTRY Note, Phase 2 cost-benefit delta SE), resolved by _overall_inference_df from event_study_df / the post-fit-refreshed survey_metadata.df_survey. dCDH placebo/normalized surfaces stay estimator-native; only the aggregation entry point moves. NOTE: the dCDH event-study CONTAINER is deliberately rejected by compute_honest_did/compute_pretrends_power (source-scoped admission; the l1 placebo semantics need honest_did's native dCDH branch) - see M-093." + notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns). The param NEVER computed aggregations on dCDH - every non-None value raised 'reserved for Phase 3' NotImplementedError since introduction - so there is no legacy surface to keep populating: a non-None value now raises ValueError pointing at the post-fit route, and the kwargs-splitting chaisemartin_dhaultfoeuille() wrapper forwards the shim unchanged. The successor is a pure VIEW, not a kit recompute: aggregate('event_study') returns build_event_study_surface(self) (Phase-1 L_max=None fits return the 2-row l=1 view - NOT an error; L_max>=1 the multi-horizon l1_first_switch surface), aggregate('simple') a one-row AggregationResult relaying overall_att/se/t/p/CI bit-exact with target = _estimand_label() (DID_M / DID_1 / delta / the trends-linear first-difference label whose overall row is all-NaN by design) and estimand-aware n/n_kind (N_S switcher_cells at L_max=None; N_1 groups at L_max==1; NaN/None for the delta, which averages horizon-specific N_l). SUPPORTED SUBSET simple|event_study - group/calendar/total fail closed via the mixin message ('total' joined the vocabulary 2026-08-16; no dCDH adoption); balance_e applies to no level here (empty _AGGREGATE_BALANCE_E_TYPES); weights= rejected. BOOTSTRAP FITS ARE PERMITTED (the library-wide per-level relay rule, since [M-027] converged the recompute adopters' 'simple' relays onto it too) because nothing is recomputed - each row relays the STORED inference: percentile-bootstrap DID_M/DID_1 rows carry NaN df, while the L_max>=2 delta keeps analytical safe_inference with a possibly-finite survey df even under bootstrap (REGISTRY Note, Phase 2 cost-benefit delta SE), resolved by _overall_inference_df from event_study_df / the post-fit-refreshed survey_metadata.df_survey. dCDH placebo/normalized surfaces stay estimator-native; only the aggregation entry point moves. NOTE: the dCDH event-study CONTAINER is deliberately rejected by compute_honest_did/compute_pretrends_power (source-scoped admission; the l1 placebo semantics need honest_did's native dCDH branch) - see M-093." - id: M-027 kind: param group: aggregate-postfit @@ -1397,7 +1397,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/staggered.py, diff_diff/staggered_results.py, diff_diff/aggregation.py] - notes: "balance_e moves from fit() onto aggregate() with [M-020]. Previously tracked only as the prose 'balance_e moves to aggregate() in the same PR' inside M-020's notes, which no test could assert - the same un-rowed-obligation class the gating-completeness amendment closed. Applies to event-study aggregation ONLY (the shipped code threads it nowhere else), so aggregate(type='simple'|'group', balance_e=...) raises rather than silently ignoring it. The other three balance_e sites (ImputationDiD, TwoStageDiD, EfficientDiD) get their own rows in the PRs that migrate them - EfficientDiD's is [M-120]." + notes: "balance_e moves from fit() onto aggregate() with [M-020]. Previously tracked only as the prose 'balance_e moves to aggregate() in the same PR' inside M-020's notes, which no test could assert - the same un-rowed-obligation class the gating-completeness amendment closed. Applies to event-study aggregation ONLY (the shipped code threads it nowhere else), so aggregate(type='simple'|'group'|'total', balance_e=...) raises rather than silently ignoring it. The other three balance_e sites (ImputationDiD, TwoStageDiD, EfficientDiD) get their own rows in the PRs that migrate them - EfficientDiD's is [M-120]." - id: M-120 kind: param group: aggregate-postfit @@ -1411,7 +1411,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/efficient_did.py, diff_diff/efficient_did_aggregation.py, diff_diff/efficient_did_results.py, diff_diff/aggregation.py] - notes: "balance_e moves from fit() onto aggregate() with [M-023] (joint FutureWarning; the M-117 twin). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group', balance_e=...) raises. EfficientDiD's balance rule is the ANCHOR-HORIZON rule - keep cohorts with a finite effect at e == balance_e, then retain all their horizons - the SAME rule CallawaySantAnna uses, divergent only from ImputationDiD/TwoStageDiD's balanced-window rule (their own rows document theirs). An anchor no cohort reaches warns (UserWarning) and yields a legal zero-row EventStudyResults container." + notes: "balance_e moves from fit() onto aggregate() with [M-023] (joint FutureWarning; the M-117 twin). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group'|'total', balance_e=...) raises. EfficientDiD's balance rule is the ANCHOR-HORIZON rule - keep cohorts with a finite effect at e == balance_e, then retain all their horizons - the SAME rule CallawaySantAnna uses, divergent only from ImputationDiD/TwoStageDiD's balanced-window rule (their own rows document theirs). An anchor no cohort reaches warns (UserWarning) and yields a legal zero-row EventStudyResults container." - id: M-118 kind: param group: aggregate-postfit @@ -1425,7 +1425,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/imputation.py, diff_diff/imputation_aggregation.py, diff_diff/imputation_results.py, diff_diff/aggregation.py] - notes: "balance_e moves from fit() onto aggregate() with [M-021] (joint FutureWarning; the M-117/M-120 twin; the imputation_did wrapper forwards the sentinel). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group', balance_e=...) raises. ImputationDiD's balance rule is the BALANCED-WINDOW rule - a cohort is retained iff its observed relative-time set (checked against the FULL panel via _build_cohort_rel_times) covers the contiguous window [-balance_e, max_h] (imputation_aggregation.py _compute_balanced_cohort_mask) - divergent from CS/EfficientDiD's anchor-horizon rule ([M-117]/[M-120] document theirs; TwoStageDiD shares the balanced-window rule, [M-119]). A window no cohort satisfies warns (UserWarning) and yields the reference-marker-only dict -> a legal near-empty container." + notes: "balance_e moves from fit() onto aggregate() with [M-021] (joint FutureWarning; the M-117/M-120 twin; the imputation_did wrapper forwards the sentinel). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group'|'total', balance_e=...) raises. ImputationDiD's balance rule is the BALANCED-WINDOW rule - a cohort is retained iff its observed relative-time set (checked against the FULL panel via _build_cohort_rel_times) covers the contiguous window [-balance_e, max_h] (imputation_aggregation.py _compute_balanced_cohort_mask) - divergent from CS/EfficientDiD's anchor-horizon rule ([M-117]/[M-120] document theirs; TwoStageDiD shares the balanced-window rule, [M-119]). A window no cohort satisfies warns (UserWarning) and yields the reference-marker-only dict -> a legal near-empty container." - id: M-119 kind: param group: aggregate-postfit @@ -1439,7 +1439,7 @@ rows: warning: FutureWarning test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/two_stage.py, diff_diff/two_stage_aggregation.py, diff_diff/two_stage_results.py, diff_diff/aggregation.py] - notes: "balance_e moves from fit() onto aggregate() with [M-022] (joint FutureWarning; the M-117/M-120 twin; the two_stage_did wrapper forwards the sentinel). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group', balance_e=...) raises. TwoStageDiD's balance rule is the BALANCED-WINDOW rule - a cohort is retained iff its observed relative-time set (checked against the full panel via _build_cohort_rel_times) covers the contiguous window [-balance_e, max_h] (two_stage_aggregation.py _stage2_event_study) - the ImputationDiD rule ([M-118]), divergent from CS/EfficientDiD's anchor-horizon rule. Zero qualifying cohorts warns ('No cohorts satisfy balance_e=...') and yields the reference-row-only dict with vcov=None -> a legal near-empty container." + notes: "balance_e moves from fit() onto aggregate() with [M-022] (joint FutureWarning; the M-117/M-120 twin; the two_stage_did wrapper forwards the sentinel). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group'|'total', balance_e=...) raises. TwoStageDiD's balance rule is the BALANCED-WINDOW rule - a cohort is retained iff its observed relative-time set (checked against the full panel via _build_cohort_rel_times) covers the contiguous window [-balance_e, max_h] (two_stage_aggregation.py _stage2_event_study) - the ImputationDiD rule ([M-118]), divergent from CS/EfficientDiD's anchor-horizon rule. Zero qualifying cohorts warns ('No cohorts satisfy balance_e=...') and yields the reference-row-only dict with vcov=None -> a legal near-empty container." - id: M-126 kind: behavior group: variance-cr1-convergence diff --git a/docs/v4-design.md b/docs/v4-design.md index 817937c4f..46d24632c 100644 --- a/docs/v4-design.md +++ b/docs/v4-design.md @@ -513,7 +513,7 @@ post-fit `aggregate()` in the same release the params die. The rule above is otherwise unchanged: no OTHER estimator may add a fit-time `aggregate=`. **Vocabulary.** Closed set: `"simple"`, `"event_study"`, `"group"`, -`"calendar"`, plus per-estimator documented extras where the estimand demands +`"calendar"`, `"total"`, plus per-estimator documented extras where the estimand demands them (ContinuousDiD adds `"dose"` [M-025]; HAD's `"overall"` maps to `"simple"` [M-027] - its fit-time mode selector, and the workflow twin `did_had_pretest_workflow(aggregate=)` [M-139], both resolve by @@ -524,6 +524,23 @@ EXISTING post-fit `aggregate(type=)` - the emfx-style prior art for this section's pattern - plus `summary(aggregation=)` and `to_dataframe(aggregation=)` [M-044] [M-086] [M-087]. +**(Amended 2026-08-16, total-incremental-outcome promotion.)** `"total"` +joins the closed set: the estimator-owned total incremental outcome, an +exact relay `C x overall` CONDITIONAL on the realized aggregation mass, +where `C` is the adopter's finite-masked complete-case aggregation support +(CallawaySantAnna: the kept post-anticipation finite cells' mass, replayed +from a fit-time kit snapshot; ImputationDiD: the finite-tau support - +fixing the documented raw-|Omega_1| overcount of the MMM `scale="auto"` +route for total exports; TwoStageDiD: the post-filter treatment-indicator +support; EfficientDiD: the kept cells' integer `n_treated` sum). Adopters: +those four, panel non-survey fits only - repeated-cross-section-routed +fits, fits declaring a `survey_design=`, and CS bare-`cluster=` fits whose +cohort-mass weighting diverges from the complete-case count all raise +`NotImplementedError` naming the reason (DEFERRED tracks the survey/RC +remainder and its att*dC mass-uncertainty variance term). StackedDiD is +staged out (estimand ill-defined under `weighting=` variants; DEFERRED). +Non-adopters keep the fail-closed vocabulary error. + **Heterogeneous-`target` rendering** (added with [M-025], the first producer of a mixed-target `AggregationResult` - ContinuousDiD's att/acrt dual estimand; widened with [M-027]: the same target-column + neutral @@ -563,7 +580,9 @@ or administrative IDs. Analytical-vs-bootstrap inference of the aggregated estimand follows the fit's inference method, and the bootstrap gate is PER-LEVEL (converged with [M-027] across CS/EDiD/Imputation/TwoStage, which previously failed closed on every level): a level that RELAYS the -fit's stored inference verbatim (`'simple'` on every adopter; all of +fit's stored inference verbatim (`'simple'` on every adopter; `'total'` +on its four adopters - the x C scaling is homogeneous of degree 1, so the +relay is exact under percentile inference too; all of dCDH/StackedDiD/HAD) is faithful under any inference regime and stays available on bootstrapped fits, with the df COLUMN NaN'd there - no df governs percentile inference, so a relay never publishes analytical diff --git a/tests/test_aggregate_contract.py b/tests/test_aggregate_contract.py index f435f2bed..5ecf978d2 100644 --- a/tests/test_aggregate_contract.py +++ b/tests/test_aggregate_contract.py @@ -286,7 +286,7 @@ def test_all_is_not_a_post_fit_type(self, fitted): def test_error_names_supported_types(self, fitted): with pytest.raises(ValueError) as exc: fitted.aggregate("nonsense") - for level in ("simple", "event_study", "group"): + for level in ("simple", "event_study", "group", "total"): assert level in str(exc.value) def test_weights_rejected(self, fitted): @@ -294,7 +294,7 @@ def test_weights_rejected(self, fitted): with pytest.raises(ValueError, match="weights"): fitted.aggregate("simple", "cohort_share") - @pytest.mark.parametrize("level", ["simple", "group"]) + @pytest.mark.parametrize("level", ["simple", "group", "total"]) def test_balance_e_rejected_where_inert(self, fitted, level): """balance_e applies to event-study aggregation ONLY - silently ignoring it elsewhere would accept a user argument that does nothing.""" @@ -311,7 +311,9 @@ def test_bootstrap_recompute_levels_fail_closed(self, panel): for level in ("event_study", "group"): with pytest.raises(NotImplementedError, match="bootstrap") as exc: boot.aggregate(level) - assert "aggregate('simple') relays" in str(exc.value) + assert "aggregate('simple') and, where supported, aggregate('total') relay" in str( + exc.value + ) def test_bootstrap_simple_relays_stored_quintet(self, panel): """Per-level policy (converged with M-027): 'simple' is a bit-exact @@ -490,7 +492,7 @@ def test_repeated_cross_sections_match_fit_time(self, level): native = CallawaySantAnna(panel=False).fit(data, aggregate="all", **FIT_KW) _assert_level_matches(post, native, level) - @pytest.mark.parametrize("level", ["simple", "group", "event_study"]) + @pytest.mark.parametrize("level", ["simple", "group", "event_study", "total"]) def test_anticipation_matches_fit_time(self, panel, level): post = CallawaySantAnna(anticipation=1).fit(panel, **FIT_KW) with warnings.catch_warnings(): @@ -498,7 +500,7 @@ def test_anticipation_matches_fit_time(self, panel, level): native = CallawaySantAnna(anticipation=1).fit(panel, aggregate="all", **FIT_KW) _assert_level_matches(post, native, level) - @pytest.mark.parametrize("level", ["simple", "group", "event_study"]) + @pytest.mark.parametrize("level", ["simple", "group", "event_study", "total"]) def test_universal_base_period_matches_fit_time(self, panel, level): """REGISTRY gives universal bases their own reference-cell and VCV-index semantics (a zero reference cell per cohort), which the @@ -510,7 +512,7 @@ def test_universal_base_period_matches_fit_time(self, panel, level): native = CallawaySantAnna(**kw).fit(panel, aggregate="all", **FIT_KW) _assert_level_matches(post, native, level) - @pytest.mark.parametrize("level", ["simple", "group", "event_study"]) + @pytest.mark.parametrize("level", ["simple", "group", "event_study", "total"]) def test_not_yet_treated_with_anticipation_matches_fit_time(self, panel, level): """The not-yet-treated control group interacts with anticipation in picking each (g, t) comparison set - a different code path from the @@ -551,6 +553,22 @@ def test_survey_numbers_match_fit_time(self, survey_fit, level): def _assert_level_matches(post, native, level): """Compare a post-fit aggregation against the fit-time surface at 1e-14.""" + if level == "total": + # Identity form (total == n x overall on the SAME fit) AND the + # independent-mass form (n pinned against a frame-derived oracle from + # the OTHER fit's public cells) - the identity alone passes for ANY C. + got = post.aggregate("total") + antic = getattr(native, "anticipation", 0) + n_exp = sum( + d["n_treated"] + for (g, t), d in native.group_time_effects.items() + if t >= g - antic and np.isfinite(d["effect"]) + ) + assert got.n[0] == float(n_exp) + assert np.allclose(got.att[0], got.n[0] * native.overall_att, rtol=1e-12) + assert np.allclose(got.se[0], got.n[0] * native.overall_se, rtol=1e-12) + assert np.allclose(got.t_stat[0], native.overall_t_stat, rtol=1e-14, atol=0, equal_nan=True) + return if level == "simple": got = post.aggregate("simple") assert np.allclose(got.att[0], native.overall_att, rtol=1e-14, atol=1e-14) @@ -798,7 +816,9 @@ def test_off_vocabulary_n_kind_raises(self): with pytest.raises(ValueError, match="vocabulary"): AggregationResult(df=None, n_kind="widgets", **self._kw()) - @pytest.mark.parametrize("level,expected", [("simple", "units"), ("group", "cells")]) + @pytest.mark.parametrize( + "level,expected", [("simple", "units"), ("group", "cells"), ("total", "obs")] + ) def test_cs_n_kinds_are_in_the_shared_vocabulary(self, fitted, level, expected): from diff_diff.results_base import N_KIND_VOCABULARY @@ -1919,7 +1939,9 @@ def test_bootstrap_recompute_levels_fail_closed(self, efficient_panel): for level in ("event_study", "group"): with pytest.raises(NotImplementedError, match="bootstrap") as exc: res.aggregate(level) - assert "aggregate('simple') relays" in str(exc.value) + assert "aggregate('simple') and, where supported, aggregate('total') relay" in str( + exc.value + ) def test_bootstrap_simple_relays_stored_quintet(self, efficient_panel): res = _fit_efficient(efficient_panel, est_kw={"n_bootstrap": 20, "seed": 1}) @@ -2433,7 +2455,9 @@ def test_bootstrap_recompute_levels_fail_closed(self, imputation_panel): for level in ("event_study", "group"): with pytest.raises(NotImplementedError, match="bootstrap") as exc: res.aggregate(level) - assert "aggregate('simple') relays" in str(exc.value) + assert "aggregate('simple') and, where supported, aggregate('total') relay" in str( + exc.value + ) def test_bootstrap_simple_relays_stored_quintet(self, imputation_panel): res = _fit_imputation(imputation_panel, est_kw={"n_bootstrap": 19, "seed": 1}) @@ -2924,7 +2948,9 @@ def test_bootstrap_recompute_levels_fail_closed(self, twostage_panel): for level in ("event_study", "group"): with pytest.raises(NotImplementedError, match="bootstrap") as exc: res.aggregate(level) - assert "aggregate('simple') relays" in str(exc.value) + assert "aggregate('simple') and, where supported, aggregate('total') relay" in str( + exc.value + ) def test_bootstrap_simple_relays_stored_quintet(self, twostage_panel): res = _fit_twostage(twostage_panel, est_kw={"n_bootstrap": 19, "seed": 1}) @@ -4000,3 +4026,647 @@ def test_no_future_warning_from_post_fit_routes(self, had_fitted_2p, had_fitted_ had_fitted_2p.aggregate("simple") had_fitted_multi.aggregate("event_study") assert [w for w in caught if issubclass(w.category, FutureWarning)] == [] + + +# ============================================================================= +# aggregate('total') - the estimator-owned total incremental outcome +# (exact relay C x overall, CONDITIONAL on the realized aggregation mass) +# ============================================================================= + + +def _raw_treated_obs(frame, time_col="time"): + """Frame-derived treated unit-period count (the container-independent oracle).""" + return int(((frame["first_treat"] > 0) & (frame[time_col] >= frame["first_treat"])).sum()) + + +class TestTotalCallawaySantAnna: + """CS total: relay identity, mass oracles, and the fail-closed routings.""" + + def test_identity_and_independent_mass(self, panel, fitted): + tot = fitted.aggregate("total") + simple = fitted.aggregate("simple") + assert tot.level == "total" + assert tot.label[0] == "total" and tot.target[0] == "total" + assert tot.n_kind == "obs" and tot.weight[0] == 1.0 + # Independent mass oracles: kept finite post cells AND the raw frame + # (clean balanced DGP, so the two coincide). + n_cells = sum( + d["n_treated"] + for (g, t), d in fitted.group_time_effects.items() + if t >= g and np.isfinite(d["effect"]) + ) + assert tot.n[0] == float(n_cells) == float(_raw_treated_obs(panel)) + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.allclose(tot.se[0], tot.n[0] * simple.se[0], rtol=1e-12) + assert tot.t_stat[0] == simple.t_stat[0] + assert tot.p_value[0] == simple.p_value[0] + assert np.allclose(tot.conf_int_lower[0], tot.n[0] * simple.conf_int_lower[0], rtol=1e-12) + assert np.allclose(tot.conf_int_upper[0], tot.n[0] * simple.conf_int_upper[0], rtol=1e-12) + + def test_df_carrier_plain_panel_is_nan(self, fitted): + # A plain (non-cluster, non-survey) analytical CS fit has no survey df + # and no df_inference, so the total row's df provenance column is NaN - + # same carrier as the simple relay. + tot = fitted.aggregate("total") + simple = fitted.aggregate("simple") + assert np.isnan(tot.df[0]) == np.isnan(simple.df[0]) + + def test_bare_cluster_admitted_with_finite_df(self, panel): + """cluster= (no survey_design) is a mainstream ADMITTED routing: the + synthesized all-ones design is not a survey. On a clean panel the + cohort-mass branch coincides with the complete-case count, and the + fit's cluster df flows into the total row's df carrier (clean + fixtures alone would hide a wrong carrier).""" + from diff_diff.aggregation import resolve_inference_df + + res = CallawaySantAnna(cluster="unit").fit(panel, **FIT_KW) + tot = res.aggregate("total") + assert tot.n[0] == float(_raw_treated_obs(panel)) + expected_df = resolve_inference_df(res) + if expected_df is not None and np.isfinite(expected_df): + assert tot.df[0] == expected_df + assert np.allclose(tot.att[0], tot.n[0] * res.aggregate("simple").att[0], rtol=1e-12) + + def test_cluster_divergence_fails_closed(self, panel): + """Bare-cluster fit whose kept cells have INCOMPLETE treated support: + the cohort-mass weighting (full cohort per period) disagrees with the + complete-case count, so the total is ambiguous and fails closed.""" + holed = panel.copy() + treated_rows = holed.index[ + (holed["first_treat"] > 0) & (holed["time"] >= holed["first_treat"]) + ] + holed.loc[treated_rows[:3], "y"] = np.nan + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + plain = CallawaySantAnna().fit(holed.dropna(), **FIT_KW) + clustered = CallawaySantAnna(cluster="unit").fit(holed.dropna(), **FIT_KW) + # The plain fit is admitted at the complete-case count... + tot_plain = plain.aggregate("total") + assert tot_plain.n[0] < _raw_treated_obs(panel) + # ...while the cluster fit's masses branch diverges and refuses. + with pytest.raises(NotImplementedError, match="incomplete treated support"): + clustered.aggregate("total") + + def test_rcs_fails_closed(self): + data = _rcs() + with warnings.catch_warnings(): + warnings.simplefilter("ignore", UserWarning) + res = CallawaySantAnna(panel=False).fit(data, **FIT_KW) + with pytest.raises(NotImplementedError, match="repeated-cross-section"): + res.aggregate("total") + + def test_genuinely_unbalanced_rc_routing_fails_closed(self, panel): + thinned = panel.drop(panel.index[::13]).reset_index(drop=True) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = CallawaySantAnna(allow_unbalanced_panel=True).fit(thinned, **FIT_KW) + with pytest.raises(NotImplementedError, match="repeated-cross-section"): + res.aggregate("total") + + def test_balanced_panel_with_unbalanced_flag_is_admitted(self, panel): + """allow_unbalanced_panel=True on a BALANCED panel stays panel-routed + (routing keys on the data, not the constructor flag) and is admitted.""" + res = CallawaySantAnna(allow_unbalanced_panel=True).fit(panel, **FIT_KW) + tot = res.aggregate("total") + assert tot.n[0] == float(_raw_treated_obs(panel)) + + def test_survey_fails_closed_and_gate_is_immutable(self, survey_fit): + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + survey_fit.aggregate("total") + # Mutating the PUBLIC field must not bypass the gate: it reads the + # kit's fit-time is_survey_fit snapshot, never self.survey_metadata. + survey_fit.survey_metadata = None + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + survey_fit.aggregate("total") + + def test_survey_plus_rc_gets_the_rc_message(self): + """A fit that is BOTH survey-declared and RC-routed deterministically + gets the (more informative) RC message - the gate order is pinned.""" + from diff_diff import SurveyDesign + + data = _rcs() + data = data.assign(psu=np.arange(len(data)) % 10) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = CallawaySantAnna(panel=False).fit( + data, survey_design=SurveyDesign(psu="psu"), **FIT_KW + ) + with pytest.raises(NotImplementedError, match="repeated-cross-section"): + res.aggregate("total") + + def test_legacy_kit_fails_closed(self, panel): + """Pre-upgrade kits (no fit-time snapshots) refuse with the refit + message - never a mutable-field or public-replay fallback.""" + for missing in ("agg_gt_cells", "is_survey_fit", "both"): + res = CallawaySantAnna().fit(panel, **FIT_KW) + bk = res._aggregation_kit.bookkeeping + if missing in ("agg_gt_cells", "both"): + del bk["agg_gt_cells"] + if missing in ("is_survey_fit", "both"): + del bk["is_survey_fit"] + with pytest.raises(NotImplementedError, match="refit"): + res.aggregate("total") + + def test_post_fit_mutation_immunity(self, panel): + """The mass comes from the immutable kit snapshot; the scalars come + from the same stored fields the simple relay reads - so a public-field + edit cannot desynchronize total == n x simple.""" + res = CallawaySantAnna().fit(panel, **FIT_KW) + n_before = res.aggregate("total").n[0] + for k in list(res.group_time_effects): + res.group_time_effects[k]["n_treated"] = 9999 + res.group_time_effects[k]["effect"] = np.nan + tot = res.aggregate("total") + assert tot.n[0] == n_before + assert np.allclose(tot.att[0], tot.n[0] * res.aggregate("simple").att[0], rtol=1e-12) + + def test_anticipation_window_cells_count(self, panel): + """anticipation=1 keeps the g-1 window cells in the post set (cells + with g - anticipation <= t < g COUNT as post), so the mass grows by + exactly the window cells' treated observations.""" + res0 = CallawaySantAnna().fit(panel, **FIT_KW) + res1 = CallawaySantAnna(anticipation=1).fit(panel, **FIT_KW) + n0 = res0.aggregate("total").n[0] + n1 = res1.aggregate("total").n[0] + window = sum( + d["n_treated"] + for (g, t), d in res1.group_time_effects.items() + if g - 1 <= t < g and np.isfinite(d["effect"]) + ) + assert n1 == n0 + window and window > 0 + + def test_universal_base_reference_cells_excluded(self, panel): + """Universal-base fits carry reference cells; the anticipation filter + alone excludes them from the mass (their period is < g by + construction), so the mass matches the varying-base fit's.""" + res_u = CallawaySantAnna(base_period="universal").fit(panel, **FIT_KW) + res_v = CallawaySantAnna().fit(panel, **FIT_KW) + assert res_u.aggregate("total").n[0] == res_v.aggregate("total").n[0] + + def test_zero_c_arms_emit_all_nan_row_without_warning(self, panel): + """Empty post set / all-NaN effects -> all-NaN row, NO UserWarning + (the no-post-fit-re-warn convention), on the plain-panel branch AND + the cohort-mass (bare-cluster) branch - the isfinite(C) guard must + run BEFORE the coincidence comparison (NaN != 0.0 is True).""" + for ctor in ({}, {"cluster": "unit"}): + res = CallawaySantAnna(**ctor).fit(panel, **FIT_KW) + cells = res._aggregation_kit.bookkeeping["agg_gt_cells"] + res._aggregation_kit.bookkeeping["agg_gt_cells"] = tuple( + (g, t, float("nan"), n) for (g, t, _, n) in cells + ) + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + tot = res.aggregate("total") + assert [w for w in caught if issubclass(w.category, UserWarning)] == [] + for col in ("att", "se", "t_stat", "p_value", "conf_int_lower", "conf_int_upper", "n"): + assert np.isnan(getattr(tot, col)[0]), (ctor, col) + assert np.isnan(tot.df[0]) + + def test_true_overflow_blanks_whole_row(self, panel): + """A FINITE overall value that becomes non-finite BY the x C scaling + blanks the row entirely - the one path allowed to blank a finite att, + pinned on both the att and se halves of the guard.""" + for field_name in ("overall_att", "overall_se"): + res = CallawaySantAnna().fit(panel, **FIT_KW) + setattr(res, field_name, 1e308) + tot = res.aggregate("total") + for col in ("att", "se", "t_stat", "p_value", "conf_int_lower", "conf_int_upper", "n"): + assert np.isnan(getattr(tot, col)[0]), (field_name, col) + + def test_bootstrap_total_relays(self, panel): + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + boot = CallawaySantAnna(n_bootstrap=49, seed=42).fit(panel, **FIT_KW) + tot = boot.aggregate("total") + simple = boot.aggregate("simple") + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.allclose(tot.se[0], tot.n[0] * simple.se[0], rtol=1e-12) + assert np.isnan(tot.df[0]) + + def test_pickle_round_trip(self, fitted): + import pickle + + clone = pickle.loads(pickle.dumps(fitted)) + a, b = fitted.aggregate("total"), clone.aggregate("total") + assert a.att[0] == b.att[0] and a.n[0] == b.n[0] + + def test_inert_across_other_levels(self, fitted): + before = fitted.aggregate("total").att[0] + fitted.aggregate("group") + fitted.aggregate("event_study") + assert fitted.aggregate("total").att[0] == before + + def test_constant_tau_recovers_total(self): + """Leg 2 (runs in the default suite; POINT-ESTIMATE estimand check - + no confidence-interval assertion intended): on a constant-effect DGP + the total approximates tau x the raw treated unit-period count, with + BOTH oracle factors container-independent.""" + rng = np.random.default_rng(7) + tau = 2.0 + rows = [] + for u in range(60): + g = [0, 4, 5][u % 3] + for t in range(1, 8): + y = 1.0 + 0.3 * t + 0.2 * (u % 7) + (tau if g and t >= g else 0.0) + rows.append({"unit": u, "time": t, "first_treat": g, "y": y + rng.normal(0, 0.05)}) + frame = pd.DataFrame(rows) + res = CallawaySantAnna().fit(frame, **FIT_KW) + tot = res.aggregate("total") + expected = tau * _raw_treated_obs(frame) + assert abs(tot.att[0] - expected) / expected < 0.05 + + +class TestTotalNonAdopters: + """'total' is vocabulary-wide; non-adopters fail closed with the suffix.""" + + def test_stacked_rejects_total_with_vocabulary_suffix(self, stacked_fitted): + # StackedDiD is the in-file AggregationMixin non-adopter exemplar: + # 'total' is vocabulary-wide, so the mixin's fail-closed error names + # it as known-but-unimplemented rather than unknown. + with pytest.raises(ValueError, match="Unsupported aggregation type") as exc: + stacked_fitted.aggregate("total") + assert "vocabulary" in str(exc.value) + + def test_wooldridge_custom_aggregate_rejects_total_its_own_way(self): + """WooldridgeDiDResults is NOT an AggregationMixin subclass - its + custom aggregate(type=...) raises its own fail-closed ValueError + naming what it supports (no vocabulary suffix expected).""" + from diff_diff import WooldridgeDiD + + rng = np.random.default_rng(9) + rows = [] + for u in range(40): + g = [0, 3, 4][u % 3] + for t in range(1, 7): + rows.append( + { + "unit": u, + "time": t, + "first_treat": g, + "y": 1.0 + 0.1 * t + (1.0 if g and t >= g else 0.0) + rng.normal(0, 0.2), + } + ) + df = pd.DataFrame(rows) + res = WooldridgeDiD().fit( + df, outcome="y", unit="unit", time="time", first_treat="first_treat" + ) + with pytest.raises(ValueError, match="type must be one of"): + res.aggregate("total") + + +class TestTotalImputation: + """ImputationDiD total: finite-support mass (sum-tau identity).""" + + def _fit(self, frame, **kw): + from diff_diff import ImputationDiD + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return ImputationDiD().fit(frame, **IMPUTATION_KW, **kw) + + def test_identity_sum_tau_and_raw_frame_oracle(self): + frame = _imputation_panel() + res = self._fit(frame) + tot = res.aggregate("total") + simple = res.aggregate("simple") + # fully identified DGP + assert tot.n[0] == float(_raw_treated_obs(frame, time_col="period")) + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.allclose(tot.se[0], tot.n[0] * simple.se[0], rtol=1e-12) + # C x overall == sum(tau) exactly (overall is the finite-support mean) + assert np.allclose(tot.att[0], res.treatment_effects["tau_hat"].sum(), rtol=1e-10) + assert tot.df[0] == simple.df[0] or (np.isnan(tot.df[0]) and np.isnan(simple.df[0])) + + def test_survey_fails_closed_analytic_and_replicate(self): + for replicate in (False, True): + frame, rep_cols = _imputation_survey_panel(replicate=replicate) + design = _imputation_survey_design(rep_cols if replicate else None) + from diff_diff import ImputationDiD + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = ImputationDiD().fit(frame, survey_design=design, **IMPUTATION_KW) + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + res.aggregate("total") + res.survey_metadata = None # gate reads the kit snapshot + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + res.aggregate("total") + + def test_live_frame_mutation_immunity(self): + """The kit's 'df' is a live _fit_data reference; the mass must come + from the fit-time total_support stash, not a frame recompute.""" + res = self._fit(_imputation_panel()) + n_before = res.aggregate("total").n[0] + res._aggregation_kit.bookkeeping["df"]["_tau_hat"] = np.nan + assert res.aggregate("total").n[0] == n_before + + def test_legacy_kit_fails_closed(self): + res = self._fit(_imputation_panel()) + del res._aggregation_kit.bookkeeping["total_support"] + with pytest.raises(NotImplementedError, match="refit"): + res.aggregate("total") + + def test_anticipation_window_obs_enter_the_mass(self): + """anticipation=1 widens the treated support to t >= g - 1, so the + total mass grows by exactly the frame-derived window observations + (ImputationDiD's mass comes from anticipation-adjusted masks).""" + from diff_diff import ImputationDiD + + frame = _imputation_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + n0 = ImputationDiD().fit(frame, **IMPUTATION_KW).aggregate("total").n[0] + n1 = ImputationDiD(anticipation=1).fit(frame, **IMPUTATION_KW).aggregate("total").n[0] + window = int( + ( + (frame["first_treat"] > 0) + & (frame["period"] >= frame["first_treat"] - 1) + & (frame["period"] < frame["first_treat"]) + ).sum() + ) + assert window > 0 + assert n1 == n0 + window + + def test_zero_support_emits_all_nan_row_without_warning(self): + """C == 0 (the all-unidentified fit's support) maps to a NaN mass and + an all-NaN row - never a finite n=0 beside NaN inference.""" + res = self._fit(_imputation_panel()) + res._aggregation_kit.bookkeeping["total_support"] = 0.0 + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + tot = res.aggregate("total") + assert [w for w in caught if issubclass(w.category, UserWarning)] == [] + for col in ("att", "se", "t_stat", "p_value", "conf_int_lower", "conf_int_upper", "n"): + assert np.isnan(getattr(tot, col)[0]), col + assert np.isnan(tot.df[0]) + + def test_degenerate_inference_passes_through(self): + """Constant outcome: overall att=0, se=0, CI=(nan, nan), t/p NaN with + C > 0. The total row MIRRORS the simple relay - att/n stay finite, + inherited NaNs pass through, nothing is blanket-blanked (the repo's + non-estimable-row convention).""" + frame = _imputation_panel() + frame = frame.assign(outcome=5.0) + res = self._fit(frame) + tot = res.aggregate("total") + simple = res.aggregate("simple") + # The constant-outcome fit really is degenerate: se == 0, CI == NaN. + assert simple.se[0] == 0.0 and np.isnan(simple.conf_int_lower[0]) + assert np.isfinite(tot.n[0]) and tot.n[0] > 0 + assert tot.att[0] == tot.n[0] * simple.att[0] + assert tot.se[0] == 0.0 + assert np.isnan(tot.conf_int_lower[0]) + + def test_bootstrap_total_relays(self): + from diff_diff import ImputationDiD + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = ImputationDiD(n_bootstrap=49, seed=3).fit(_imputation_panel(), **IMPUTATION_KW) + tot = res.aggregate("total") + simple = res.aggregate("simple") + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.allclose(tot.se[0], tot.n[0] * simple.se[0], rtol=1e-12) + assert tot.t_stat[0] == simple.t_stat[0] + assert tot.p_value[0] == simple.p_value[0] + assert np.isnan(tot.df[0]) + + def test_leg4_divergence_finite_support_not_raw(self): + """On a partially unidentified fit the total uses the FINITE support - + the fix for the documented scale='auto' overcount ('simple''s n stays + raw |Omega_1| by contract).""" + frame = _imputation_panel() + # Drop the never-treated observations at the last period so some + # treated cells lose identification (the tutorial divergence recipe). + keep = ~((frame["first_treat"] == 0) & (frame["period"] == frame["period"].max())) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = self._fit(frame[keep]) + tot = res.aggregate("total") + simple = res.aggregate("simple") + raw_n = simple.n[0] + # The recipe MUST produce reduced finite support (166 < 250 on this + # fixture) - an unconditional pin, so a fixture drift that stops + # exercising the divergence fails loudly instead of passing vacuously. + assert np.isfinite(tot.n[0]) and tot.n[0] < raw_n, (tot.n[0], raw_n) + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert not np.allclose(tot.att[0], raw_n * simple.att[0], rtol=1e-12) + + +class TestTotalTwoStage: + """TwoStageDiD total: post-filter D-support.""" + + def _fit(self, frame, **kw): + from diff_diff import TwoStageDiD + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return TwoStageDiD().fit(frame, **TWOSTAGE_KW, **kw) + + def test_identity_and_raw_frame_oracle(self): + frame = _twostage_panel() + res = self._fit(frame) + tot = res.aggregate("total") + simple = res.aggregate("simple") + assert tot.n[0] == float(_raw_treated_obs(frame, time_col="period")) + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.allclose(tot.se[0], tot.n[0] * simple.se[0], rtol=1e-12) + + def test_survey_fails_closed_and_gate_is_immutable(self): + from diff_diff import SurveyDesign, TwoStageDiD + + frame = _twostage_panel() + frame = frame.assign(psu=frame["unit"] % 10) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = TwoStageDiD().fit(frame, survey_design=SurveyDesign(psu="psu"), **TWOSTAGE_KW) + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + res.aggregate("total") + res.survey_metadata = None + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + res.aggregate("total") + + def test_anticipation_window_obs_enter_the_mass(self): + """anticipation=1 widens the treated support to t >= g - 1, so the + total mass grows by exactly the frame-derived window observations + (TwoStageDiD's mass comes from anticipation-adjusted masks).""" + from diff_diff import TwoStageDiD + + frame = _twostage_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + n0 = TwoStageDiD().fit(frame, **TWOSTAGE_KW).aggregate("total").n[0] + n1 = TwoStageDiD(anticipation=1).fit(frame, **TWOSTAGE_KW).aggregate("total").n[0] + window = int( + ( + (frame["first_treat"] > 0) + & (frame["period"] >= frame["first_treat"] - 1) + & (frame["period"] < frame["first_treat"]) + ).sum() + ) + assert window > 0 + assert n1 == n0 + window + + def test_reduced_post_filter_support(self): + """total.n is the POST-FILTER treatment-indicator support: non-finite + treated `_y_tilde` rows leave D, so n drops below raw |Omega_1| (the + 'simple' row's documented pre-filter count).""" + res = self._fit(_twostage_panel()) + bk = res._aggregation_kit.bookkeeping + d_mask = np.asarray(bk["omega_1_mask"], dtype=bool) + raw_n = float(d_mask.sum()) + # Degrade three treated rows' first-stage residuals in the kit's + # PRIVATE frame copy (the state the fit-time masker would have seen). + idx = bk["df"].index[d_mask][:3] + bk["df"].loc[idx, "_y_tilde"] = np.nan + tot = res.aggregate("total") + assert tot.n[0] == raw_n - 3 + simple = res.aggregate("simple") + assert simple.n[0] == raw_n # 'simple' keeps the raw count by contract + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + + def test_zero_support_emits_all_nan_row_without_warning(self): + res = self._fit(_twostage_panel()) + bk = res._aggregation_kit.bookkeeping + d_mask = np.asarray(bk["omega_1_mask"], dtype=bool) + bk["df"].loc[bk["df"].index[d_mask], "_y_tilde"] = np.nan + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + tot = res.aggregate("total") + assert [w for w in caught if issubclass(w.category, UserWarning)] == [] + for col in ("att", "se", "t_stat", "p_value", "conf_int_lower", "conf_int_upper", "n"): + assert np.isnan(getattr(tot, col)[0]), col + + def test_bootstrap_total_relays(self): + from diff_diff import TwoStageDiD + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = TwoStageDiD(n_bootstrap=49, seed=3).fit(_twostage_panel(), **TWOSTAGE_KW) + tot = res.aggregate("total") + simple = res.aggregate("simple") + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.isnan(tot.df[0]) + + +class TestTotalEfficientDiD: + """EfficientDiD total: integer sum of kept cells' n_treated.""" + + def _fit(self, frame, **kw): + from diff_diff import EfficientDiD + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return EfficientDiD().fit(frame, **EFFICIENT_KW, **kw) + + def test_identity_and_integer_mass(self): + frame = _efficient_panel() + res = self._fit(frame) + tot = res.aggregate("total") + simple = res.aggregate("simple") + # Integer by construction: sum of per-cell integer n_treated, never + # the float n_units x sum(cohort_fractions) product. + assert tot.n[0] == int(tot.n[0]) + n_cells = sum( + c["n_treated"] + for (g, t), c in res._aggregation_kit.bookkeeping["group_time_effects"].items() + if t >= g and np.isfinite(c["effect"]) + ) + assert tot.n[0] == float(n_cells) + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.allclose(tot.se[0], tot.n[0] * simple.se[0], rtol=1e-12) + + def test_survey_fails_closed_all_design_shapes(self): + """The gate is weight-type-agnostic: analytic pweight, replicate, + UNWEIGHTED psu-only, and analytic FWEIGHT declared designs all fail + closed (fweight resolved weights stay RAW - the gate must not depend + on any weight-scale property).""" + from diff_diff import EfficientDiD, SurveyDesign + + frame, _ = _efficient_survey_panel() + frame_rep, rep_cols = _efficient_survey_panel(replicate=True) + plain = _efficient_panel() + arms = [ + (frame, _efficient_survey_design()), + (frame_rep, _efficient_survey_design(rep_cols)), + (plain.assign(psu=plain["unit"] % 10), SurveyDesign(psu="psu")), + ( + plain.assign(w=10.0 + (plain["unit"] % 5)), + SurveyDesign(weights="w", weight_type="fweight"), + ), + ] + for arm_frame, arm_design in arms: + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = EfficientDiD().fit(arm_frame, survey_design=arm_design, **EFFICIENT_KW) + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + res.aggregate("total") + res.survey_metadata = None + with pytest.raises(NotImplementedError, match="declaring a survey_design"): + res.aggregate("total") + + def test_bootstrap_total_relays(self): + from diff_diff import EfficientDiD + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = EfficientDiD(n_bootstrap=49, seed=3).fit(_efficient_panel(), **EFFICIENT_KW) + tot = res.aggregate("total") + simple = res.aggregate("simple") + assert np.allclose(tot.att[0], tot.n[0] * simple.att[0], rtol=1e-12) + assert np.allclose(tot.se[0], tot.n[0] * simple.se[0], rtol=1e-12) + assert tot.t_stat[0] == simple.t_stat[0] + assert tot.p_value[0] == simple.p_value[0] + assert np.isnan(tot.df[0]) + + def test_post_fit_mutation_immunity(self): + """The mass reads the kit's deep-copied cells; a NaN edit of the + public field (which WOULD perturb finite-keeper selection) is inert.""" + res = self._fit(_efficient_panel()) + n_before = res.aggregate("total").n[0] + for k in list(res.group_time_effects): + res.group_time_effects[k]["effect"] = np.nan + res.group_time_effects[k]["n_treated"] = 777 + assert res.aggregate("total").n[0] == n_before + + def test_anticipation_window_cells_enter_the_mass(self): + """anticipation=1 keeps the g-1 window cells as post, so C grows by + exactly those cells' n_treated (the CS twin of this pin).""" + from diff_diff import EfficientDiD + + frame = _efficient_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res0 = EfficientDiD().fit(frame, **EFFICIENT_KW) + res1 = EfficientDiD(anticipation=1).fit(frame, **EFFICIENT_KW) + n0 = res0.aggregate("total").n[0] + n1 = res1.aggregate("total").n[0] + window = sum( + c["n_treated"] + for (g, t), c in res1._aggregation_kit.bookkeeping["group_time_effects"].items() + if g - 1 <= t < g and np.isfinite(float(c["effect"])) + ) + assert n1 == n0 + window and window > 0 + + def test_zero_keepers_emit_all_nan_row_without_warning(self): + res = self._fit(_efficient_panel()) + gte = res._aggregation_kit.bookkeeping["group_time_effects"] + for cell in gte.values(): + cell["effect"] = float("nan") + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + tot = res.aggregate("total") + assert [w for w in caught if issubclass(w.category, UserWarning)] == [] + for col in ("att", "se", "t_stat", "p_value", "conf_int_lower", "conf_int_upper", "n"): + assert np.isnan(getattr(tot, col)[0]), col + + def test_degenerate_inference_passes_through(self): + frame = _efficient_panel().assign(outcome=5.0) + res = self._fit(frame) + tot = res.aggregate("total") + simple = res.aggregate("simple") + # The constant-outcome fit really is degenerate: se == 0. + assert simple.se[0] == 0.0 + assert np.isfinite(tot.n[0]) and tot.n[0] > 0 + assert tot.att[0] == tot.n[0] * simple.att[0] + assert tot.se[0] == 0.0 diff --git a/tests/test_mmm.py b/tests/test_mmm.py index 55aab5007..b2c2a5987 100644 --- a/tests/test_mmm.py +++ b/tests/test_mmm.py @@ -11,6 +11,7 @@ """ import math +import re import warnings import numpy as np @@ -1037,6 +1038,7 @@ def _make_agg( ses=(0.5,), ns=(30.0,), targets=None, + weight=None, ): """Hand-built AggregationResult mirroring the producers' field conventions.""" k = len(labels) @@ -1053,6 +1055,7 @@ def _make_agg( n=np.array(ns, dtype=float), df=np.full(k, np.nan), n_kind=n_kind, + weight=(None if weight is None else np.array(weight, dtype=float)), estimator=estimator, ) @@ -1114,7 +1117,7 @@ def test_results_object_without_scale_still_type_error(self): def test_rejects_unsupported_level(self): agg = _make_agg(level="calendar") - with pytest.raises(ValueError, match="level must be 'simple' or 'group'"): + with pytest.raises(ValueError, match="level must be 'simple', 'group', or 'total'"): to_pymc_marketing_lift_test( channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg, scale=1.0 ) @@ -1693,3 +1696,147 @@ def test_public_exports(): assert diff_diff.to_meridian_roi_prior is to_meridian_roi_prior assert diff_diff.meridian_calibration_mask is meridian_calibration_mask assert diff_diff.MeridianROIPrior is MeridianROIPrior + + +class TestTotalContainerAdmission: + """level='total' containers: admitted with NO scale, rejected with any.""" + + def _tot(self, att=6.0, se=1.5, n=30.0, estimator="ImputationDiD", **kw): + kw.setdefault("labels", ("total",)) + kw.setdefault("targets", ("total",)) + kw.setdefault("weight", (1.0,)) + return _make_agg( + level="total", + estimator=estimator, + atts=(att,), + ses=(se,), + ns=(n,), + **kw, + ) + + def test_lift_test_accepts_total_without_scale(self): + frame = to_pymc_marketing_lift_test( + channel="tv", x=100.0, delta_x=50.0, aggregation_result=self._tot() + ) + assert frame["delta_y"].iloc[0] == 6.0 + assert frame["sigma"].iloc[0] == 1.5 + + def test_meridian_accepts_total_without_scale(self): + prior = to_meridian_roi_prior(aggregation_result=self._tot(), spend=60.0) + assert prior.roi_mean == pytest.approx(0.1) + + @pytest.mark.parametrize("scale", [2.0, "auto"]) + def test_any_scale_with_total_rejected(self, scale): + with pytest.raises(ValueError, match="already totals"): + to_pymc_marketing_lift_test( + channel="tv", + x=100.0, + delta_x=50.0, + aggregation_result=self._tot(), + scale=scale, + ) + + @pytest.mark.parametrize("estimator", ["StackedDiD", "SomeNewEstimator", None]) + def test_unaudited_provenance_rejected(self, estimator): + """The 'already scaled' claim is provenance-gated like scale='auto': + StackedDiD is staged out, and unknown/missing provenance could be any + hand-built container whose att is NOT a total.""" + agg = self._tot(estimator=estimator) + with pytest.raises(ValueError, match="audited total adopters"): + to_pymc_marketing_lift_test(channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg) + with pytest.raises(ValueError, match="audited total adopters"): + to_meridian_roi_prior(aggregation_result=agg, spend=10.0) + + @pytest.mark.parametrize( + "estimator", + ["CallawaySantAnna", "EfficientDiD", "ImputationDiD", "TwoStageDiD"], + ) + def test_all_four_adopters_admitted(self, estimator): + frame = to_pymc_marketing_lift_test( + channel="tv", x=1.0, delta_x=1.0, aggregation_result=self._tot(estimator=estimator) + ) + assert frame["delta_y"].iloc[0] == 6.0 + + def test_drifted_producer_contract_rejected(self): + """label/n_kind/weight must match what aggregate('total') emits - + schema drift on a hand-altered container fails closed.""" + for kw in ( + {"n_kind": "units"}, + {"labels": ("overall",)}, + {"weight": None}, + {"weight": (0.5,)}, + ): + agg = self._tot(**kw) + with pytest.raises(ValueError, match="producer contract"): + to_pymc_marketing_lift_test( + channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg + ) + + def test_multi_row_total_rejected(self): + agg = _make_agg( + level="total", + labels=("total", "extra"), + targets=("total", "total"), + atts=(1.0, 2.0), + ses=(0.5, 0.5), + ns=(10.0, 10.0), + weight=(0.5, 0.5), + ) + with pytest.raises(ValueError, match="single 'total' row"): + to_pymc_marketing_lift_test(channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg) + + def test_att_target_total_rejected(self): + agg = _make_agg(level="total", labels=("total",), targets=("att",), weight=(1.0,)) + with pytest.raises(ValueError, match="target='total'"): + to_pymc_marketing_lift_test(channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg) + + def test_non_finite_total_rejected(self): + agg = self._tot(att=float("nan"), se=1.0) + with pytest.raises(ValueError, match="no usable point"): + to_pymc_marketing_lift_test(channel="tv", x=1.0, delta_x=1.0, aggregation_result=agg) + + def test_missing_input_errors_name_the_total_route(self): + with pytest.raises(ValueError, match=r"aggregate\('total'\)"): + to_pymc_marketing_lift_test(channel="tv", x=1.0, delta_x=1.0) + with pytest.raises(ValueError, match=r"aggregate\('total'\)"): + to_meridian_roi_prior(spend=10.0) + + def test_hints_keep_pins_and_name_the_total_route(self): + from diff_diff.mmm import _SCALE_HINTS + + assert re.search(r"treated\+control", _SCALE_HINTS["CallawaySantAnna"]) + for est in ("CallawaySantAnna", "EfficientDiD"): + assert re.search(r"aggregate\('total'\)", _SCALE_HINTS[est]) + # EDiD's hint must NOT claim RC/cluster-mass failure modes (it has + # neither); CS's must name them. + assert "repeated-cross-section" not in _SCALE_HINTS["EfficientDiD"] + assert "repeated-cross-section" in _SCALE_HINTS["CallawaySantAnna"] + + def test_end_to_end_cs_and_imputation(self): + from diff_diff import CallawaySantAnna, ImputationDiD + + rng = np.random.default_rng(21) + rows = [] + for u in range(36): + g = [0, 3, 4][u % 3] + for t in range(1, 7): + rows.append( + { + "unit": u, + "time": t, + "first_treat": g, + "y": 1.0 + 0.1 * t + (2.0 if g and t >= g else 0.0) + rng.normal(0, 0.2), + } + ) + frame = pd.DataFrame(rows) + raw_n = int(((frame["first_treat"] > 0) & (frame["time"] >= frame["first_treat"])).sum()) + for cls in (CallawaySantAnna, ImputationDiD): + res = cls().fit(frame, outcome="y", unit="unit", time="time", first_treat="first_treat") + tot = res.aggregate("total") + simple = res.aggregate("simple") + assert tot.n[0] == float(raw_n) + lift = to_pymc_marketing_lift_test( + channel="tv", x=100.0, delta_x=50.0, aggregation_result=tot + ) + assert lift["delta_y"].iloc[0] == pytest.approx(raw_n * simple.att[0]) + assert lift["sigma"].iloc[0] == pytest.approx(raw_n * simple.se[0])