Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- **MMM exporters: container mode on the 3.9 aggregation surface.** Both
`to_pymc_marketing_lift_test` and `to_meridian_roi_prior` now accept
`aggregation_result=` - the pinned `AggregationResult` returned by post-fit
`results.aggregate('simple')` (one experiment row) or `'group'` (one row per
cohort) - together with the new `scale=`, deriving `effect = att * scale` and
`se = se * scale` per row in `to_dataframe()` order (the order every per-row
sequence kwarg aligns to). `scale` is always required with a container: a
finite positive number (scalar or per-row), or `"auto"`, which reads the
container's per-row `n` and is honored ONLY for ImputationDiD/TwoStageDiD
fits - the audited producers whose `n` is exactly the treated unit-periods
the ATT averages over. `"auto"` is an explicit acknowledgement of three
assumptions the container cannot verify (additive-level outcome, unweighted
fit, fully identified effects - both estimators warn at fit time on the
degenerate identification branch, where `att * n` overcounts; behaviorally
pinned in the tests). Everything else fails closed with the remedy inline:
raw results objects and `EventStudyResults` (the exporters never call
`aggregate()` themselves), `calendar`/`dose`/unknown levels, non-`"att"`
target rows (ContinuousDiD's `acrt`, HAD's WAS, dCDH's estimand relays -
container rejected whole, no silent row filtering), rows without usable
inference, and `scale="auto"` on any other estimator provenance
(CallawaySantAnna/EfficientDiD/StackedDiD hints explain their `n` semantics;
routing keys on the container's `estimator` field, never on `n_kind`, which
serves only as a drift guard). The explicit `delta_y`/`sigma` and
`incremental_outcome`/`incremental_outcome_se` route is unchanged and the two
routes are mutually exclusive. Docs: the REGISTRY MMM section, `api/mmm.rst`
(whose examples are now self-contained and executed by
`tests/test_doc_snippets.py`), and both LLM guides drop the pre-3.9 "does
not introspect result objects / deferred to the post-4.0 aggregate() layer"
framing - that layer shipped in 3.9 and this is its consumer. The
estimator-owned-totals remainder for CS/EfficientDiD/StackedDiD stays
tracked in DEFERRED.md.

### Changed
- **Narrative docs migrated off the deprecated fit-time `aggregate=`** (the
3.9 M-020 family; TODO "fit-time aggregate= teachings" sweep, RST half):
Expand Down
2 changes: 1 addition & 1 deletion DEFERRED.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ For survey-specific limitations (`NotImplementedError` paths), see the
| Issue | Location | PR | Priority |
|-------|----------|----|----------|
| EfficientDiD/ImputationDiD/TwoStageDiD kit retention has no memory opt-out: EfficientDiD's kit always retains the per-(g,t) EIF dict (M-023), ImputationDiD's references the `_fit_data` panel objects (M-021, pre-existing retention), and TwoStageDiD's retains a column-subset working-frame copy (M-022, the first new panel retention); a `store_kit=False` constructor knob would restore an opt-out at the cost of post-fit aggregation on that result | `diff_diff/efficient_did.py`, `diff_diff/imputation.py`, `diff_diff/two_stage.py` | 2(b) PR-3a | Low |
| MMM interop: result-derived scaling for the exporters - let a fitted result produce the scoped incremental outcome + SE directly (via the post-4.0 `results.aggregate()` layer, where the estimator owns its aggregation weights/balance/survey masses), so callers need not hand-scale the ATT. v1 is explicit-in by design; this is the seamless follow-up. | `diff_diff/mmm.py`, `docs/v4-design.md` | mmm-interop | Low |
| MMM interop: estimator-owned total incremental outcome for CallawaySantAnna/EfficientDiD/StackedDiD aggregations - their `AggregationResult` containers report units/cells (no treated-obs mass), so container-derived scaling (`aggregation_result=` + `scale="auto"`, shipped for ImputationDiD/TwoStageDiD) cannot cover them and their exporter route requires a caller-supplied numeric `scale=`. Closing the gap means each estimator's own aggregation (which owns its weights/balance/survey masses) computing a total effect + SE. | `diff_diff/mmm.py`, `diff_diff/staggered_aggregation.py` | mmm-interop | Low |
| `SyntheticControl` fit-snapshot residency (`_SyntheticControlFitSnapshot`) — **investigated 2026-07-07, parked**: the snapshot ALIASES the fit's own working pivots (zero extra construction cost); the retained residency implements the documented freeze contract (post-fit mutation of estimator inputs must not change `in_space_placebo()` / `leave_one_out()` / conformal output on an already-returned results object, and `__getstate__` already excludes it from pickles). A compact array representation saves only pandas overhead (the float panel dominates); releasing residency needs new API surface (`release`/opt-out flag) or a freeze-contract change. Revisit on user demand for very large donor panels. | `synthetic_control.py`, `synthetic_control_results.py` | follow-up | Low |
| Stratified survey-PSU multiplier-weight draw-tiling — **investigated 2026-07-07, parked**: the stratified generator (`generate_survey_multiplier_weights_batch`) consumes ONE sequential rng stream stratum-major (`rng.choice(size=(n_bootstrap, n_h))` per stratum, then lonely-PSU pooling), so draw-chunked assembly CANNOT reproduce the stream bit-identically (contra the old row's parenthetical) — it would need per-stratum generator state skipping (PCG64.advance + per-weight-type variate accounting; fragile) or a stream-layout change (MC-level SE changes → baseline/golden recapture + REGISTRY note). Stratified designs have few PSUs, so the full `(n_bootstrap × n_psu)` matrix rarely matters; unstratified (the large-`n_units` case) is already tiled. Revisit only if a large-PSU stratified design hits memory, as a documented stream change. | `diff_diff/bootstrap_chunking.py::iter_survey_multiplier_weight_blocks` | follow-up | Low |
| ChangesInChanges FULL Melly-Santangelo covariate estimator (monotonized integrated-indicator conditional CDFs, treated-post `F_{X|11}` integration, exchangeable bootstrap with variance-weighted KS bands, tail trimming, pre-period specification test). The simplified qte-`xformla` form of the MS pipeline is already supported (`covariates=` on both estimators, parity-tested vs qte 1.3.1); the FULL estimator has no reference implementation (the MS Stata code is the only one; distinct from Kranker's `cic`) and would need simulation-based validation. Reviewed: `docs/methodology/papers/melly-santangelo-2015-review.md`. | `diff_diff/changes_in_changes.py` | #682 | Low |
Expand Down
49 changes: 37 additions & 12 deletions diff_diff/guides/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2873,21 +2873,28 @@ stability policy.
## MMM Calibration Export

Interop builders converting experiment results into Marketing Mix Model
(MMM) calibration inputs. Pure numpy/pandas - no MMM package is imported,
no result object is introspected (the module is purely additive). Design:
EXPLICIT IN, VALIDATED OUT. Reconciling an experiment estimate to a
calibration input needs the target MMM's row granularity (per-geo vs
national), its time window, and the outcome scale (additive levels vs
log/rate/share) - none of which diff-diff can see - so the CALLER supplies
the already-scoped incremental outcome and its SE (read off summary(),
aggregated to the population/window one MMM row represents). diff-diff
assembles the schema, enforces each consumer's guards, converts to the
lognormal parameterization, pools, and emits snippets. Deriving totals
from a fit is deferred to the post-4.0 results.aggregate() layer, where
the estimator owns its aggregation weights/balance/survey masses.
(MMM) calibration inputs. Pure numpy/pandas - no MMM package is imported.
Design: EXPLICIT NUMBERS IN, OR THE PINNED AGGREGATION CONTRACT IN -
VALIDATED OUT. Reconciling an experiment estimate to a calibration input
needs the target MMM's row granularity (per-geo vs national), its time
window, and the outcome scale (additive levels vs log/rate/share), so the
default route stays explicit: the CALLER supplies the already-scoped
incremental outcome and its SE (read off summary(), aggregated to the
population/window one MMM row represents). Alternatively both exporters
take aggregation_result= (the AggregationResult from post-fit
results.aggregate('simple'|'group')) + scale=, deriving effect =
att * scale and se = se * scale per row in to_dataframe() order;
scale="auto" (per-row n off the container) is honored ONLY for
ImputationDiD/TwoStageDiD fits - it acknowledges an additive-level
outcome, an unweighted fit, and fully identified effects - and every
other estimator, raw results objects, and EventStudyResults fail closed
(the module never calls aggregate() itself). diff-diff assembles the
schema, enforces each consumer's guards, converts to the lognormal
parameterization, pools, and emits snippets.

```python
from diff_diff import (
ImputationDiD,
SyntheticDiD,
to_pymc_marketing_lift_test,
to_meridian_roi_prior,
Expand Down Expand Up @@ -2936,9 +2943,27 @@ print(prior.to_code( # ready-to-paste PriorDistribution +
roi_calibration_period="mask", # AND time scope (mask expr) or
# full_model_window=True; sets
)) # media_prior_type accordingly.

# Container route (ImputationDiD/TwoStageDiD): derive the totals from the
# post-fit aggregation instead of hand-scoping.
res = ImputationDiD().fit(data, outcome="outcome", unit="unit",
time="period", first_treat="first_treat")
prior2 = to_meridian_roi_prior(
aggregation_result=res.aggregate("simple"),
scale="auto", # att x n (n = treated unit-periods here)
spend=200_000.0,
)
```

Key points:
- Container mode: aggregation_result= accepts ONLY AggregationResult
('simple' = one experiment row; 'group' = one per cohort, rows in
to_dataframe() order - per-row scale/spend/x/delta_x align to it).
scale= is always required: numeric, or "auto" for the audited
ImputationDiD/TwoStageDiD producers only (CS/EfficientDiD/StackedDiD
n counts units/cells, not treated unit-periods - pass a numeric scale).
A 'group' container's cohorts come from ONE fit: widen se_widening for
Meridian pooling, and note PyMC scores each row as independent evidence.
- Lift frame: one row per experiment, columns [channel, *dims, x, delta_x,
delta_y, sigma]; guards on sigma>0, delta_x!=0, x>=0, x+delta_x>=0,
finite delta_y; dims must not collide with reserved columns and must
Expand Down
2 changes: 1 addition & 1 deletion diff_diff/guides/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ The site is organized into 5 sections, each with a landing page:
- [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html): Rambachan & Roth (2023) sensitivity analysis — robust CI under parallel trends violations, breakdown values
- [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html): Roth (2022) Section II.A-B no-individually-significant (NIS) box-probability pretest power + minimum detectable violation; `pretest_form='nis'` (default) implements the paper's primary form, `pretest_form='wald'` retained as paper-supported alternative (Propositions 1+3+4 all apply); linear-violation MDV in Roth's γ units when relative-time labels are threaded through `fit()`; full Σ_22 routing on non-bootstrap CallawaySantAnna and SunAbraham adapters and on admitted CS-/StackedDiD-sourced `aggregate('event_study')` containers (StackedDiD persists its ES VCV in every inference mode)
- [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html): Analytical and simulation-based power analysis — MDE, sample size, power curves for study design
- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html): Assemble Marketing Mix Model calibration inputs from experiment results (explicit-in / validated-out - the caller passes the already-scoped incremental outcome + SE, the module does NOT rescale a headline ATT). `to_pymc_marketing_lift_test(channel, x, delta_x, delta_y, sigma, dims=, on_wrong_sign=)` builds the PyMC-Marketing/prophetverse lift-test DataFrame with sign/zero/positivity guards. `to_meridian_roi_prior(incremental_outcome, incremental_outcome_se, spend, parameter="roi_m"|"mroi_m", se_widening=)` builds Google Meridian lognormal ROI priors (spend-weighted pooling, lognormal parity with `lognormal_dist_from_mean_std`, channel- and time-scoped `.to_code()` snippet setting `media_prior_type`). Pure numpy/pandas; imports no MMM package; does not introspect result objects. Deriving totals from a fit is deferred to the post-4.0 `results.aggregate()` layer.
- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html): Assemble Marketing Mix Model calibration inputs from experiment results. Two input routes: explicit already-scoped numbers, or the pinned `AggregationResult` container from post-fit `results.aggregate('simple'|'group')` via `aggregation_result=` + `scale=` (effect = att x scale per row, in `to_dataframe()` order; `scale="auto"` reads the container's treated-obs count and is honored ONLY for ImputationDiD/TwoStageDiD fits - it acknowledges an additive-level outcome, an unweighted fit, and fully identified effects; other estimators and raw results objects fail closed). `to_pymc_marketing_lift_test(channel, x, delta_x, delta_y=, sigma=, aggregation_result=, scale=, dims=, on_wrong_sign=)` builds the PyMC-Marketing/prophetverse lift-test DataFrame with sign/zero/positivity guards. `to_meridian_roi_prior(incremental_outcome=, incremental_outcome_se=, aggregation_result=, scale=, spend, parameter="roi_m"|"mroi_m", se_widening=)` builds Google Meridian lognormal ROI priors (spend-weighted pooling, lognormal parity with `lognormal_dist_from_mean_std`, channel- and time-scoped `.to_code()` snippet setting `media_prior_type`; widen `se_widening` when pooling a group-level container's same-fit cohorts). Pure numpy/pandas; imports no MMM package; never calls `aggregate()` itself.
- Conley spatial HAC SE (`vcov_type="conley"`) on cross-sectional `LinearRegression` / `compute_robust_vcov` PLUS panel `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` (with `conley_lag_cutoff=<int>` for within-unit Bartlett temporal HAC) — Conley (1999) spatial-correlation-aware SEs with haversine/euclidean/callable distance metric and Bartlett/uniform spatial kernel; panel path uses the R `conleyreg`-form block-decomposed sandwich (within-period spatial + within-unit Bartlett serial, same-time excluded); parity vs R `conleyreg` (Düsterhöft 2021) on cross-sectional AND panel `lag_cutoff > 0` fixtures. Combining with explicit `cluster=<col>` applies the combined spatial + cluster product kernel `K_total[i,j] = K_space · 1{c_i = c_j}` (cluster must be constant within each unit across periods on the panel path; validator-enforced). DiD takes `unit=<col>` as a fit-time kwarg when `vcov_type="conley"` (not on `__init__`). Sparse k-d-tree fast path auto-activates for `n > 5_000` with bartlett kernel + haversine/euclidean metric

## Tutorials
Expand Down
5 changes: 4 additions & 1 deletion diff_diff/imputation_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,7 +386,10 @@ def _aggregate_simple_result(self, kit: Any) -> AggregationResult:
CS/EDiD disjoint-units convention cannot apply (the StackedDiD
carve-out class); the treated-observation count is the population
the overall ATT averages over and matches every other Imputation
row's n semantics.
row's n semantics. Of that population, only finite-tau-hat
observations enter the average - ``n`` reports the raw count, so
on partially unidentified fits (which warn at fit time) ``n``
exceeds the averaged support.

``df`` is the kit's ``survey_df_final`` snapshot - the exact value
the STORED overall ``safe_inference`` received (on a replicate fit
Expand Down
Loading
Loading