feat: results.aggregate('total') - estimator-owned totals for CS/EDiD/Imputation/TwoStage + MMM no-scale admission

[igerber]

Summary

• Promote "total" into AGGREGATION_VOCABULARY: results.aggregate('total') is the estimator-owned total incremental outcome on CallawaySantAnna, EfficientDiD, ImputationDiD, and TwoStageDiD - a single target="total" row, the exact relay C x overall CONDITIONAL on the realized aggregation mass (att/se/CI scaled by C; t/p/df inherited; bootstrap-safe RELAY with a NaN df column per the M-027 per-level policy).
• Per-estimator mass C (finite-masked complete-case support): CS replays _aggregate_simple's cell selection over a new immutable fit-time kit snapshot (agg_gt_cells + is_survey_fit); ImputationDiD uses the finite-tau support (total_support, snapshot at kit build), so C x overall = sum(tau) exactly - fixing the documented raw-support overcount of the MMM scale="auto" route for total exports; TwoStageDiD uses the post-filter treatment-indicator support from its private kit frame copy; EfficientDiD uses the integer sum of kept cells' n_treated from its deep-copied kit snapshot.
• Scope is PANEL, NON-SURVEY fits, fail-closed with the reason elsewhere: repeated-cross-section routings (RC-first gate order), fits declaring a survey_design= (any weight type; gates read immutable kit provenance, so post-fit mutation of survey_metadata cannot bypass them; explicit unweighted SurveyDesign(psu=) fits fail closed by declaration), the CS bare-cluster= divergent corner (cohort-mass weighting vs complete-case count), and - on CS/ImputationDiD, whose masses need the new fit-time snapshots - pre-upgrade pickled kits.
• NaN contract: no-support or true scaling overflow blanks the whole row; otherwise the relay is verbatim and inherited NaNs pass through (att/n never blanked - the repo's non-estimable-row convention).
• MMM admission: both exporters accept a total container ALONE (any scale rejected as double-counting); level gate/hints/docstrings/errors swept; for overall-total exports aggregate('total') supersedes scale="auto", which remains the group-container route.
• Docs: v4-design section 6 amendment + v4-deprecations M-020..M-027/M-117..M-120 note updates; REGISTRY CS total Note (a)-(g) + EDiD/Imputation/TwoStage subclauses + MMM Notes; DEFERRED row split (StackedDiD remainder Parked; survey/RC remainder incl. the att*dC variance term moved to Paper-gated); survey-roadmap Current Limitations entry; migration-4.0/r_comparison/troubleshooting/CONTRIBUTING/api-rst/guides/practitioner/CHANGELOG/tutorial sweeps.

Methodology references (required if estimator / math changes)

• Method name(s): Post-fit aggregation totals over Callaway & Sant'Anna (2021), Chen, Sant'Anna & Xie (2025) EfficientDiD, Borusyak, Jaravel & Spiess (2024) imputation, Gardner (2022) two-stage.
• Paper / source link(s): The total is a documented library extension, not a paper-defined estimand - each source estimator's point estimates, weights, and variance are unchanged; the new surface is the positive fixed-mass transformation C x overall conditional on the realized aggregation mass. See the REGISTRY post-fit aggregate('total') Notes for each estimator.
• Any intentional deviations from the source (and why): The relay estimand is conditional on the realized aggregation mass (the random-mass att*dC variance term is deliberately out of scope - tracked in DEFERRED.md with the survey/RC remainder); documented with reviewer-recognized - **Note (...)** labels in REGISTRY.md for all four estimators and the MMM section.

Validation

• Tests added/updated: tests/test_aggregate_contract.py (new TestTotal* batteries: relay identity with independent frame-derived mass oracles per estimator; fail-closed routing arms incl. analytic/replicate/unweighted-psu/fweight survey designs, combined survey+RC gate-order pin, balanced-panel-flag admission; gate-immutability mutation arms on all four; snapshot-isolation arms; zero-support and degenerate-inference pass-through arms; per-field overflow blanking; legacy-kit refit arms; bootstrap relays on all four; anticipation-mass pins for CS and EDiD; TwoStage post-filter divergence; pickle; constant-tau estimand check; design-sweep total arms + _assert_level_matches branch; vocabulary/n_kind/balance_e enrollment; Wooldridge custom-aggregate arm), tests/test_mmm.py (TestTotalContainerAdmission + hint pins + end-to-end fit->aggregate('total')->export).
• Backtest / simulation / notebook evidence (if applicable): MC coverage for the total CI is deliberately omitted with a recorded rationale - C*lo <= tau*C <= C*hi iff lo <= tau <= hi, so coverage is mathematically identical to the overall CI's, the estimator's already-validated surface. Tutorials 02/15 markdown enumerations updated (no executed cells touched).

Security / privacy

• Confirm no secrets/PII in this PR: Yes

[github-actions]

Overall assessment

⚠️ Needs changes — one unmitigated P1 semantic-contract issue.

Executive summary

• The four total estimands match their documented Registry definitions.
• Conditional realized-mass inference is explicitly documented; it is not a defect.
• MMM exporters do not verify that a total container came from a supported estimator.
• Survey/RC and StackedDiD limitations are properly tracked.
• Runtime tests could not run because the environment lacks pytest/numpy; changed Python files parse successfully.

Methodology

• P3 — Documented conditional-mass variance
  • Impact: Total inference conditions on realized C and omits the att*dC term.
  • Fix: None. This is explicitly documented in docs/methodology/REGISTRY.md:L1056 and tracked in DEFERRED.md:L39.
  • CS, EfficientDiD, ImputationDiD, and TwoStageDiD mass definitions agree with their existing overall-ATT support and weighting. No unmitigated methodology mismatch found.

Code Quality

• P1 — MMM total admission does not verify estimator provenance (diff_diff/mmm.py:L305-L340)
  • Impact: Any hand-built or altered AggregationResult with level="total" and target="total" is accepted, including estimator=None, unknown estimators, or StackedDiD. Its att is silently treated as already scaled, bypassing the documented four-estimator allowlist and StackedDiD’s staged-out restriction. This can produce incorrectly scaled MMM inputs.
  • Concrete fix: Require provenance in {CallawaySantAnna, EfficientDiD, ImputationDiD, TwoStageDiD} before accepting a total container. Also validate the full producer contract (label="total", n_kind="obs", weight=1) and add rejection tests for StackedDiD, unknown, and missing provenance.

Performance

No findings.

Maintainability

No additional findings.

Tech Debt

• P3 — Tracked limitations
  • Impact: Survey/RC totals and StackedDiD totals remain unavailable.
  • Concrete fix: None required for this PR; both are tracked in DEFERRED.md:L39 and DEFERRED.md:L76.

Security

No findings or apparent secrets.

Documentation/Tests

• P2 — Stale tutorial output and comment (docs/tutorials/02_staggered_did.ipynb:L1133, L1155)

  • Impact: The stored warning and nearby code comment still say only aggregate('simple') relays bootstrap inference, contradicting the new total behavior.
  • Concrete fix: Re-execute the cell or update its stored output and comment to mention aggregate('total').

• P2 — Missing anticipation tests for two adopters (tests/test_aggregate_contract.py:L4320-L4509)

  • Impact: ImputationDiD and TwoStageDiD derive mass from anticipation-adjusted masks, but only CS and EfficientDiD have explicit anticipation-mass tests.
  • Concrete fix: Add anticipation=1 tests asserting total.n includes the anticipation-window finite support for both estimators.

Path to Approval

1. Gate MMM total containers by the four supported estimator provenances and add rejection tests for StackedDiD, unknown, and missing provenance.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: 407a3ae2acb06a86c739eaae5bc06d8c9280d192

---

Overall assessment

✅ Looks good — no unmitigated P0/P1 findings.

Executive summary

• The prior MMM provenance P1 is resolved with an estimator allowlist and producer-contract validation.
• All four total-mass definitions match their underlying overall estimands and cited methodology.
• Conditional realized-mass inference is explicitly documented and therefore informational only.
• Empty support, NaN/overflow, anticipation, bootstrap, survey, and legacy-kit paths are covered.
• Runtime tests were unavailable because pytest/numpy are not installed; all 13 changed Python files parsed successfully.

Methodology

• Severity: P3 — Documented conditional-mass variance
  • Impact: Total inference omits the random-mass att*dC term and conditions on realized C.
  • Concrete fix: None. This is explicitly documented in docs/methodology/REGISTRY.md:L1056 and tracked in DEFERRED.md:L39.
  • CS/EDiD cell weighting, Imputation finite-τ support, and TwoStage treated-observation averaging agree with their source-method definitions.

Code Quality

• Severity: P3 — Prior provenance finding resolved (diff_diff/mmm.py:L99, diff_diff/mmm.py:L310)
  • Impact: Missing, unknown, and StackedDiD provenance now fails closed; label, target, row count, n_kind, and weight contracts are validated.
  • Concrete fix: None.

Performance

No findings.

Maintainability

No findings. The shared relay builder avoids four separate inference implementations.

Tech Debt

• Severity: P3 — Tracked limitations
  • Impact: Survey/RC totals and StackedDiD totals remain unavailable.
  • Concrete fix: None for this PR; tracked in DEFERRED.md:L39 and DEFERRED.md:L76.

Security

No findings or apparent secrets.

Documentation/Tests

No unmitigated findings. The prior stale tutorial text is updated, and anticipation tests now cover all four adopters (tests/test_aggregate_contract.py:L4183, L4369, L4489, L4632). Provenance rejection tests are present at tests/test_mmm.py:L1740.