Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,99 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- **Event-study container consumers + dCDH post-fit `aggregate()`** (v4
program 2(b) PR-1; ledger row [M-026] flips to shimmed, rows [M-092] /
[M-093] amended).
- `compute_honest_did`, `compute_pretrends_power` and
`plot_event_study` now accept the unified `EventStudyResults`
container from `CallawaySantAnnaResults.aggregate('event_study')`
directly, and `plot_honest_event_study` renders an
`HonestDiDResults` whose `original_results` is such a container -
the post-fit route no longer dead-ends at the consumers. HonestDiD
bounds are identical across routes (route-parity gated at 1e-14);
PreTrendsPower's extraction is bit-exact (its Monte-Carlo power
carries scipy's inherent MVN-CDF jitter either way). Admission in
HonestDiD/PreTrendsPower is SOURCE-SCOPED to CS-produced containers:
dCDH l1 containers are rejected BY DESIGN (their placebo semantics
need HonestDiD's native dCDH branch), other producers arrive with
their own `aggregate()` migrations; the plotters take no source
guard. Inputs whose coefficients mix reference normalizations fail
closed in HonestDiD and PreTrendsPower on BOTH routes - a container
with multiple marked reference rows, and any universal-base fit
whose cohorts carry more than one positional-base event time (the
`reference_event_times` guard below).
- `EventStudyResults` gains four optional provenance fields (declared
last - positional compatibility preserved): `base_period`,
`anticipation`, `df_survey` (the fit's resolved scalar inference
df; `0.0` = replicate design with undefined df, which the per-row
`df` column cannot encode), and `reference_event_times` (the
common-reference provenance described below). Threaded by the
builders, including the
requested-but-empty path; `to_dict()` gains the four keys. The
`df_survey` typing widened `Optional[int]` -> `Optional[float]`
through the HonestDiD call path.
- `ChaisemartinDHaultfoeuille.fit(aggregate=)` is DEPRECATED
([M-026]): it never computed anything (every non-None value raised
"reserved for Phase 3"), so supplying it warns and a non-None value
raises `ValueError` pointing at the new post-fit route.
`ChaisemartinDHaultfoeuilleResults.aggregate()` ships as a pure
VIEW: `'event_study'` returns the unified container (Phase-1 fits
the 2-row l=1 view), `'simple'` a one-row `AggregationResult`
relaying the overall estimand bit-exactly with estimand-aware
`target`/`n`/`n_kind` (DID_M/switcher-cells, DID_1/groups,
delta/no-scalar-count) and a `df` column resolved from the ACTUAL
inference path (the `L_max>=2` delta keeps its analytical survey df
even under bootstrap, per the REGISTRY delta-SE note). Because
nothing is recomputed, bootstrap fits are permitted - unlike
CallawaySantAnna's kit-based `aggregate()`, which stays fail-closed.
- Consumer hardening that landed with the wave: `plot_event_study`
renders MULTI-reference containers via row-aligned hollow anchors
(never dropped, never filled estimates; explicit `reference_period=`
re-basing of such a surface around a non-anchor fails closed), and
warns when a container's stored intervals sit at a different level
than the requested `alpha` (stored bootstrap/t intervals cannot be
re-leveled from the SE) - `plot_honest_event_study` names the same
mismatch against the HonestDiD level. HonestDiD's MPD and dCDH
branches now drop `se == 0` rows exactly like the CS/container
branches (undefined inference is never laundered into finite
bounds). PreTrendsPower warns on CS `base_period='varying'` input on
BOTH routes: its `linear` violation benchmark assumes level
coefficients against a common reference, while varying-base
pre-treatment effects are consecutive-period comparisons (REGISTRY
PreTrendsPower note; violation-vector transformation tracked in
TODO.md).
- Common-reference guard: CS `base_period='universal'` fits (and
their containers) record `reference_event_times` - the distinct
per-cohort positional-base event times. On gapped time grids the
bases land at different event times, and a cohort's base can
OVERLAP another cohort's estimated horizon where no reference-only
row marks it; HonestDiD and PreTrendsPower now FAIL CLOSED on both
input routes instead of silently returning bounds/power over
coefficients normalized against different bases (REGISTRY HonestDiD
common-reference-guard note). Regular (ungapped) grids carry a
single entry and are unaffected; varying-base fits carry None.
- HonestDiD's no-reference (varying-base) pre/post split is now
anticipation-aware on both routes: with `anticipation=k` the window
`[e=-k, -1]` carries anticipated treatment effects (the REGISTRY
contract PreTrendsPower already applied), so the clean pre-trend
set is `e < -k` and `beta_post` starts at `-k` - previously the
split at 0 misclassified anticipated effects as pre-trend
coefficients. The pretrends varying-base warning is scoped to the
`linear` violation benchmark it concerns.
- PreTrendsPower's `linear` violation weights are REFERENCE-ANCHORED
on the CS-universal, SunAbraham and container routes (`|t - t_ref|`;
Roth's violation is normalized like the coefficients and vanishes at
the omitted period - MPD already anchored via its reference helper):
raw treatment-relative labels overstated each pre-period's
hypothesized violation by the reference offset and correspondingly
understated the γ-unit MDV. Universal-base CS/SA linear
pretrends power and MDV numbers CHANGE to the Roth-correct values
(e.g. weights `[3, 2]` -> `[2, 1]` for pre `[-3, -2]` around
`t_ref = -1`); varying-base input keeps raw labels under its
existing warning (REGISTRY linear-convention note records the
correction).

### Deprecated
- **Semantic rename wave** (v4 program 2(c)-ii PR-B, ledger rows [M-030]
[M-031] [M-043] [M-044] [M-045]..[M-047] [M-084] [M-086] [M-087] [M-095]
Expand Down
2 changes: 1 addition & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ Related tracking surfaces:

| Issue | Location | Origin | Effort | Priority |
|-------|----------|--------|--------|----------|
| PreTrendsPower `violation='linear'` on CS `base_period='varying'` input targets the wrong alternative: `δ_pre = M · \|t\|` assumes level coefficients against a common reference, but varying-base pre-treatment effects are consecutive-period comparisons (constant increments under a linear trend). Both CS-sourced routes now WARN (REGISTRY PreTrendsPower Note), and universal-base GAPPED grids fail closed via the `reference_event_times` common-reference guard; what remains is the varying-base resolution - either transforming the violation vector through each coefficient's actual base mapping (needs per-horizon base provenance) or requiring `base_period='universal'` for the linear benchmark - a per-estimator methodology decision with a hand-calculated linear-violation gate | `diff_diff/pretrends.py` | 2(b) PR-1 R5 | Mid | Medium |
| `run_placebo_test`/`run_all_placebo_tests` expose ONE overloaded `time` param (calendar for timing/group tests, 0/1 post dummy forwarded into the M-137/M-138 callees - works only because a two-period 0/1 calendar column is both); redesign the wrapper signatures (dual `time=`/`post=` params or a split) so each callee gets an honestly-named argument | `diff_diff/diagnostics.py` | 2(c)-ii PR-B | Mid | Medium |
| `absorbed_fe_rank` N>=3 general rank: the helper keeps `sum(levels-1)` for 3+ absorbed dims, exact for independent connected dims but an over-count for duplicated/nested triples (measured `a(5),b(4),c==b(4)`: true 7 vs formula 10) and for disconnected N-way graphs. Two-way is component-exact. Deriving general N-way FE rank is a hypergraph problem; do it with a reference (fixest's `fixef.rm`/reghdfe df_a) rather than a guess. See `docs/methodology/variance-conventions.md` D3. | `diff_diff/utils.py` | #variance-inventory | Mid | Low |
| `SyntheticControl` conformal (CWZ 2021) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies — the residual-permutation shortcut is only valid for time-permutation-invariant proxies (SC/Lasso/DiD); an AR proxy needs innovation permutation. | `diff_diff/conformal.py`, `diff_diff/synthetic_control_results.py` | CWZ-2021 | Heavy | Low |
| Make the post-fit `results.aggregate("event_study")` container consumable downstream. `EventStudyResults` is rejected by all THREE consumers that read a CS event study — `compute_honest_did` (`honest_did.py`, dispatches on `CallawaySantAnnaResults` and raises `TypeError`), `compute_pretrends_power` (`pretrends.py`, same), and `plot_event_study` (`visualization`, same) — so `fit(aggregate="event_study")` is still the only route for them and their error messages say so explicitly. Needs an `EventStudyResults` branch in each extraction path (consuming `event_time` / `is_reference` / `vcov` / `vcov_index` / per-row `df`) PLUS `base_period` and `anticipation` provenance, which the unified container does not carry and HonestDiD needs for its universal-base-period warning and pre-period classification. Gate with end-to-end tests: `compute_honest_did(res.aggregate("event_study"))` at `base_period="universal"`, and `compute_pretrends_power(...)` at `anticipation=1`. | `diff_diff/honest_did.py`, `diff_diff/pretrends.py`, `diff_diff/results_base.py` | #726 | Mid | Medium |
| Re-run the R-dependent benchmark refresh so `docs/benchmarks.rst`'s TWFE "SE Rel Diff 0.1%" cell reflects the 3.9 K_reference convergence (expected 0.1% -> 0.0%; the table is generated, never hand-edited — the movement is noted in the CHANGELOG entry). | `docs/benchmarks.rst`, `benchmarks/R/` | #variance-inventory | Quick | Low |
| `SunAbraham`: a cohort not observed at its own reference relative period (`e = -1 - anticipation`) makes that cohort's block collinear, so QR drops an unnamed column (`dropping 1 of 12 columns (column 9)`) and `overall_att` comes back **NaN**. Found by auditing the sibling estimator while fixing the ETWFE analogue (#724); PRE-EXISTING, not introduced there. Lower severity than #724 — that returned a silently WRONG finite number, this returns NaN with a rank warning — but the event-study surface still looks complete, so a user may not notice the loss. SA already omits its reference explicitly and tracks `_reference_observed`, so the fix is per-cohort support for that flag rather than the ETWFE-style redesign. | `diff_diff/sun_abraham.py` | #724-audit | Mid | Low |
| Define `N_g` (W2025 Eqs. 7.4/7.6) for UNBALANCED panels where comparison-support filtering removes every observation of some units in an estimated cohort, then replace the fail-closed guard with the defined behavior. `_n_g_per_cohort` is read off the final sample, so those units vanish from the cohort-share weights; measured on a cohort supplied with 100 units of which 90 appear only at a dropped period, `aggregate(weights="cohort_share")` moves 1.8078 -> 3.8157. The paper assumes a balanced panel and does not say whether `N_g` counts the supplied cohort or the surviving units, and the two disagree materially, so `aggregate` currently raises naming the cohorts and counts ([M-125]); `weights="cell"` is unaffected and balanced panels never trip it. Settle the estimand (likely: count the supplied cohort, since ATT(g,t) is a cohort-level quantity, but that weights units with no retained observation) and gate with a test computing Eq. 7.4 by hand on unequal cohort sizes. | `diff_diff/wooldridge_results.py`, `diff_diff/wooldridge.py` | #729-followup | Mid | Medium |
Expand Down
32 changes: 22 additions & 10 deletions diff_diff/aggregation.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"""

from dataclasses import dataclass, field
from typing import Any, Dict, Optional, Tuple
from typing import Any, ClassVar, Dict, Optional, Tuple

import numpy as np
import pandas as pd
Expand Down Expand Up @@ -134,9 +134,13 @@ class AggregationResult(BaseResults):
aligned estimands over the same labels (ContinuousDiD's ATT(d) and
ACRT(d) become 2N rows). ``"att"`` where an estimator has one.
att, se, t_stat, p_value : np.ndarray
The canonical quintet, per row. On a bootstrapped fit ``t_stat`` /
``p_value`` / the interval are the producer's percentile-bootstrap
statistics carried through unchanged - NOT recomputed analytically.
The canonical quintet, per row, carrying WHATEVER inference the fit
stored - never recomputed. On a bootstrapped fit that usually means
the producer's percentile-bootstrap statistics carried through
unchanged; view-relay producers can mix regimes per row where the
fit itself did (dCDH's ``L_max >= 2`` cost-benefit delta keeps
analytical ``safe_inference`` even under ``n_bootstrap > 0`` - see
the REGISTRY Phase 2 cost-benefit delta SE note).
conf_int_lower, conf_int_upper : np.ndarray
Interval bounds at the fit's ``alpha``.
n : np.ndarray
Expand All @@ -156,8 +160,10 @@ class AggregationResult(BaseResults):
one would be a fabricated number.
df : np.ndarray
Per-row inference degrees of freedom, NaN where none governed the
stored p-value. Entirely NaN on bootstrap fits, whose percentile
inference uses no df.
stored p-value. NaN on percentile-bootstrap rows (no df governs
them); a bootstrapped fit's rows can still carry a finite df where
the fit kept analytical inference for that row (the dCDH delta
case above).
alpha : float
Significance level the interval was computed at.
estimator : str or None
Expand Down Expand Up @@ -460,12 +466,14 @@ class AggregationMixin:

#: Aggregation types this results class implements. A subset of
#: :data:`AGGREGATION_VOCABULARY` plus any documented per-estimator extra.
_AGGREGATE_SUPPORTED: Tuple[str, ...] = ()
#: ClassVar so that dataclass results classes overriding these hooks with
#: an annotation do not grow a spurious ``__init__`` field.
_AGGREGATE_SUPPORTED: ClassVar[Tuple[str, ...]] = ()

#: Types for which ``balance_e`` is meaningful. CallawaySantAnna threads it
#: only through event-study aggregation, so accepting it elsewhere would
#: silently ignore a user's argument.
_AGGREGATE_BALANCE_E_TYPES: Tuple[str, ...] = ("event_study",)
_AGGREGATE_BALANCE_E_TYPES: ClassVar[Tuple[str, ...]] = ("event_study",)

def aggregate(
self,
Expand Down Expand Up @@ -505,10 +513,14 @@ def aggregate(
)
raise ValueError(f"Unsupported aggregation type {type!r}. Supported: {known}.{extra}")
if balance_e is not None and type not in self._AGGREGATE_BALANCE_E_TYPES:
usable = ", ".join(repr(t) for t in self._AGGREGATE_BALANCE_E_TYPES)
if self._AGGREGATE_BALANCE_E_TYPES:
usable = ", ".join(repr(t) for t in self._AGGREGATE_BALANCE_E_TYPES)
applies = f"It applies to: {usable}."
else:
applies = "It applies to no aggregation type on this estimator."
raise ValueError(
f"balance_e is not used by aggregate(type={type!r}) and would be "
f"silently ignored. It applies to: {usable}."
f"silently ignored. {applies}"
)
self._aggregate_validate_weights(weights)
return self._aggregate_compute(type, weights=weights, balance_e=balance_e)
Expand Down
Loading