Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions METHODOLOGY_REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ A **Complete** entry has a documented review pass against the primary academic s

The catalog grew incrementally over several quarters, so formats vary across the existing Complete entries; the consistent invariant is that someone walked through the implementation against the academic source and captured the result here. New reviews going forward should aim for the fuller structure (Verified Components + Corrections Made + Deviations + dedicated methodology test file) used by the more recent entries.

**In Progress** entries have a REGISTRY.md section and unit-test coverage but no formal walk-through captured here yet, carrying a "Documentation in place" / "Outstanding for promotion" pair until promoted. **As of 2026-06-27 no In Progress rows remain** — every estimator, diagnostic, and cross-cutting inference feature has been reviewed to Complete (Survey Data Support was the last, promoted 2026-06-27). The band description is retained for surfaces that enter the tracker later.
**In Progress** entries have a REGISTRY.md section and unit-test coverage but no formal walk-through captured here yet, carrying a "Documentation in place" / "Outstanding for promotion" pair until promoted. The earlier consolidation completed on 2026-06-27. DurationDiD now enters as **In Progress** pending a recorded primary-source implementation walk-through.

**Not Started** entries have neither a tracker walk-through nor an REGISTRY.md section. This tracker no longer carries any Not Started rows; new estimators are expected to enter as In Progress when their REGISTRY entry lands.

Expand All @@ -39,6 +39,7 @@ The catalog grew incrementally over several quarters, so formats vary across the
| DifferenceInDifferences | `estimators.py` | `fixest::feols()` | **Complete** | 2026-01-24 |
| MultiPeriodDiD | `estimators.py` | `fixest::feols()` | **Complete** | 2026-02-02 |
| TwoWayFixedEffects | `twfe.py` | `fixest::feols()` | **Complete** | 2026-02-08 |
| DurationDiD | `duration_did.py` | Deaner–Ku paper (no executed software parity) | **In Progress** | — |

### Staggered Treatment Estimators

Expand Down Expand Up @@ -93,13 +94,19 @@ The catalog grew incrementally over several quarters, so formats vary across the

**Status legend** (matches the contract in [§ What "Complete" means in this tracker](#what-complete-means-in-this-tracker) above):
- **Not Started**: No REGISTRY.md entry yet. Reserved for future surfaces; this tracker currently carries no Not Started rows.
- **In Progress**: REGISTRY.md entry and unit-test coverage exist, but no formal walk-through has been captured in this document yet (a "Documentation in place" / "Outstanding for promotion" pair tracks the gap). No rows currently carry this status — the tracker is fully Complete as of 2026-06-27.
- **In Progress**: REGISTRY.md entry and unit-test coverage exist, but no formal walk-through has been captured in this document yet (a "Documentation in place" / "Outstanding for promotion" pair tracks the gap). DurationDiD currently carries this status.
- **Complete**: A documented review pass against the primary academic source is captured here (minimum: Corrections Made, Deviations or `(None)`, and Verified Components / Edge Cases Verified / R Comparison Results in some form).

---

## Detailed Review Notes

### DurationDiD (Core DiD) — In Progress

**Documentation in place:** REGISTRY.md DurationDiD entry, dated Deaner–Ku source review, API and executed tutorial; `tests/test_methodology_duration_did.py` checks paper-numbered identities and both bootstrap algorithms.

**Outstanding for promotion:** Capture a formal primary-source-to-implementation walk-through, verified components, corrections and deviations in this tracker. Unit tests and workflow approval alone do not establish Complete. No executed author-software parity or application replication is claimed.

### Core DiD Estimators

#### DifferenceInDifferences
Expand Down Expand Up @@ -1514,7 +1521,7 @@ more graceful handling of edge cases while still signaling invalid inference to

### Priority Order (updated 2026-06-27)

**No In Progress entries remain.** **Survey Data Support** was promoted to Complete on 2026-06-27 — the last consolidation-pass row (PlaceboTests was promoted 2026-06-26). The methodology-review tracker is now Complete across all core/staggered/continuous/triple-difference/synthetic estimators, diagnostics, and cross-cutting inference features.
**DurationDiD remains In Progress.** **Survey Data Support** was promoted to Complete on 2026-06-27 — the last consolidation-pass row (PlaceboTests was promoted 2026-06-26). Those consolidation-pass entries remain Complete; the newly added DurationDiD entry awaits its documented promotion evidence.

- Going forward, a new surface enters as **In Progress** when its REGISTRY.md entry lands and is promoted via a documented review pass (primary-source fidelity walk → dedicated methodology test file with paper-equation-numbered Verified Components → R-parity / deviation documentation), per the contract in [§ What "Complete" means in this tracker](#what-complete-means-in-this-tracker).

Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,19 @@ For rigorous DiD analysis, follow these 8 steps. Skipping diagnostic steps produ
2. **State identification assumptions** - which parallel trends variant (unconditional, conditional, PT-GT-Nev, PT-GT-NYT), no-anticipation, overlap.
3. **Test parallel trends** - simple 2x2: `check_parallel_trends()`, `equivalence_test_trends()`; staggered: inspect CS event-study pre-period coefficients (generic PT tests are invalid for staggered designs). Insignificant pre-trends do NOT prove PT holds.
4. **Choose estimator** - staggered adoption -> CS/SA/BJS (NOT plain TWFE); few treated units -> SDiD; factor confounding -> TROP; simple 2x2 -> DiD. Run `BaconDecomposition` to diagnose TWFE bias.
5. **Estimate** - `estimator.fit(data, ...)`. Always print the cluster count first and choose inference method based on the result (cluster-robust if >= 50 clusters, wild bootstrap if fewer - for DifferenceInDifferences pass `cluster=`; TwoWayFixedEffects auto-clusters at unit level).
5. **Estimate** - `estimator.fit(data, ...)`. For estimators supporting cluster inference, print the cluster count first and choose inference method based on the result (cluster-robust if >= 50 clusters, wild bootstrap if fewer - for DifferenceInDifferences pass `cluster=`; TwoWayFixedEffects auto-clusters at unit level).
6. **Sensitivity analysis** - `compute_honest_did(results)` for bounds under PT violations (MultiPeriodDiD, CS, or dCDH natively; the TwoWayFixedEffects `event_study=True` surface and a StackedDiD `results.aggregate('event_study')` container also admit - Stacked needs `kappa_pre >= 2`), `run_all_placebo_tests()` for 2x2 falsification, specification comparisons for staggered designs.
7. **Heterogeneity** - CS: `results.aggregate('group')`/`'event_study'` (post-fit, no refit); SA: `results.event_study_effects` / `to_dataframe(level='cohort')`; Stacked: `results.aggregate('event_study')`/`'simple'` post-fit views (surface always computed since 3.9); EDiD: `results.aggregate(...)` post-fit from retained EIFs (3.9); ImputationDiD/TwoStageDiD: `results.aggregate(...)` post-fit from panel-backed kits (3.9); ContinuousDiD: `results.aggregate('dose'/'simple'/'event_study')` post-fit (3.9; dose/simple are views, event_study recomputes); subgroup re-estimation.
8. **Robustness** - compare 2-3 estimators (CS vs SA vs BJS), report with and without covariates (shows whether conditioning drives identification), present pre-trends and sensitivity bounds.
8. **Robustness** - compare 2-3 estimators (CS vs SA vs BJS), report with and without covariates where supported (shows whether conditioning drives identification), present pre-trends and sensitivity bounds.

For DurationDiD, use hazard assumptions and the stored `pretrend_test()`, pooled individual bootstrap, and comparisons of CD/PH or calibration dates. Covariates and higher-level clustering are unsupported.

Full guide: `diff_diff.get_llm_guide("practitioner")`.

## Estimators

- [DurationDiD](https://diff-diff.readthedocs.io/en/stable/api/duration_did.html) - Deaner & Ku (2026) cumulative absorption ATT under common dynamics or proportional hazards; balanced individual panels, common timing, pooled bootstrap and stored hazard pretest

- [DifferenceInDifferences](https://diff-diff.readthedocs.io/en/stable/api/estimators.html) - basic 2x2 DiD with robust/cluster-robust SEs, wild bootstrap, formula interface, and fixed effects
- [TwoWayFixedEffects](https://diff-diff.readthedocs.io/en/stable/api/estimators.html) - panel data DiD with unit and time fixed effects via within-transformation or dummies
- [MultiPeriodDiD](https://diff-diff.readthedocs.io/en/stable/api/estimators.html) - event study design with period-specific treatment effects for dynamic analysis (deprecated 3.9 - use TwoWayFixedEffects `event_study=True`)
Expand Down
11 changes: 5 additions & 6 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ Forward-looking plan for diff-diff, organized as queued work, candidates under c

## Shipping Next

DurationDiD now provides the two-group common-timing core for absorbing individual
outcomes, with CD/PH hazards, pooled bootstrap and native pretests. Covariate,
staggered, censoring, survey and clustering extensions remain outside this release;
see [the API](docs/api/duration_did.rst) and the dated methodology review.

Queued work, ordered by expected leverage. Each item is its own PR. Ordering is priority-sequenced, not time-committed.

### 4.0 API unification
Expand Down Expand Up @@ -81,12 +86,6 @@ Long-running program, framed as "building toward" rather than with discrete ship

Frontier methods that may graduate to Under Consideration given time and research signals.

### Causal Duration Analysis with DiD

Extends DiD to duration / survival outcomes where standard methods fail (hazard rates, time-to-event). Duration analogue of parallel trends; avoids distributional and hazard-function assumptions.

**Reference**: Deaner & Ku (2025), *AEA Conference Paper*.

### CATT Meta-Learner for Heterogeneous Effects

ML-powered conditional ATT, using a doubly robust meta-learner to discover which units benefit most from treatment.
Expand Down
9 changes: 9 additions & 0 deletions changelog.d/20260906-duration-did.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
### Added
- **Duration DiD**: `DurationDiD` estimates cumulative absorption effects in balanced
individual panels with common treatment timing under common dynamics or proportional
untreated hazards. Includes pooled individual-bootstrap pointwise/simultaneous inference,
a stored fixed-anchor hazard pretest, owned results and event-study aggregation, native
reporting, and an executed tutorial. Unsupported domains and failed bootstrap families
retain explicit availability metadata; no survey, covariate or cluster extensions.
Constructor parameters are revalidated on every fit, including after direct attribute
updates; invalid configurations raise without replacing a previous fitted result.
5 changes: 5 additions & 0 deletions diff_diff/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,8 @@
)
from diff_diff.dml_did import DMLDiD
from diff_diff.dml_did_results import DMLDiDResults
from diff_diff.duration_did import DurationDiD
from diff_diff.duration_did_results import DurationDiDPretestResults, DurationDiDResults
from diff_diff.efficient_did import (
EDiDBootstrapResults,
EfficientDiD,
Expand Down Expand Up @@ -468,6 +470,9 @@ def __getattr__(name: str) -> _Any:
"LWDiD",
"LWDiDResults",
# DMLDiD (Chang 2020 double/debiased ML DiD)
"DurationDiD",
"DurationDiDResults",
"DurationDiDPretestResults",
"DMLDiD",
"DMLDiDResults",
"SieveLearner",
Expand Down
48 changes: 47 additions & 1 deletion diff_diff/_reporting_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@

Current contents:

- ``_duration_native_diagnostics(results)`` — extracts the stored DurationDiD
hazard payload for DiagnosticReport emission and BusinessReport input validation.
- ``describe_target_parameter(results)`` — returns the
``target_parameter`` block documenting what scalar the headline
represents. Introduced for BR/DR gap #6 (target-parameter
Expand All @@ -21,7 +23,42 @@

from typing import Any, Dict

from diff_diff.results_base import Diagnostic
from diff_diff.results_base import Diagnostic, _coverage_pct


def _duration_native_diagnostics(results: Any) -> Dict[str, Any]:
"""Extract DurationDiD's stored native payload without recomputing inference."""
return {
"status": "ran",
"estimator": "DurationDiD",
"method": results.method,
"pretrend_test": results.pretrend_test().to_dict(),
"estimation_status": results.estimation_status,
"inference_status": dict(results.inference_status),
"inference_reasons": {k: list(v) for k, v in results.inference_reasons.items()},
"support_warnings": list(results.support_warnings),
"n_bootstrap_valid": results.n_bootstrap_valid,
}


def _duration_hazard_sentence(native: Dict[str, Any]) -> str:
"""Describe the stored hazard decision consistently across BR and DR."""
if native.get("estimator") != "DurationDiD" or native.get("status") != "ran":
return ""
diagnostic = native.get("pretrend_test") or {}
level = _coverage_pct(diagnostic["alpha"])
if diagnostic.get("status") != "available" or diagnostic.get("reject") is None:
reasons = "; ".join(diagnostic.get("reasons", [])) or "no usable test statistic"
return (
f"Hazard pretest unavailable at the {level}% simultaneous confidence level: "
f"{reasons}. No rejection decision is available."
)
decision = "rejects" if diagnostic["reject"] else "does not reject"
return (
f"The fixed-anchor hazard pretest {decision} at the {level}% simultaneous "
f"confidence level (p={diagnostic['p_value']:.4g}). "
"Non-rejection does not establish identification or adequate power."
)


def describe_target_parameter(results: Any) -> Dict[str, Any]:
Expand Down Expand Up @@ -78,6 +115,15 @@ def describe_target_parameter(results: Any) -> Dict[str, Any]:
"""
name = type(results).__name__

if name == "DurationDiDResults":
return {
"name": "Mean cumulative absorption ATT",
"definition": "Uniform average across declared post-periods of cumulative absorption effects on the whole treated population, including baseline-absorbed individuals. Positive effects increase absorption.",
"aggregation": "uniform_post_periods",
"headline_attribute": "att",
"reference": "Deaner and Ku (2026); REGISTRY.md DurationDiD",
}

if name == "DiDResults":
# Covers both ``DifferenceInDifferences`` (2x2 DiD) and
# ``TwoWayFixedEffects`` (TWFE with unit + time FE). Both
Expand Down
Loading
Loading