Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ Full guide: `diff_diff.get_llm_guide("practitioner")`.
- [Manipulation Testing](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html) - Cattaneo, Jansson & Ma (2020) density-discontinuity test (`RDDensityTest`): rddensity 3.0 parity, robust bias-corrected inference, unrestricted/restricted models, mass-point adjustment
- [Parallel Trends Testing](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html) - simple and Wasserstein-robust parallel trends tests, equivalence testing (TOST)
- [Placebo Tests](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html) - placebo timing, group, permutation, leave-one-out
- [TWFE Weight Diagnostics](https://diff-diff.readthedocs.io/en/stable/api/twfe_weights.html) - Baker et al. (2025) implicit weights a TWFE regression places on each ATT(g,t), against the ATT^O / ATT^simple targets, with the pre-trend contribution. Ports Callaway's `twfeweights` (MIT)
- [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html) - Rambachan & Roth (2023) sensitivity analysis: robust CI under PT violations, breakdown values
- [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html) - Roth (2022) minimum detectable violation and power curves
- [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html) - analytical and simulation-based MDE, sample size, power curves for study design
Expand Down
507 changes: 507 additions & 0 deletions benchmarks/R/generate_twfeweights_golden.R

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion benchmarks/R/requirements.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ required_packages <- c(
"nprobust", # Calonico-Cattaneo-Farrell local-linear (DIDHAD dependency)
"Synth", # Abadie-Diamond-Hainmueller (2010) synthetic control (SyntheticControl R-parity; ships data(basque))
"qte", # Callaway qte package (Athey-Imbens CiC + QDiD R-parity; ships data(lalonde))
"BMisc", # Callaway utility package (twfeweights dependency: weighted_ecdf, orig2t)
"DRDID", # Sant'Anna & Zhao (2020) doubly-robust DiD (twfeweights AIPW dependency)

# Utilities
"jsonlite", # JSON output for Python interop
Expand All @@ -27,7 +29,9 @@ required_packages <- c(

# synthdid must be installed from GitHub
github_packages <- list(
synthdid = "synth-inference/synthdid"
synthdid = "synth-inference/synthdid",
# TWFE weight diagnostics parity goldens (not on CRAN)
twfeweights = "bcallaway11/twfeweights"
)

install_if_missing <- function(pkg) {
Expand Down
586 changes: 586 additions & 0 deletions benchmarks/data/twfeweights_golden.json

Large diffs are not rendered by default.

1,501 changes: 1,501 additions & 0 deletions benchmarks/data/twfeweights_sim_panel.csv

Large diffs are not rendered by default.

1,501 changes: 1,501 additions & 0 deletions benchmarks/data/twfeweights_unbalanced_panel.csv

Large diffs are not rendered by default.

20 changes: 20 additions & 0 deletions changelog.d/20260831-twfe-weight-diagnostics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
### Added
- **TWFE weight diagnostics** (port of Brantly Callaway's `twfeweights` R
package, MIT): what a two-way fixed effects regression *implicitly* weights
on staggered-adoption data.
- `attgt_weights(results, aggregation="twfe"|"overall"|"simple")` reports the
weight a TWFE regression, ATT^O, or ATT^simple places on each ATT(g,t),
plus post-period negative-weight counts. Returns `ATTGTWeightsResult`.
- `decompose_twfe_weights(data, ..., method="fwl")` re-derives the estimate
from its ATT(g,t) building blocks and returns `TWFEDecompositionResult`
with `pretrend_bias` - the contribution of pre-treatment cells, i.e. of
parallel-trends violations rather than of treatment - and, with
`balance_covariates=`, implicit-weight covariate balance.
`plot_twfe_weights()` renders either view (matplotlib or plotly).
- Validation: rejects NaN / `-inf` cohort labels, covariate-adjusted fits
under `aggregation="twfe"`, duplicated or non-finite ATT(g,t) cells, an
incomplete group-time grid, and invalid sampling weights. Two structural
gaps are handled as R does instead of raising: a cohort with no estimable
post cell is dropped (`did`'s first-period drop), and under
`control_group="not_yet_treated"` the CS estimands average over each
cohort's available post periods (`aggte`).
17 changes: 17 additions & 0 deletions diff_diff/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,14 @@
TROPResults,
trop,
)
from diff_diff.twfe_weights import (
attgt_weights,
decompose_twfe_weights,
)
from diff_diff.twfe_weights_results import (
ATTGTWeightsResult,
TWFEDecompositionResult,
)
from diff_diff.two_stage import (
TwoStageBootstrapResults,
TwoStageDiD,
Expand All @@ -321,6 +329,7 @@
plot_sensitivity,
plot_staircase,
plot_synth_weights,
plot_twfe_weights,
)
from diff_diff.wooldridge import WooldridgeDiD
from diff_diff.wooldridge_results import WooldridgeDiDResults
Expand Down Expand Up @@ -457,6 +466,13 @@ def __getattr__(name: str) -> _Any:
"TWFEWeightsResult",
"chaisemartin_dhaultfoeuille",
"twowayfeweights",
# TWFE weight diagnostics (Callaway `twfeweights` port) - distinct from
# the dCDH `twowayfeweights` surface above: these weight ATT(g,t)
# parameters, not (unit, time) cells.
"ATTGTWeightsResult",
"TWFEDecompositionResult",
"attgt_weights",
"decompose_twfe_weights",
# WooldridgeDiD (ETWFE)
"WooldridgeDiD",
"WooldridgeDiDResults",
Expand All @@ -473,6 +489,7 @@ def __getattr__(name: str) -> _Any:
"SieveLearner",
# Visualization
"plot_bacon",
"plot_twfe_weights",
"plot_event_study",
"plot_group_effects",
"plot_sensitivity",
Expand Down
1 change: 1 addition & 0 deletions diff_diff/dml_did.py
Original file line number Diff line number Diff line change
Expand Up @@ -2411,6 +2411,7 @@ def fit(
# stays admitted).
is_survey_fit=survey_metadata is not None,
bootstrap_results=bootstrap_results,
covariates=covariates,
)
self.results_ = results
self.is_fitted_ = True
Expand Down
99 changes: 99 additions & 0 deletions diff_diff/guides/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1503,6 +1503,78 @@ results.print_summary()
plot_bacon(results)
```

### TWFE Weight Diagnostics

What a TWFE regression implicitly weights on staggered data. Distinct from
`twowayfeweights` (dCDH), which weights (unit, time) cells: these weight
ATT(g,t) parameters. Ported from Brantly Callaway's `twfeweights` R package
(MIT); methodology Baker, Callaway, Cunningham, Goodman-Bacon & Sant'Anna
(2025).

```python
attgt_weights(
results, # CallawaySantAnnaResults, or a (g,t) frame
aggregation="twfe", # "twfe" | "overall" (ATT^O) | "simple"
data=None, unit=None, time=None, first_treat=None, # frame path only
weights=None, # unit-level sampling weights
) -> ATTGTWeightsResult

decompose_twfe_weights(
data, # balanced long panel (it re-estimates)
outcome=, unit=, time=, first_treat=,
method="fwl",
covariates=None,
base_period="first_period", # or "gmin1"
balance_covariates=None, # enables result.covariate_balance()
weights=None,
) -> TWFEDecompositionResult

plot_twfe_weights(result, kind="auto") # "weights" | "balance"
```

`aggregation="twfe"` requires a fit with `base_period="universal"`,
`control_group="never_treated"` AND no covariates (R twfe_weights' three
restrictions); it raises otherwise. ATT^O and ATT^simple weights are
non-negative and sum to one, so comparing `implied_att` across the three
aggregations shows what the TWFE specification costs.

Both entry points fail closed on input R never faced: NaN / `-inf` cohort
labels (never-treated is exactly `0` or `+inf`), duplicated or non-finite
ATT(g,t) cells, an incomplete group-time grid (`"twfe"` needs every cohort x
period cell, the CS estimands every post cell), and sampling weights that are
not finite, non-negative and positive-mass. Two structural gaps mirror R
rather than raising, each with a `UserWarning`: a cohort with no estimable
post cell is dropped from the table and the cohort shares (`did`'s
first-period drop), and under `control_group="not_yet_treated"` the cells CS
marks `zero_treated_control` are treated as structurally absent, so
`"overall"`/`"simple"` average over each cohort's AVAILABLE post periods
(`aggte`). `n_negative_post` / `negative_post_weight_share` report the
pathology (negative weight on POST cells); `n_negative` counts pre cells too,
and is near-half in every staggered design because the TWFE weights sum to
zero over the full grid.

### plot_twfe_weights

```python
plot_twfe_weights(
results, # ATTGTWeightsResult | TWFEDecompositionResult
kind="auto", # "weights" | "balance" ("auto" picks balance
# when the result carries a balance table)
standardize=True, absolute_value=True, # balance view
annotate=False, ax=None, show=True,
backend="matplotlib", # or "plotly"
)
```

`kind="weights"` scatters weight against ATT(g,t), coloured by pre/post - points
left of the vertical zero line carry negative weight. `kind="balance"` scatters
unweighted against implicitly-weighted covariate differences; points near the
horizontal axis are covariates the implicit weights balance.

`decompose_twfe_weights` takes the raw panel rather than a fitted result
because it re-estimates. It is tied to `attgt_weights` by an identity:
`attgt_weights(cs, aggregation="twfe").implied_att == decompose_twfe_weights(panel, ...).estimate`.

### StaggeredTripleDifference

DEPRECATED in 3.9, removed in 4.0 (ledger row M-013). Use
Expand Down Expand Up @@ -1967,6 +2039,33 @@ Returned by `BaconDecomposition.fit()` (and the deprecated `bacon_decompose()` w

**Methods:** `summary()`, `print_summary()`, `to_dataframe()`

### ATTGTWeightsResult

Diagnostic result from `attgt_weights`. No inference quintet - the
decomposition is an algebraic identity.

- `weights`: DataFrame with `group`, `time`, `post`, `weight`, `att`
- `implied_att`: `sum(weight * att)` - the TWFE coefficient when
`aggregation="twfe"`
- `n_negative`, `negative_weight_share`: the staggered-TWFE pathology
- `aggregation`, `source`, `control_group`, `base_period`, `n_cells`
- `summary()`, `to_dataframe()`, `to_dict()`

### TWFEDecompositionResult

Diagnostic result from `decompose_twfe_weights`.

- `cells`: DataFrame with `group`, `time`, `post`, `att`, `weight`, `ess`,
`remainder`
- `estimate` == `decomposition` + `remainder`
- `pretrend_bias`: contribution of PRE-treatment cells, i.e. of
parallel-trends violations rather than of treatment
- `post_only`, `effective_sample_size`, `covariates`, `base_period`
- `covariate_balance(level="summary"|"cell", standardize=True,
post_only=True)`: implicit-weight covariate balance; raises when
`balance_covariates=` was not requested
- `summary()`, `to_dataframe()`, `to_dict()`

### Comparison2x2

Individual 2x2 DiD comparison (used in BaconDecompositionResults).
Expand Down
1 change: 1 addition & 0 deletions diff_diff/guides/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ The site is organized into 5 sections, each with a landing page:
- [Manipulation Testing](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html): Cattaneo, Jansson & Ma (2020) density-discontinuity manipulation test (`RDDensityTest`), parity with R rddensity 3.0 - boundary-adaptive local polynomial density estimation at the cutoff, robust bias-corrected inference, unrestricted/restricted models, jackknife/plugin variances, data-driven bandwidths, mass-point adjustment
- [Parallel Trends Testing](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html): Simple and Wasserstein-robust parallel trends tests, equivalence testing (TOST)
- [Placebo Tests](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html): Placebo timing, group, permutation, and leave-one-out diagnostics
- [TWFE Weight Diagnostics](https://diff-diff.readthedocs.io/en/stable/api/twfe_weights.html): Baker et al. (2025) implicit weights on ATT(g,t) - `attgt_weights(results, aggregation='twfe'|'overall'|'simple')` takes a fitted `CallawaySantAnnaResults` (raw ATT(g,t) frame + panel as fallback) and returns the weight each estimand places on each group-time effect, with the negative-weight share; `decompose_twfe_weights(data, outcome=, unit=, time=, first_treat=, method='fwl', covariates=)` re-derives the TWFE estimate from its ATT(g,t) building blocks with `pretrend_bias`, and `result.covariate_balance()` reports implicit-weight covariate balance. Plot with `plot_twfe_weights`. R `twfeweights` 0.9.0 output parity
- [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html): Rambachan & Roth (2023) sensitivity analysis — robust CI under parallel trends violations, breakdown values
- [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html): Roth (2022) Section II.A-B no-individually-significant (NIS) box-probability pretest power + minimum detectable violation; `pretest_form='nis'` (default) implements the paper's primary form, `pretest_form='wald'` retained as paper-supported alternative (Propositions 1+3+4 all apply); linear-violation MDV in Roth's γ units when relative-time labels are threaded through `fit()`; full Σ_22 routing on non-bootstrap CallawaySantAnna and SunAbraham adapters and on admitted CS-/StackedDiD-sourced `aggregate('event_study')` containers (StackedDiD persists its ES VCV in every inference mode)
- [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html): Analytical and simulation-based power analysis — MDE, sample size, power curves for study design
Expand Down
9 changes: 8 additions & 1 deletion diff_diff/staggered.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import bisect
import warnings
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Sequence, Tuple

import numpy as np
import pandas as pd
Expand Down Expand Up @@ -3056,6 +3056,7 @@ def fit(
group_time_effects,
is_survey_fit=survey_metadata is not None,
bootstrap_results=bootstrap_results,
covariates=covariates,
)

self.is_fitted_ = True
Expand Down Expand Up @@ -5151,6 +5152,7 @@ def _build_aggregation_kit(
*,
is_survey_fit: bool = False,
bootstrap_results: Optional["CSBootstrapResults"] = None,
covariates: Optional[Sequence[str]] = None,
) -> Optional["AggregationKit"]:
"""Distil the fit-time state post-fit re-aggregation needs.

Expand Down Expand Up @@ -5185,6 +5187,11 @@ def _build_aggregation_kit(
# (or DDD) survey fit does not warn as "CallawaySantAnna" on post-fit
# aggregate(). Legacy kits without the key default at the read site.
bookkeeping["bootstrap_label"] = getattr(estimator, "_BOOTSTRAP_LABEL", "CallawaySantAnna")
# Covariate usage, recorded so downstream diagnostics can refuse designs
# their formulas do not cover (``attgt_weights(aggregation="twfe")``
# mirrors R twfe_weights' ``xformla == ~1`` restriction). Column NAMES
# only - never values - so the data-minimization contract holds.
bookkeeping["covariates"] = tuple(covariates or ())

# Data minimization: the results object is picklable and users share
# result artifacts, so the kit must not turn it into a carrier for raw
Expand Down
Loading