Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,5 +102,4 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
| Clean-negative precision re-run: Campaign 1's `s3_negative` plans were base_sha-contaminated (real defects at the pinned base), so hallucination rate cleared (dual 3.4%) but trivia-flooding on a *genuinely* clean plan is unmeasured. Small future run (~30 s3-style reviews on 3-5 constructed-clean plans, not the full 120-matrix). | `tools/plan-review-eval/` | campaign-1 verdict | Mid | Low |
| `trop-silent-drop` criteria regression: the rewritten `criteria.md` dropped a silent-failure catch the OLD criteria had (arm A caught it, B/C missed — campaign-1 A-vs-B contrast). Patch the criteria + re-validate (re-opens the criteria identity, so gated behind a re-validation run). | `.claude/skills/plan-review/criteria.md`, `tools/plan-review-eval/` | campaign-1 verdict | Mid | Medium |
| The Stata `jwdid` parity arms cover BALANCED panels only. `jwdid_alltreated` pins the all-eventually-treated cell set, `N` and SE ratio on the `mpdta` subset (191 units, 955 rows, 764 estimated), but every arm is a balanced frame, so REGISTRY's parity claim is scoped to that case. The comparison-support predicate's UNBALANCED behavior -- where `G_max` may be unobserved at later periods so the closed form `t < G_max - anticipation` does not hold and support is set by whichever cohorts are actually observed -- is verified against the predicate directly, not against Stata. Add an unbalanced `jwdid` arm to close that gap. | `benchmarks/stata/generate_etwfe_cs_golden.do`, `tests/test_etwfe_cs_stata_parity.py` | #729-followup | Mid | Low |
| Bad-controls tutorial section for `docs/tutorials/32_dml_did.ipynb` (PR-C): the `fit(bad_control=)` workflow, the `ATT_X(g,t)` pre-test reading, and the Approach-1 comparison via CS base-period covariates. | `docs/tutorials/32_dml_did.ipynb` | bad-controls PR-B | Quick | Medium |
| DurationDiD tutorial notebook (Appendix C-style simulation showing the sign reversal of standard DiD on an absorbing outcome, CD vs PH, the fitting window, and the pretest), registered in `docs/tutorials/index.rst` with a drift test; deferred from the estimator PR, which relies on the executed API-page examples | `docs/tutorials/` | #822 | Mid | Low |
19 changes: 19 additions & 0 deletions changelog.d/20260907-bad-controls-tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
### Documentation
- **Tutorial 33, "Bad Controls - Covariates That Treatment Can Affect"**
(`docs/tutorials/33_bad_controls.ipynb`; PR-C of the Caetano, Callaway, Payne &
Sant'Anna 2026 bad-controls initiative). On a staggered version of the paper's DGP 1
it shows the naive TWFE regression with the bad control at `t` missing by the full
treatment effect on the covariate, Approach 1 through base-period covariates with and
without the confounders `W`, the `DMLDiD` bad-control lane and the choice of `W`
(Remark 5's lagged outcome), how to read `bad_control_summary()` (pre-period rows
pre-test MP-5/MP-8 and should be zero; post-period rows check that treatment moves the
covariate), the event study, a ridge refit through the split-half nested stage, and
the lane's restrictions. Registered in the tutorials index, `docs/tutorials/README.md`,
`diff_diff/guides/llms.txt` (together with a line for tutorial 32) and
`docs/doc-deps.yaml`; pinned by `tests/test_t33_bad_controls_drift.py` (code-cell
hashes, quoted numbers, DGP re-derivation, pre-period narrative guard).

### Changed
- **`practitioner_next_steps()` names the bad-control lane**: on a `DMLDiD` fit with
`bad_control` set the guidance banner reads "DMLDiD (CCPS 2026 bad-control score)"
instead of the Chang (2020) label, matching the results `summary()` banner.
2 changes: 2 additions & 0 deletions diff_diff/guides/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ The site is organized into 5 sections, each with a landing page:
- [16 Wooldridge ETWFE](https://diff-diff.readthedocs.io/en/stable/tutorials/16_wooldridge_etwfe.html): Wooldridge (2023, 2025) ETWFE — saturated OLS, logit/Poisson (ASF-based ATT), aggregation types
- [22 HAD Survey-Weighted Workflow](https://diff-diff.readthedocs.io/en/stable/tutorials/22_had_survey_design.html): HeterogeneousAdoptionDiD + did_had_pretest_workflow under SurveyDesign(strata, psu, weights, fpc) — BRFSS-shape panel, modest SE inflation explanation, Phase 4.5 C0 QUG-deferred verdict
- [26 Composition Drift & Calibration](https://diff-diff.readthedocs.io/en/stable/tutorials/26_composition_drift_calibration.html): When differential non-response biases the DiD itself — per-state raking with Meta's balance package, `balance.interop.diff_diff` adapter, raking-granularity lesson (requires `pip install balance`)
- [32 Double ML DiD](https://diff-diff.readthedocs.io/en/stable/tutorials/32_dml_did.html): Chang (2020) DMLDiD — cross-fitted ML nuisances on a staggered panel with nonlinear confounding, learner comparison, cross-fitting diagnostics, event study + HonestDiD, the repeated-cross-section lane with a survey design
- [33 Bad Controls](https://diff-diff.readthedocs.io/en/stable/tutorials/33_bad_controls.html): Caetano, Callaway, Payne & Sant'Anna (2026) bad controls — the include-at-t bias, Approach 1 via base-period covariates (with and without W), the DMLDiD bad-control lane (`fit(bad_control=, bad_control_covariates=)`), reading `bad_control_summary()` (pre rows pre-test MP-5/MP-8, post rows check treatment moves the covariate), ridge refit via the split-half nested stage

## Survey Support

Expand Down
11 changes: 9 additions & 2 deletions diff_diff/practitioner.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,15 +76,22 @@ def _estimator_display(type_name: str, results: Any) -> str:

``ChangesInChangesResults`` is shared by CiC and QDiD (``QDiDResults``
is an alias), so the static per-type map cannot distinguish them; the
``method`` field ("cic"/"qdid") does. Defensive: mock results may
lack the field, in which case the static entry is the fallback.
``method`` field ("cic"/"qdid") does. ``DMLDiDResults`` likewise
serves two lanes: a fit with ``bad_control`` set runs the Caetano,
Callaway, Payne & Sant'Anna (2026) bad-control score and is named for
it, the plain fit keeps the Chang (2020) label. Defensive: mock results
may lack the fields, in which case the static entry is the fallback.
"""
if type_name == "ChangesInChangesResults":
kind = _distributional_kind(results)
if kind == "cic":
return "ChangesInChanges (CiC)"
if kind == "qdid":
return "QDiD"
if type_name == "DMLDiDResults" and getattr(results, "bad_control", None) is not None:
# The bad-control lane runs the Caetano, Callaway, Payne & Sant'Anna
# (2026) score, not Chang's; the results banner says the same.
return "DMLDiD (CCPS 2026 bad-control score)"
return _ESTIMATOR_NAMES.get(type_name, type_name)


Expand Down
21 changes: 21 additions & 0 deletions docs/doc-deps.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,9 @@ sources:
- path: docs/tutorials/16_survey_did.ipynb
type: tutorial
note: "CallawaySantAnna survey examples"
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial
note: "Approach 1 (pre-treatment bad control read at the cell's base period) with and without W"
- path: README.md
section: "Estimators (one-line catalog entry)"
type: user_guide
Expand Down Expand Up @@ -1059,6 +1062,8 @@ sources:
type: user_guide
- path: docs/tutorials/32_dml_did.ipynb
type: tutorial
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial

diff_diff/dml_did_results.py:
drift_risk: low
Expand All @@ -1074,6 +1079,8 @@ sources:
type: methodology
- path: docs/tutorials/32_dml_did.ipynb
type: tutorial
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial

# ── TROP (trop group) ──────────────────────────────────────────────

Expand Down Expand Up @@ -1387,6 +1394,9 @@ sources:
- path: docs/methodology/variance-conventions.md
type: methodology
note: "Clustered-variance conventions inventory (CR1 k + tail-df map, D/L classification); table generated by tests/test_variance_conventions.py"
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial
note: "LinearRegression within-regression benchmark with cluster_k_adjustment (absorbed-FE CR1 convention)"
- path: docs/methodology/REGISTRY.md
section: "Variance Estimation, Cluster-Robust SE"
type: methodology
Expand Down Expand Up @@ -1419,6 +1429,9 @@ sources:
- path: docs/methodology/variance-conventions.md
type: methodology
note: "Clustered-variance conventions inventory (CR1 k + tail-df map, D/L classification); table generated by tests/test_variance_conventions.py"
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial
note: "within_transform used for the TWFE benchmark"
- path: docs/methodology/REGISTRY.md
section: "Inference, safe_inference NaN gating"
type: methodology
Expand Down Expand Up @@ -1515,6 +1528,8 @@ sources:
diff_diff/_crossfit.py:
drift_risk: low
docs:
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial
- path: docs/methodology/REGISTRY.md
section: "Cross-fitting, DR-score, and ridge infrastructure (DML)"
type: methodology
Expand All @@ -1532,10 +1547,14 @@ sources:
note: "Duck-typed learner protocol (RegressorLearner/ClassifierLearner Protocols, validate_learner, _validate_predictions) + native learners (LinearLearner/RidgeLearner/LogitLearner/SieveLearner) wrapping linalg solvers. Contracts documented in REGISTRY: raw-X-no-intercept input, fit-reset semantics (documented limitation for stateful user learners), identified-columns prediction under rank deficiency."
- path: docs/tutorials/32_dml_did.ipynb
type: tutorial
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial

diff_diff/_dr_scores.py:
drift_risk: low
docs:
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial
- path: docs/methodology/REGISTRY.md
section: "Cross-fitting, DR-score, and ridge infrastructure (DML)"
type: methodology
Expand Down Expand Up @@ -1599,6 +1618,8 @@ sources:
diff_diff/practitioner.py:
drift_risk: low
docs:
- path: docs/tutorials/33_bad_controls.ipynb
type: tutorial
- path: diff_diff/guides/llms-practitioner.txt
type: user_guide
- path: diff_diff/guides/llms-full.txt
Expand Down
Loading