Skip to content

Add DurationDiD for absorbing individual outcomes - #826

Closed
igerber wants to merge 3 commits into
mainfrom
codex/eval-estimator-4535c327
Closed

Add DurationDiD for absorbing individual outcomes#826
igerber wants to merge 3 commits into
mainfrom
codex/eval-estimator-4535c327

Conversation

@igerber

@igerber igerber commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Adds DurationDiD for absorbing binary outcomes in complete individual panels with two groups and common treatment timing. It estimates cumulative absorption effects under common-dynamics or proportional untreated-hazard restrictions. The headline uniformly averages declared post-period ATTs over the whole treated population, including individuals already absorbed at baseline; positive effects mean increased absorption.

Includes configurable pre-period calibration, pooled individual-bootstrap pointwise and simultaneous inference, DurationDiDResults, and the stored DurationDiDPretestResults hazard diagnostic. Results support simple/event-study aggregation and owned, JSON-safe exports. All four constructor parameters are validated on construction, through set_params() and before every fit. Invalid direct updates raise a parameter-specific error and preserve any previous fitted result; valid updates apply on the next fit. Summaries print distinct inference-unavailability reasons once while retaining the full per-family metadata. DiagnosticReport and BusinessReport show the stored hazard verdict, fitted confidence level and availability reasons. BusinessReport validates supplied native diagnostics. API documentation, packaged guides and an executed tutorial accompany the estimator.

Methodology references (required if estimator / math changes)

  • Method name(s): Duration DiD, with method="common_dynamics" or method="proportional_hazards".
  • Paper / source link(s): Deaner and Ku (2026), Causal Duration Analysis with Diff-in-Diff, arXiv:2405.05220v2, especially Theorem 1, equations 2.16 and 3.2–3.4, and Appendix B Algorithms 1–2. Implementation conventions are documented in docs/methodology/REGISTRY.md under DurationDiD.
  • Any intentional deviations from the source (and why): The PH coefficient resolves the ambiguity in printed equation 3.5 using the mean of ratios between treated and control cumulative-hazard increments from baseline, consistent with Theorem 1. Library policies specify uniform headline weights, frozen positive calibration support, empirical centered-bootstrap quantiles and tail probabilities, zero-SE handling, and survivor/exit support warnings. P-values are finite-bootstrap companions to the paper's bands. The stored pretest compares interior pre-period hazard contrasts with the last untreated anchor; it does not establish identification or adequate power through non-rejection.

Material limitations

  • Requires a balanced, equally spaced individual panel, absorbing outcomes and fixed group membership. Covariates, staggered adoption, censoring/dropout, repeated cross-sections, survey/sampling weights and higher-level clustering are unsupported. Bootstrap inference assumes independent individuals.
  • Counterfactual probability and monotonicity checks include the factual last-pre boundary. Invalid curves retain raw extrapolations and suppress canonical causal output. Any failed bootstrap draw suppresses inference for its affected family; effect and diagnostic families are assessed independently. These conservative policies can make estimates or inference unavailable under sampling noise even when population hazard restrictions hold.
  • No executed author-software parity or empirical application replication is claimed.

Validation

  • Tests added/updated: 180 DurationDiD estimator, methodology, results and reporting tests cover formula/independent-bootstrap oracles, panel and calibration domains, failed draws, invalid curves, inference, serialization, aggregation and native reporting. Fifty parameter-mutation cases cover invalid values before and after fitting, state preservation and valid-update equivalence to fresh construction. Shared API, naming, diagnostic, reporting and documentation checks include the new estimator.
  • Affected Python suites: 1,534 passed, 45 skipped across 15 suites. The four DurationDiD suites also passed with the Rust backend selected: 180 passed. These counts overlap.
  • Backtest / simulation / notebook evidence: docs/tutorials/33_duration_did.ipynb executed successfully; subsequent validation passed without modifying the saved notebook. Its six executed code cells and two saved plots were inspected. Synthetic examples exercise both hazard methods.
  • Sphinx HTML with warnings treated as errors: passed, zero warnings. Ruff, Black, changelog validation and diff whitespace checks passed. Changed source also passed Python 3.9 grammar checks.
  • mypy --python-version 3.12 diff_diff: passed, 110 source files. Local default configured mypy was blocked by installed NumPy stubs using Python 3.12 type syntax while the configuration targets Python 3.10; project dependency constraints and type-check configuration are unchanged.

Security / privacy

  • Confirm no secrets/PII in this PR: no credentials or private individual records added. Tests and the tutorial use synthetic panels.

Changelog

  • Fragment added: changelog.d/20260906-duration-did.md.

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

Overall assessment: ✅ Looks good

Executive summary

  • DurationDiD CD/PH estimators, hazard pretest, and bootstrap inference align with the cited source audit.
  • Numerical and policy deviations are explicitly documented in Registry **Note:** entries.
  • Panel, domain, empty-support, failed-bootstrap, and NaN-inference paths fail safely.
  • One minor summary-rendering issue exists; it does not affect estimates.
  • A changelog fragment and extensive targeted tests are included.

Methodology

  • Severity: P3 — informational, mitigated
    Impact: PH mean-of-ratios, fixed-anchor diagnostics, uniform headline weighting, centered-bootstrap p-values, and conservative failure policies differ from some printed/reference conventions but are explicitly documented. Implementation matches those conventions at diff_diff/duration_did.py:L38-L170 and diff_diff/duration_did.py:L343-L496.
    Concrete fix: None required. See docs/methodology/REGISTRY.md:L50-L112.

No undocumented methodology mismatch, missing assumption guard, or incorrect SE/variance calculation found.

Code Quality

  • Severity: P3
    Impact: When several inference families are unavailable for the same reason, summary() repeats the identical reason once per family, producing noisy output without changing statistics (diff_diff/duration_did_results.py:L264-L276).
    Concrete fix: Deduplicate reasons or prefix each with its family name before rendering.

Performance

No findings.

Maintainability

No findings. Constructor parameters propagate through results, bootstrap, aggregation, serialization, and reporting.

Tech Debt

  • Severity: P3 — informational
    Impact: The formal methodology tracker intentionally remains “In Progress”; this is disclosed and does not undermine the completed source audit (METHODOLOGY_REVIEW.md:L104-L108).
    Concrete fix: None required for approval; record the formal implementation walk-through when promoting the entry.

Security

No findings or exposed secrets identified.

Documentation/Tests

No findings. The PR includes methodology tests, edge-case coverage, reporting/serialization tests, an executed tutorial, and changelog.d/20260906-duration-did.md:L1-L7.

Test execution was unavailable in this review environment because its Python test dependencies are not installed.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 5fb2266aee71bdeb4c91a0a474dfa4d44edc96e4


Overall assessment: ✅ Looks good

Executive summary

  • No unmitigated P0/P1 findings.
  • CD/PH estimation, pooled bootstrap, diagnostics, and NaN gates match the Registry and source audit.
  • Documented numerical and policy deviations are informational, not defects.
  • The previous duplicated-summary-reasons finding is resolved.
  • A changelog fragment and focused methodology/edge-case tests are present.

Methodology

  • Severity: P3 — informational, mitigated
    Impact: PH mean-of-ratios, normalized time intervals, fixed-anchor diagnostics, centered-bootstrap inference, and conservative failure handling are intentional Registry conventions, faithfully implemented at diff_diff/duration_did.py:L38-L170 and diff_diff/duration_did.py:L343-L496.
    Concrete fix: None. See docs/methodology/REGISTRY.md:L50-L112 and docs/methodology/papers/deaner-ku-2026-review.md:L281-L463.

No undocumented mismatch, missing assumption guard, or incorrect variance/SE calculation found.

Code Quality

  • Severity: P3 — resolved
    Impact: The previous review’s duplicated inference-reason output has been fixed using ordered deduplication at diff_diff/duration_did_results.py:L256-L277.
    Concrete fix: None.

Performance

No findings.

Maintainability

No findings. Constructor configuration propagates through fitting, bootstrap inference, results, aggregation, serialization, and reporting.

Tech Debt

  • Severity: P3 — informational
    Impact: The formal implementation walkthrough remains explicitly marked “In Progress” at METHODOLOGY_REVIEW.md:L104-L108; this does not indicate a correctness defect.
    Concrete fix: Record the formal walkthrough when promoting the tracker entry.

Security

No findings or exposed secrets identified in the changed feature files.

Documentation/Tests

No findings. The PR includes the required changelog fragment at changelog.d/20260906-duration-did.md:L1-L7 and targeted methodology, failure, zero-SE, serialization, aggregation, and reporting tests.

Tests could not be executed in this review environment because pytest, NumPy, and pandas are unavailable; changed Python files passed AST parsing.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: d9eacb55f96006c1b53dabb5e1b8e4a6b4073e59


Overall assessment: ✅ Looks good

Executive summary

  • No unmitigated P0/P1 findings.
  • CD/PH equations, pooled bootstrap, diagnostics, and NaN gates match the Registry and Deaner–Ku source audit.
  • Documented implementation conventions are not defects.
  • The prior duplicated-summary-reason issue is resolved.
  • Changelog, methodology tests, and edge-case coverage are present.

Methodology

  • Severity: P3 — informational, mitigated
    Impact: PH mean-of-ratios, normalized intervals, fixed-anchor diagnostics, centered-bootstrap inference, and conservative draw-failure handling are documented conventions and correctly implemented at diff_diff/duration_did.py:L38-L170 and diff_diff/duration_did.py:L354-L505.
    Concrete fix: None. See docs/methodology/REGISTRY.md:L50-L112 and docs/methodology/papers/deaner-ku-2026-review.md:L281-L478.

No incorrect estimator formula, variance/SE calculation, control composition, or missing assumption guard found.

Code Quality

  • Severity: P3 — resolved
    Impact: Inference-unavailability reasons are now deduplicated in first-appearance order at diff_diff/duration_did_results.py:L256-L277, with regression coverage at tests/test_duration_did_results.py:L96-L106.
    Concrete fix: None.

Performance

No findings. The O(BnT) whole-history bootstrap follows the documented algorithm.

Maintainability

No findings. All constructor parameters propagate through fitting, inference, results, serialization, aggregation, and reporting.

Tech Debt

  • Severity: P3 — informational
    Impact: The formal implementation walkthrough remains explicitly tracked as “In Progress” at METHODOLOGY_REVIEW.md:L104-L108; this is not a correctness defect.
    Concrete fix: Record the walkthrough when promoting the tracker entry.

Security

No findings or exposed secrets identified in the changed files.

Documentation/Tests

No findings. The required fragment exists at changelog.d/20260906-duration-did.md:L1-L9, with focused methodology, zero-SE, failed-draw, panel-validation, serialization, aggregation, and reporting tests.

Runtime tests could not execute because NumPy, pandas, SciPy, and pytest are unavailable; all 20 changed Python files and the notebook JSON parsed successfully.

@igerber igerber closed this Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant