diff --git a/TODO.md b/TODO.md index 96e8d23b9..b0c7e859a 100644 --- a/TODO.md +++ b/TODO.md @@ -22,7 +22,6 @@ Related tracking surfaces: | Issue | Location | Origin | Effort | Priority | |-------|----------|--------|--------|----------| | Consolidate the remaining estimator-entangled DR/logit score variants (`staggered.py::_doubly_robust` + RC twins, `triple_diff.py`, `lwdid.py`, `wooldridge.py`) onto the shared `_dr_scores.py` module, each migration with its own committed oracle capture (the ContinuousDiD lift's two-tier pattern in `tests/test_dr_scores.py`); and add a ridge vcov path to `solve_ridge` if an estimator ever needs analytical ridge inference | `diff_diff/_dr_scores.py` | dml-b0 | Mid | Low | -| hc2/hc2_bm floor `1 - h_ii` at 1e-10 in the shared leverage meat, fabricating finite (if inflated) variances for leverage-one observations - hc3 now fails closed there (LWDiD fix wave) but the pre-existing hc2 family behavior is released surface; decide fail-closed vs keep-floor for hc2/hc2_bm | `diff_diff/linalg.py` | #588 | Quick | Low | | Numeric between-period cohorts (e.g. `first_treat=4.5` with integer times) are rejected by LWDiD while CallawaySantAnna estimates them and LWDiD's own datetime/Period cohorts map to the next observed period — close the dtype asymmetry by adopting the next-observed-period mapping for numeric cohorts too (contract documented in REGISTRY cohort-encodings Note + `docs/api/lwdid.rst` Input Contract). Lands only after PR #588 merges | `diff_diff/lwdid.py` | #588 | Quick | Low | | Implement the LW 2026 eq. 7.9/7.10 unit-average cohort estimand (regress per-unit post-average transformed outcomes on `[1, D_g]` vs never-treated) as an alternative to the documented cell-mass `cohort_effects` convention (REGISTRY within-cohort aggregation Note; the two differ on unbalanced panels, where cell-mass weights units by observed post periods). Needs the 7.10 regression + its covariance on the NT path. Lands only after PR #588 merges | `diff_diff/lwdid_staggered.py` | #588 | Quick | Low | | Expose cell-mass overall ATT (Stata `Post_avg` convention; = CS-simple on balanced panels) as an aggregate extra on LWDiD results — the fit's `.att` is the paper's `tau_omega` (cohort-mean-then-treated-weight, eq. 7.18); the authors' large-N display uses cell-mass weighting instead, and both are legitimate estimands (see the REGISTRY LWDiD Aggregation note). Lands only after PR #588 merges | `diff_diff/lwdid_results.py` | #588 | Quick | Low | diff --git a/changelog.d/20260906-hc2-leverage-one-fail-closed.md b/changelog.d/20260906-hc2-leverage-one-fail-closed.md new file mode 100644 index 000000000..3651ba73c --- /dev/null +++ b/changelog.d/20260906-hc2-leverage-one-fail-closed.md @@ -0,0 +1,51 @@ +### Fixed +- **`hc2` / `hc2_bm` no longer report an understated finite variance at a + leverage-one observation**: the shared one-way leverage meat floored + `1 - h_ii` at 1e-10, which did not inflate the perfectly-fitted row's + term but silently dropped its outcome noise (probe `[1, D]` with a single + treated unit: HC2 SE 0.19 against the exact classical 0.47), and + `hc2_bm` paired that finite SE with a NaN Satterthwaite DOF. Both now + fail closed exactly like `hc3` (see Behavioral Changes). The Rust HC2 + kernel mirrors the change (no floor; it signals and the Python dispatcher + re-dispatches to the NumPy branch, which emits the single warning). The + sibling floor in the one-way Bell-McCaffrey DOF helper is removed: at + leverage one every contrast's DOF is NaN because the HC2 variance it + belongs to is undefined; below leverage one nothing changes. +- **Zero-count `fweight` rows no longer trip the HC3 leverage guard or the + HC1 fallback**: under frequency weights the leverage of a zero-count row + is the unweighted quadratic form against the weighted bread and is + unbounded, so an inert row could NaN the `hc3` vcov or push `hc2` into an + HC1 fallback that broke expansion parity. Zero-count rows are excluded + from the guard and from the meat, so compressed HC2/HC3 equal the literal + `np.repeat` expansion at any leverage. +- `LWDiD` no longer emits a second, differently worded "HC2 variance is + undefined" warning per cell on leverage-one designs; the shared kernel's + warning is the only one (behavior otherwise unchanged: LWDiD already + failed closed there). + +### Behavioral Changes +- **Leverage-one designs under `vcov_type="hc2"` (every weight type) and + under unweighted, unclustered `vcov_type="hc2_bm"` now return an all-NaN + covariance and DOF vector with a `UserWarning`** ("HC2 variance is + undefined: N observation(s) have hat-matrix leverage ~1 ...") whenever a + positive-weight row has `h_ii >= 1 - 1e-8`, matching the released `hc3` + contract and R `sandwich::vcovHC` (NaN at hat values ~1). Point + estimates are unchanged; `se`, `t_stat`, `p_value` and confidence + intervals are NaN. The former warn-and-fall-back-to-HC1 branch for + over-one leverage is retired (that case is inside the new guard). + Affected surfaces: `DifferenceInDifferences`, `MultiPeriodDiD` and + `LinearRegression` on both families; `TwoWayFixedEffects` on explicit + `hc2` and on `hc2_bm` in event-study `spec="pooled"` without `unit=`; + `SunAbraham` and `WooldridgeDiD` (OLS) full-dummy fits under one-way + `hc2` whenever the design has a singleton cohort x period cell. Weighted + (pweight) `hc2_bm` — including a no-op `weights=np.ones(n)` — keeps the + clubSandwich singleton-CR2 generalized-inverse result and stays finite, + as does clustered CR2; this asymmetry is a documented deviation from + `clubSandwich` for the unweighted case (maintainer decision, 2026-09). + Remedy: `vcov_type="classical"` (exact inference under homoskedastic + normal errors; needs positive residual df) or add observations to the + perfectly-fitted cell. `hc1` is deliberately not offered as a remedy: it + also omits the zero-residual row and would return the understated number. +- The shared leverage-one warning (also used by `hc3`) now names the + offending row indices and recommends classical exact inference or more + observations in the cell instead of "add treated units". diff --git a/diff_diff/estimators.py b/diff_diff/estimators.py index 07d81da20..60d5800db 100644 --- a/diff_diff/estimators.py +++ b/diff_diff/estimators.py @@ -109,7 +109,12 @@ class DifferenceInDifferences(BaseEstimator): (library default). With ``cluster=``, uses CR1 (Liang-Zeger). - ``"hc2"``: leverage-corrected meat (one-way only). Errors with ``cluster=``; use ``"hc2_bm"`` for clustered Bell-McCaffrey. - - ``"hc2_bm"``: one-way HC2 + Imbens-Kolesar (2016) Satterthwaite DOF; + A leverage-one observation has no defined HC2 variance and the + vcov fails closed (warning + NaN inference, point estimate + preserved) rather than flooring ``1 - h_ii``. + - ``"hc2_bm"``: one-way HC2 + Imbens-Kolesar (2016) Satterthwaite DOF + (unweighted and unclustered, it shares the ``hc2`` leverage-one + fail-closed contract); with ``cluster=``, Pustejovsky-Tipton (2018) CR2 cluster-robust. ``MultiPeriodDiD(cluster=..., vcov_type="hc2_bm")`` is supported and uses a cluster-aware Bell-McCaffrey contrast DOF for the @@ -2513,9 +2518,14 @@ class MultiPeriodDiD(DifferenceInDifferences): (library default). With ``cluster=``, uses CR1 (Liang-Zeger). - ``"hc2"``: leverage-corrected meat (one-way only). Errors with ``cluster=``; use ``"hc2_bm"`` without cluster for Bell-McCaffrey. + A leverage-one observation has no defined HC2 variance and the + vcov fails closed (warning + NaN inference, point estimate + preserved) rather than flooring ``1 - h_ii``. - ``"hc2_bm"``: one-way HC2 + Imbens-Kolesar (2016) Satterthwaite DOF per coefficient plus a contrast-aware DOF for the post-period-average - ATT. With ``cluster=``, dispatches to Pustejovsky-Tipton (2018) + ATT (unclustered, it shares the ``hc2`` leverage-one fail-closed + contract: all-NaN vcov and DOF, point estimates preserved). With + ``cluster=``, dispatches to Pustejovsky-Tipton (2018) CR2 cluster-robust with a Bell-McCaffrey Satterthwaite contrast DOF on the post-period average (see ``cluster`` above for parity details). Weighted CR2-BM (``survey_design=``) is still gated. diff --git a/diff_diff/linalg.py b/diff_diff/linalg.py index 8252222e1..d68b2dabf 100644 --- a/diff_diff/linalg.py +++ b/diff_diff/linalg.py @@ -1170,7 +1170,11 @@ def solve_ols( (default). With ``cluster_ids``, dispatches to CR1 (Liang-Zeger). - ``"hc2"``: leverage-corrected meat. One-way only; raises with ``cluster_ids`` (use ``"hc2_bm"`` for clustered Bell-McCaffrey). - - ``"hc2_bm"``: HC2 + Imbens-Kolesar (2016) Satterthwaite DOF one-way; + An observation with leverage ``h_ii ~ 1`` has no defined HC2 + variance and the vcov fails closed (warning + NaN) rather than + flooring ``1 - h_ii``; point estimates are unaffected. + - ``"hc2_bm"``: HC2 + Imbens-Kolesar (2016) Satterthwaite DOF one-way + (unweighted: same leverage-one fail-closed contract as ``hc2``); Pustejovsky-Tipton (2018) CR2 Bell-McCaffrey with ``cluster_ids``. With ``weights``, dispatches to the clubSandwich WLS-CR2 port — supported for ``weight_type="pweight"`` only. ``aweight`` and @@ -2158,13 +2162,26 @@ def compute_robust_vcov( - ``"hc2"``: leverage-corrected meat ``sum_i (u_i^2 / (1 - h_ii)) x_i x_i'`` where ``h_ii`` are hat-matrix diagonals. No DOF adjustment beyond ``n - k``. One-way only; errors with - ``cluster_ids``. + ``cluster_ids``. A positive-weight observation with leverage + ``h_ii ~ 1`` (a perfectly fitted row, e.g. a single treated unit) has + no defined HC2 variance: the vcov fails closed (warning + all-NaN + vcov and DOF) rather than flooring ``1 - h_ii``, matching R + ``sandwich::vcovHC`` (NaN) and the ``hc3`` contract; point estimates + are unaffected. Under ``fweight`` a zero-count row is excluded from + the guard and the meat (compressed equals literal expansion). - ``"hc3"``: jackknife-style leverage correction, meat ``sum_i (u_i^2 / (1 - h_ii)^2) x_i x_i'`` (matches ``sandwich::vcovHC`` type="HC3": no DOF factor). One-way only; errors with ``cluster_ids``. + Same leverage-one fail-closed contract as ``hc2``. - ``"hc2_bm"``: one-way HC2 meat plus Imbens-Kolesar (2016) Bell-McCaffrey Satterthwaite degrees of freedom per coefficient when ``cluster_ids`` is - ``None``. When ``cluster_ids`` is supplied, dispatches to the + ``None``; unweighted, this shares the ``hc2`` leverage-one fail-closed + contract (a documented deviation from R ``clubSandwich``, whose + singleton-cluster CR2 generalized inverse returns a finite variance + there). With ``weights`` (pweight) it routes through the clubSandwich + WLS-CR2 port and keeps that finite generalized-inverse result at + leverage one — so ``hc2_bm`` and ``hc2_bm + weights=ones`` differ on + such designs. When ``cluster_ids`` is supplied, dispatches to the Pustejovsky-Tipton (2018) CR2 Bell-McCaffrey cluster-robust estimator (matches R ``clubSandwich::vcovCR(..., type="CR2")``). Required by the Pierce-Schott (2016) TWFE application in de Chaisemartin et al. (2026) @@ -2270,7 +2287,11 @@ def compute_robust_vcov( For HC2 one-way (weighted per review MEDIUM #3): h_ii = w_i * x_i' * (X'WX)^{-1} * x_i (unweighted: w_i = 1) meat = sum_i (u_i^2 / (1 - h_ii)) x_i x_i' - Guards against h_ii > 1 - eps with a fall-back to HC1 plus warning. + Fails closed (warning + all-NaN vcov/DOF) when any positive-weight + row has h_ii >= 1 - 1e-8 (leverage one; the meat term is 0/0), the + same contract as hc3 and R sandwich::vcovHC. No floor, no HC1 + fallback. Under fweight, zero-count rows are excluded from the + guard and the meat (replicated-data no-ops). For HC2 + Bell-McCaffrey one-way DOF (per Imbens-Kolesar 2016): For each coefficient j, let q_j = X (X'X)^{-1} e_j, let M = I - H. @@ -2286,11 +2307,15 @@ def compute_robust_vcov( weights = _validate_weights(weights, weight_type, X.shape[0]) # Rust HC2 (one-way, unweighted, no DOF): mirrors the NumPy hc2 branch - # exactly (leverage meat, no n/(n-k) factor). The near-singular - # hat-diagonal guard stays Python-side: the kernel returns a sentinel - # error and the documented warn-and-fall-back-to-HC1 fires here, - # identical to the NumPy branch's behavior. Imported independently - # (mixed-version safe) — None on a stale extension. + # exactly (leverage meat, no n/(n-k) factor, no floor). The leverage-one + # guard decision stays Python-side: the kernel only signals with a + # sentinel error, and the dispatcher re-dispatches to the NumPy branch, + # which recomputes the hat diagonals and fails closed (NaN vcov + the + # single user-facing warning, same stacklevel as the direct path). + # Imported independently (mixed-version safe) — None on a stale + # extension; a present-but-stale kernel still raises the legacy + # over-one sentinel, which takes the same re-dispatch (it cannot signal + # h == 1 exactly, so rebuild after changing the kernel). if ( HAS_RUST_BACKEND and _rust_compute_robust_vcov_hc2 is not None @@ -2305,19 +2330,21 @@ def compute_robust_vcov( return _rust_compute_robust_vcov_hc2(X_c, residuals_c) except ValueError as e: error_msg = str(e) - if "Hat-matrix diagonal exceeds 1" in error_msg: - warnings.warn( - f"{error_msg} Falling back to HC1.", - UserWarning, - stacklevel=2, - ) + if ( + "Hat-matrix leverage ~1" in error_msg + or "Hat-matrix diagonal exceeds 1" in error_msg + ): + # Leverage-one sentinel (current kernel) or the legacy + # over-one sentinel (stale kernel): re-dispatch to the NumPy + # hc2 branch, whose family-wide guard emits the warning and + # returns the all-NaN vcov. No HC1 fallback (retired). return _compute_robust_vcov_numpy( X, residuals, cluster_ids=None, weights=None, weight_type=weight_type, - vcov_type="hc1", + vcov_type="hc2", return_dof=return_dof, ) if "Matrix inversion failed" in error_msg: @@ -2436,8 +2463,9 @@ def _compute_hat_diagonals( ``sandwich::vcovHC(..., type="HC2")`` in R and matches the per-observation effective leverage under WLS. - Returns an ``(n,)`` array. Values are clamped to ``[0, 1 - 1e-10]`` to - guard against numerical `` h_ii > 1`` from near-singular designs. + Returns an ``(n,)`` array. Values are NOT clamped: a diagonal at or + above ``1 - 1e-8`` is the leverage-one condition the callers fail closed + on (see :func:`compute_robust_vcov`). """ # Compute x_i' (X'WX)^{-1} x_i via a single solve rather than per-row. # np.linalg.solve(bread, X.T) has shape (k, n); multiplying element-wise by @@ -2455,8 +2483,8 @@ def _compute_hat_diagonals( h_diag = np.einsum("ij,ji->i", X, proj) if weights is not None: h_diag = weights * h_diag - # Numerical guard. Do not silently clip values materially exceeding 1 — that - # indicates a real design pathology; the caller warns and falls back. + # No clipping: a value at or above 1 - 1e-8 is a real design pathology + # (a perfectly fitted row); the callers warn and fail closed. return np.asarray(h_diag, dtype=np.float64) @@ -3316,9 +3344,11 @@ def _compute_bm_dof_from_contrasts( Returns ------- - ndarray of shape (m,) of Satterthwaite DOF per contrast column. NaN when - the denominator is non-positive or at/below the cancellation noise - floor (degenerate / extreme-leverage case; see the inline guard note). + ndarray of shape (m,) of Satterthwaite DOF per contrast column. All NaN + when any row has hat-matrix leverage ``>= 1 - 1e-8`` (the one-way HC2 + vcov is undefined there, so its DOF is too); otherwise NaN per contrast + when the denominator is non-positive or at/below the cancellation noise + floor (extreme-but-defined leverage; see the inline guard note). """ n, k = X.shape if contrasts.ndim != 2 or contrasts.shape[0] != k: @@ -3350,7 +3380,16 @@ def _compute_bm_dof_from_contrasts( raise # q has shape (n, m); column j is X @ (bread_inv @ contrasts[:, j]). q = X @ bread_inv_c - one_minus_h = np.maximum(1.0 - h_diag, 1e-10) + # Leverage one (h_ii >= 1 - 1e-8): the one-way HC2 vcov is undefined on + # every path that reaches this helper (the shared kernel fails closed + # before calling it; the MultiPeriodDiD / TWFE pooled event-study caller + # receives the same all-NaN vcov from solve_ols), so the Satterthwaite + # DOF of that undefined variance is NaN for every contrast. The former + # max(1 - h, 1e-10) floor is gone; below leverage one the per-contrast + # noise-floor cancellation guard further down is unchanged. + if np.any(h_diag >= 1.0 - 1e-8): + return np.full(contrasts.shape[1], np.nan) + one_minus_h = 1.0 - h_diag one_minus_2h = 1.0 - 2.0 * h_diag m = contrasts.shape[1] dof = np.empty(m) @@ -3388,6 +3427,42 @@ def _compute_bm_dof_from_contrasts( return dof +_LEVERAGE_ONE_LABELS = {"hc2": "HC2", "hc2_bm": "HC2-BM", "hc3": "HC3"} + + +def _leverage_one_message(label: str, lev_rows: np.ndarray) -> str: + """User-facing text for the leverage-one fail-closed warning. + + ``lev_rows`` is the FULL index array of offending (positive-weight) rows; + the message states the total count and previews the first five. The + ``"