[ealt]

Fixes #201.

Summary

linear_regression called jnp.linalg.lstsq(..., rcond=None). That cutoff is
max(n_rows, n_columns) * eps — scaled by the row count, so fitting activations on more
samples discards more of the design. Under float32 it bites at ordinary sizes.

The change makes the cutoff an explicit, row-count-independent parameter and reports how much
of the design survived it:

• linear_regression(..., rcond=...), defaulting to default_rcond(n_columns) =
  n_columns * eps. The column count governs the conditioning of the solve; the row count
  does not. rcond=0.0 gives an untruncated solve.
• Spectrum diagnostics on both solvers: effective_rank, n_singular_values, rcond,
  condition_number. Issue analysis.linear_regression: lstsq rcond=None silently truncates the design with a row-count-scaled cutoff, moving peak layers #201's point that "no summary a caller can see predicts how much
  is discarded" was true; now one does.
• score_regression(x, y, weights, coeffs, intercept) — a held-out scorer. The fit
  functions report in-sample metrics, and every caller wanting an out-of-sample R² was
  reimplementing this with its own weighting convention.
• rcond is threaded through _validate_linear_regression_kwargs and
  LinearRegressionAnalysis, so it is reachable from a config and not only from Python.

Motivation

Measured independently of the issue, building the reported design from scratch (66,930 × 321,
float32, condition number 5.32e3 — a design that needs no truncation at all):

	cutoff	directions kept	held-out R² vs float64
before	7.979e-3	181 of 321	up to 0.433 worse
after	3.83e-5	321 of 321	0.000

The cutoff arithmetic reproduced the filed numbers exactly. The harm did not at first — and
chasing that gap sharpened the diagnosis. Held-out R² loss is not a function of the
condition number; it is a function of where the target's signal sits in the spectrum. Holding
the design fixed and moving signal from high- to low-variance directions swept the gap:

signal alignment	0.0	0.25	0.5	0.75	1.0
R² gap (float64 − shipped)	0.0000	0.0007	0.0077	0.0762	0.4334
after this change	0.0000	0.0000	0.0000	0.0000	0.0000

The issue's measured 0.13 sits inside that range, and this explains its own observation that
the condition number predicts the damage only loosely (Spearman −0.68). Real activations put
belief signal in low-variance directions, which is why the bug bites there.

Cutoff, not precision. Confirmed separately: float32 with an explicit rcond agrees with
float64 to ~1e-10 at every alignment above. So this is an rcond change, not a dtype change.

Reshape rather than append

linear_regression and linear_regression_svd were two implementations of one operation —
both standardize, weight, build a design and solve it; one via lstsq, one via an explicit
SVD with a threshold. Adding a third parameterised path would have been the easy change.
Instead:

• the duplicated preamble (triplicated, counting _compute_regression_metrics) is extracted
  into _prepare_problem;
• linear_regression now solves through the same SVD + _compute_beta_from_svd the SVD
  variant already used, so there is one thresholding rule, not two;
• the duplicated fit_intercept result-packing is extracted into _beta_arrays;
• _split_concat_results filtered kwargs with a denylist (!= "rcond_values"), which a
  new parameter would have silently broken — it is now an allowlist.

Net diff is +291/−83 across four files, most of it docstrings and tests.

A reviewer should push back here if they would rather see the two public functions collapse
into one with the cutoff as a parameter. I kept both because linear_regression_svd's
candidate search is a distinct operation with a distinct (and, as noted below, flawed)
selection rule, and merging them would have made this PR a signature break for existing
config users on top of a correctness fix.

Deliberately out of scope

A ridge with a validation-chosen penalty, which #201 argues a caller needs. Doing it
correctly in this module requires decisions this PR should not make silently: a ridge here
would penalise the intercept, and standardize_features only casts dtype — it does not
centre or scale — so the penalty would be feature-scale dependent. Shipping that is a
footgun, not a remedy. Filed separately rather than left as a paragraph.

linear_regression_svd selects its candidate on in-sample error, which cannot identify a
value that regularises: a cutoff only ever helps out-of-sample. I expected the search to be
outright degenerate (in-sample error is monotone in directions kept) and measured that it is
not — float32 roundoff gives in-sample error a genuine optimum, and it picked 1e-6 over
1e-15 in both candidate orderings on a κ=2e8 design. So it is not broken, just selecting on
the wrong criterion. That is now stated in its docstring, pointing at score_regression for
honest selection, rather than changed under existing callers.

Breaking changes

None to signatures — rcond is keyword-only with a default. Two behaviour changes callers
should know about:

1. Numeric results move for any caller whose design was being truncated. That is the fix,
   but it is not one-signed: for a genuinely near-singular design the old accidental cutoff
   was regularising, and removing it can make held-out fit worse. effective_rank and
   condition_number now make that visible, and rcond lets a caller choose it.
2. New scalar keys (effective_rank, n_singular_values, rcond, condition_number)
   appear in the returned metrics, and therefore in MLflow metric names under the existing
   layer-key scheme.

Blast radius

Verified by grep against the actual refs, with positive controls (my first two greps ran
against branches where the directories did not exist and returned a confident, empty,
meaningless result):

• Calls the changed line — simplex-research@a5d401baa: stage2/belief.py:110,
  targets/selfcheck.py:170 and :380. Belief-regression R² reported from stage2 and the
  targets programme will move. selfcheck.py exists partly to demonstrate this defect.
• Mentions only — ablate/subspaces.py imports get_robust_basis, which has its own
  max(matrix.shape) * eps cutoff but applies it to the coefficient matrix, not a
  row-scaled design; polar/report.md and the READMEs are prose.
• Unaffected — eden-experiments@loop-belief. DESIGN.md names the module twice, both in
  planning prose ("reuse, not rewrite" and an inventory). Every probe there uses
  sklearn.linear_model.Ridge(1e-3) scored on a held-out split.
• No caller inside simplexity itself beyond the registry wrappers in this diff.

Testing

uv run --extra dev --extra pytorch --extra penzai --extra aws pytest -q →
1052 passed, 2 skipped. ruff format --check clean. pyright → 0 errors.

Nine new tests. The headline regression test asserts recovered coefficients against a float64
reference on a tall well-conditioned design, and was run against the pre-fix module first:

negative control for the #201 regression test:
  pre-fix    -> FAIL   max|beta - float64| = 5833
  post-fix   -> PASS   max|beta - float64| = 0.01758

The sharpest of them, test_linear_regression_cutoff_is_independent_of_row_count, fits the
same column space at 2,000 / 20,000 / 60,000 rows and asserts the effective rank does not
change — which is the bug stated directly.

Also driven end-to-end through LinearRegressionAnalysis rather than only the bare function,
confirming rcond reaches the solve from a config (default → rank 9/9; rcond=1e-1 → rank
3/9, R² 0.189) and that the validator refuses a negative, non-numeric, or wrongly-paired
value.

Suppressions

None added.

One note on the repo's gates

ruff check . exits 1 on this branch and on origin/main — 3 errors (2× SIM117, 1×
PT012) in tests/generative_processes/test_data_prefetcher.py, which this PR does not touch
(verified byte-identical to origin/main). I left it alone because the PT012 finding needs a
judgement about that test's semantics — it raises inside its context manager deliberately, to
test cleanup — which does not belong bundled into a correctness fix. Filed separately.

Separately, CLAUDE.md's documented command uv run --extra dev --extra pytorch pyright
reports 20 errors and pytest fails collection on 3 modules, purely from missing penzai
and aws extras. Adding --extra penzai --extra aws gives 0 pyright errors and a clean
collection. The documented command is under-specified.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DYtgMenFgbPzNoN1MqxCxG