Release 2.0.0 blocked: T5 runtime gate fails deterministically (+27-34% on CI) — budget/baseline decision needed #156

Release 2.0.0 (tag v2.0.0 at 629429f, PR #155) reached the release pipeline with everything green except the CleanBench T5 runtime gate, which failed at +27.68% against the 20% budget (run 29736705242). Nothing was published: PyPI and GitHub Releases still have 1.1.1 as latest. This issue documents why the failure is real, what causes it, and the decision needed to unblock.

The gate failure is deterministic, not runner variance

The July 18 v1.2.0 post-mortem attributed the second failed attempt to a "±30% runner lottery vs absolute baseline". The full nightly cleanbench-full history disproves that:

The gate passed on CI runners as recently as Jul 10, so the committed baseline (0.3878s / 50k rows) is CI-achievable and the instrument works. A regression landed Jul 13, was partially recovered (incl. #151's vectorized sci-notation guard), and the code has sat at a stable **+27–34% on CI** for seven consecutive runs. Re-running the workflow is dice-rolling, not a fix.

Where the time goes

Controlled same-machine A/B (T5 fixture, 50k rows, min-of-5, interleaved rounds, arm64):

cProfile attribution of the delta on the default path:

• engine/cache.py:build_engine_cache (~0.08s new): column contexts + numeric correlation for the engine-backed auto_missing/auto_outliers default steps — intended safety-by-default work, and the correlation matrix is genuinely consumed by engine/missing.py.
• steps/dtypes.py:_try_numeric roughly doubled (10 pd.to_numeric passes vs 5): the guard-then-coerce pattern from the coerced-cell quarantine work.
• Assorted smaller costs from typography protection, duplicate detection without dropping, and quarantine bookkeeping.

CI (x86) shows ~2× the slowdown of arm64 (+27–34% vs +14.5%), consistent with string/regex-heavy guard code costing relatively more there.

The actual conflict

The gate budgets +20% vs a "v1.0-equivalent default path", but 2.0.0's entire direction — decided in the production audit and shipped in #154 — is that the default path deliberately does more work (detection without mutation, quarantine, provenance). The budget was never revisited when that direction was set. The gate is now enforcing a v1.0-era performance contract against a v2.0-era safety contract; one of them has to give, and that is a product decision, not a release-engineering one. (Related history: #152/#153 neutering the gate with || true was itself flagged and reverted in #154 — silently accommodating the gate is not an option.)

Options

A. Accept the 2.0.0 default-path cost. Re-record benchmarks/cleanbench/results/baseline_v1.json on a CI-class runner at the release commit (or explicitly raise FULL_GATE_RUNTIME_SLOWDOWN), with the decision recorded in the changelog. Requires a commit, so the release becomes either 2.0.1 or a re-tag of v2.0.0 (see tag note below).

B. Claw the cost back first. Profile-guided candidates: share one to_numeric pass between guard and coercion in _try_numeric; dedupe context profiling between build_engine_cache and the step-level profilers. Estimated recovery ~10–15% on CI, which would bring the code back under the existing budget. Then release as above.

C. Redesign the gate (the durable fix). Measure the v1.0-reference implementation in the same run and gate on the relative ratio, eliminating cross-runner drift entirely. This was already identified as the open question in the v1.2.0 post-mortem; the Jul 5–12 marginal flip-flops (+20.4%/+21.4% fails around genuine passes) show the absolute design has near-zero noise margin even when the code is fine.

Tag state

Two tags now point at commits that were never published: v1.2.0 (9368ea4) and v2.0.0 (629429f). Since no artifacts were ever produced from either, deleting and re-tagging v2.0.0 on the eventual release commit is defensible; going to 2.0.1/2.1.0 and leaving them is the conservative alternative. Either way, main's version metadata (2.0.0 in pyproject.toml/__init__.py, changelog dated 2026-07-20) is release-ready and validated: full test matrix green on PR #155, artifacts built and twine check-clean, wheel smoke-tested in a clean environment.

Exact unblock path (if Option A is chosen)

1. On a branch from main: regenerate baseline_v1.json via the documented baseline harness on a ubuntu-latest runner (python -m benchmarks.cleanbench --update-baseline, in CI, not locally), commit with the rationale, PR, merge.
2. Delete and re-create v2.0.0 on the merge commit (or bump to 2.0.1 and tag that).
3. The tag push runs release.yml end-to-end: trusted publishing to PyPI, GitHub Release with artifacts, checksums, and attestations — no further manual steps.