Problem
policyengine.py v4.14.1 (#390) promoted the microplex "MP eCPS" artifact (mp-ecps-2024-2cdd45d-20260605, HF revision a091769a) to the certified US dataset, replacing the production Enhanced CPS (policyengine-us-data 1.115.5).
That promotion was decided on a target surface that mis-scored the incumbent eCPS (unweighted MSRE 3.464; e.g. unemployment compensation +615%), so it cleared a content baseline gate rather than the numeric baseline-sanity gate.
Evidence
On a correctly-scored target surface (eCPS unweighted MSRE 0.174), two independent symmetric-refit comparisons agree to the digit that production eCPS beats MP 2cdd45d:
| metric (lower is better) |
eCPS |
MP 2cdd45d |
| full PE-native loss |
0.02662 |
0.04275 |
| holdout loss |
0.00769 |
0.00938 |
| unweighted MSRE |
0.174 |
0.290 |
| target wins (of 2809) |
2334 |
267 (208 ties) |
MP is 14–49× worse on every income component (interest, retirement, dividends, capital gains, wages, self-employment) — the variables that drive AGI, brackets, and phase-outs in a tax model — and only better on SNAP.
Fix
Restore the certified US dataset to enhanced_cps_2024 (policyengine-us-data 1.115.5, revision d47fb547, model policyengine-us==1.715.2) via the supported scripts/refresh_release_bundle.py path, and cut a patch release. MP remains a research track, to be re-promoted only once it wins on a correctly-scored surface (after the CPS-passthrough income-component fix).
Problem
policyengine.py v4.14.1 (#390) promoted the microplex "MP eCPS" artifact (
mp-ecps-2024-2cdd45d-20260605, HF revisiona091769a) to the certified US dataset, replacing the production Enhanced CPS (policyengine-us-data 1.115.5).That promotion was decided on a target surface that mis-scored the incumbent eCPS (unweighted MSRE
3.464; e.g. unemployment compensation+615%), so it cleared a content baseline gate rather than the numeric baseline-sanity gate.Evidence
On a correctly-scored target surface (eCPS unweighted MSRE
0.174), two independent symmetric-refit comparisons agree to the digit that production eCPS beats MP 2cdd45d:MP is 14–49× worse on every income component (interest, retirement, dividends, capital gains, wages, self-employment) — the variables that drive AGI, brackets, and phase-outs in a tax model — and only better on SNAP.
Fix
Restore the certified US dataset to
enhanced_cps_2024(policyengine-us-data 1.115.5, revisiond47fb547, modelpolicyengine-us==1.715.2) via the supportedscripts/refresh_release_bundle.pypath, and cut a patch release. MP remains a research track, to be re-promoted only once it wins on a correctly-scored surface (after the CPS-passthrough income-component fix).