Repository navigation
Add opt-in FTC09 platform cost cohort without repeating fixed15 - #1878
Conversation
|
The first FTC09 opt-in run (36330010666) failed correctly on all three platforms because the then-current candidate lost one PMI carrier and the 190.5 width. PR #1879 contains the pre-render fix; its changed-source candidate-only holdout restores full parity against the saved Linux baseline and remains honestly unsafe off-page. I will leave this PR and its red run intact until #1879 merges, then restack on current main and trigger one new exact-head FTC09 platform run. The passed fixed15 cost cohort will not be repeated. |
80becbb to
b5a9ced
Compare
|
Review of the one opt-in FTC09 run 36332797155 at CAD/source head b5a9ced: all three platform jobs passed, selected iso-growth before rendering, and have exact semantic parity (missing/added/introduced blockers all empty), a quality tie, and byte-identical baseline/candidate SVGs within each platform. Candidate/baseline process-time ratios: Linux 0.975× (126.46/129.74 s), macOS 0.760× (119.93/157.80 s), Windows 0.939× (126.49/134.76 s). Peak-RSS ratios: 0.971×, 1.026×, 0.960× respectively. This is one sample per platform, not a representative median/p95 or large-user-CAD budget pass. The independent safety checker still fails eight checks, including off-page views/ink, on all three; those are shared with baseline, not candidate-specific losses. Artifact review found the Windows checked-out JSON uses CRLF: raw-file manifest SHA-256 ac16a903… differs from Linux/macOS 9012b99e…, while both are the same committed Git blob. Follow-up bf11fd3 changes only provenance hashing to the committed blob; local LF/CRLF reproduction exactly matched both artifact hashes, 75 workflow tests and Ruff passed. I am not rerunning the completed CAD cohort for that provenance-only correction. The artifact's source SHA remains b5a9ced and should be cited as such; bf11fd3 is the workflow fix head. |
Why
#1832 has completed fixed-15 cost evidence on Linux/macOS/Windows, but its manual workflow is hard-wired to that already-passed cohort. FTC-09 is a distinct PMI-rich public real part with an existing versioned one-case manifest. It is not a substitute for representative user-supplied large CAD.
Change
Add an allowlisted
ftc09cohort to manual dispatch and a separate opt-in PR label (annotation-layout-cost-ftc09). Existingannotation-layout-costand default manual dispatch continue to select fixed15. The workflow records the chosen manifest and SHA-256 alongside exact source-head/machine provenance, retains serial workers and their 900-second bound, and names artifacts by cohort/platform. No default CI CAD reruns.Review
tests/test_workflows.py: 75 passed.Partial #1832/#1831 evidence path only. The large user-CAD and concurrency/fallback-inclusive cost gates remain open.