Skip to content

Add opt-in FTC09 platform cost cohort without repeating fixed15 - #1878

Merged
pzfreo merged 2 commits into
mainfrom
issue-1832-ftc09-platform-cost
Sep 27, 2026
Merged

pzfreo merged 2 commits into
mainfrom
issue-1832-ftc09-platform-cost

Conversation

@pzfreo

@pzfreo pzfreo commented Sep 27, 2026

Copy link
Copy Markdown
Owner

Why

#1832 has completed fixed-15 cost evidence on Linux/macOS/Windows, but its manual workflow is hard-wired to that already-passed cohort. FTC-09 is a distinct PMI-rich public real part with an existing versioned one-case manifest. It is not a substitute for representative user-supplied large CAD.

Change

Add an allowlisted ftc09 cohort to manual dispatch and a separate opt-in PR label (annotation-layout-cost-ftc09). Existing annotation-layout-cost and default manual dispatch continue to select fixed15. The workflow records the chosen manifest and SHA-256 alongside exact source-head/machine provenance, retains serial workers and their 900-second bound, and names artifacts by cohort/platform. No default CI CAD reruns.

Review

  • tests/test_workflows.py: 75 passed.
  • Ruff check/format and diff check passed.
  • No CAD worker run by this PR update; the FTC09 label must be applied explicitly for one supported-platform evidence run.

Partial #1832/#1831 evidence path only. The large user-CAD and concurrency/fallback-inclusive cost gates remain open.

@pzfreo

pzfreo commented Sep 27, 2026

Copy link
Copy Markdown
Owner Author

The first FTC09 opt-in run (36330010666) failed correctly on all three platforms because the then-current candidate lost one PMI carrier and the 190.5 width. PR #1879 contains the pre-render fix; its changed-source candidate-only holdout restores full parity against the saved Linux baseline and remains honestly unsafe off-page. I will leave this PR and its red run intact until #1879 merges, then restack on current main and trigger one new exact-head FTC09 platform run. The passed fixed15 cost cohort will not be repeated.

@pzfreo
pzfreo force-pushed the issue-1832-ftc09-platform-cost branch from 80becbb to b5a9ced Compare September 27, 2026 16:19
@pzfreo pzfreo added annotation-layout-cost-ftc09 Run opt-in three-platform FTC09 annotation layout cost cohort and removed annotation-layout-cost-ftc09 Run opt-in three-platform FTC09 annotation layout cost cohort labels Sep 27, 2026
@pzfreo

pzfreo commented Sep 27, 2026

Copy link
Copy Markdown
Owner Author

Review of the one opt-in FTC09 run 36332797155 at CAD/source head b5a9ced: all three platform jobs passed, selected iso-growth before rendering, and have exact semantic parity (missing/added/introduced blockers all empty), a quality tie, and byte-identical baseline/candidate SVGs within each platform. Candidate/baseline process-time ratios: Linux 0.975× (126.46/129.74 s), macOS 0.760× (119.93/157.80 s), Windows 0.939× (126.49/134.76 s). Peak-RSS ratios: 0.971×, 1.026×, 0.960× respectively. This is one sample per platform, not a representative median/p95 or large-user-CAD budget pass. The independent safety checker still fails eight checks, including off-page views/ink, on all three; those are shared with baseline, not candidate-specific losses.

Artifact review found the Windows checked-out JSON uses CRLF: raw-file manifest SHA-256 ac16a903… differs from Linux/macOS 9012b99e…, while both are the same committed Git blob. Follow-up bf11fd3 changes only provenance hashing to the committed blob; local LF/CRLF reproduction exactly matched both artifact hashes, 75 workflow tests and Ruff passed. I am not rerunning the completed CAD cohort for that provenance-only correction. The artifact's source SHA remains b5a9ced and should be cited as such; bf11fd3 is the workflow fix head.

@pzfreo
pzfreo enabled auto-merge (squash) September 27, 2026 16:26
@pzfreo
pzfreo merged commit b7ea911 into main Sep 27, 2026
18 checks passed
@pzfreo
pzfreo deleted the issue-1832-ftc09-platform-cost branch September 27, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

annotation-layout-cost-ftc09 Run opt-in three-platform FTC09 annotation layout cost cohort

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant