You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Candidate-first holdout: AP203, additional user parts, and authored Sheet workflows #1831
Child of #1813. The 15-part fixed-sheet corpus is the quality gate, not the full rollout sample. Add a documented holdout set that includes AP203 CTC variants, additional real/user STEP models not in the fixed 15 (including larger CAD), and authored Sheet / generated-script paths. Run candidate-first once per case with fixed caller page/scale/policy and compare to baseline offline, recording source SHA, format, machine, ownership/requirement parity, new blockers, view size/legibility, and representative PDF review. Avoid treating AP203 siblings as independent geometric wins. Do not change defaults or claim safety admission from aggregate quality alone. Link the resulting evidence PR to #1813 and keep any failing cases explicit.
Holdout progress (not a gate pass): authored Sheet and generated-script candidate-preview semantic comparisons both passed locally in draft PR #1835, with the same fixed A4 2:1 caller constraints. A six-case additional STEP holdout is running once from manifest/source commit 74fb65ec (bounded chooser implementation e0a828d0); five baseline/preview SVG pairs are rendered, but the final if_step_flat_across_cylinder.step baseline worker has exceeded 31 minutes of active CPU and no aggregate report exists yet. AP203 CTC siblings will be reported as format variants, not independent geometry wins. No default switch or safety admission follows from these partial results.
Partial AP203 visual holdout review, independent of the still-unfinished six-case semantic aggregate: I rendered the already-produced CTC02 and CTC04 baseline/preview SVG pairs through draftwright PDFium at 100 dpi and inspected full A2 sheets. CTC02 preview still crowds blue dimension/callout text around the upper-left orthographic view and visibly reduces the isometric relative to baseline. CTC04 preview gains a section A–A view, but retains dense stacked dimension bands around the left/lower orthographic views and introduces very long blue horizontal annotation strokes across the lower-right blank corridor; this is not a visual safety pass. These are qualitative page-level findings only; the final flat-across-cylinder worker is still active, so no holdout quality/parity aggregate is claimed. The PDFs were generated under /tmp/draftwright-holdout-v1-u0R2yF/ from the existing SVGs and should be attached as review artifacts when the holdout PR is published.
The first six-case run is now terminal, not merely slow: command exit 143 (SIGTERM), zero-byte aggregate, and no per-case JSON from the pre-#1836 runner. The flat-across-cylinder baseline worker had no SVG after ~51 min wall / ~34 min CPU; candidate for that case never ran. Five other baseline/preview SVG pairs survived, but cannot establish semantic parity or quality verdicts. I did not restart the run. Draft PR #1837 now versions the manifest, exact source/manifest provenance, this failure, and reviewer-accessible CTC02/04 AP203 baseline/preview page images with explicit visual limitations. The broader holdout gate remains open, including novel large user STEP.
Follow-up runner safeguard is draft PR #1838 (stacked after the interrupted-run evidence): optional per-isolated-worker time bound, typed baseline/candidate timeout failures, nonzero aggregate status, and no false baseline selection or successful latency sample. It has unit coverage only; it was not used for the already-interrupted run, and it does not complete the broader holdout or authorize a rerun of the five rendered cases.
A new independent public real-part holdout is in progress, without rerunning the fixed15 or the five already-rendered AP203 cases: NIST FTC-09 AP242, 5.83 MiB, one imported solid (~227 × 279 × 5.25 mm), source SHA-256 f1215fe15a78085a9fa78dd81714caf774b59071be92580b04ebfdc19f52a1bd. NIST permits unrestricted reuse and requests acknowledgement: https://www.nist.gov/ctl/smart-connected-systems-division/smart-connected-manufacturing-systems-group/mbe-pmi-0 . It is distinct FTC geometry, not a CTC AP203 sibling and not a user-supplied model. One fixed A2 1:1 candidate-first comparison has been started with a 600-second bound per isolated worker; a timeout is a failed, censored case, not a pass. Await the actual report before drawing any quality/parity conclusion.
New independent public real-part AP242 holdout: #1839 (stack #1816, draft pending exact-head CI). NIST FTC-09 A2/1:1 candidate-first bounded comparison completed both workers once. Parity passed and rendered pages were byte-identical, but quality tied (0 wins), independent safety failed, and the page remains crowded/clipped; no production selection. This is a 6.1-MB PMI-rich public file, not a physically huge or user-supplied model. The previous AP203 interrupted evidence and authored Sheet tests remain separate. Real/user STEP visual and safety gates stay open; no fixed15 rerun.
FTC-09 first-run triage (same source/report as #1839; no rerun): candidate and baseline have identical 63 blocker identities. By class: 31 off-axis locations, 9 PMI, 9 slot dimensions, 7 callouts, 4 unsatisfiable placements, 3 circular-channel locations. Quality is also identical: 21 hard layout violations, 11 overlaps, 9 crossings. Most stark fixed-A2 failure: reported plan-view bounds are y=442.90..445.94 mm on a 420-mm-high page, i.e. the entire plan view is off-sheet; the side view is only ~3.04 mm wide. This is an absolute page-fit/legibility defect in both modes, not an introduced candidate regression or a verified gain. It needs an explicit fixed-constraint resolution or honest gated limitation before #1813 rollout.
Safety-gate design note from current stack top #1848: schema-8 declared Sheet reports expose final-IR declarations/representations and lint, but deliberately have no raw schema-3 recognition requirement ledger. The current candidate gate checks declared inventory and four aspect kinds from live declaration provenance; its own limitations still mark other authored non-dimension parity and generated-script parity incomplete. A holdout or future admission check must verify those obligations from declaration identities/sidecar evidence, not pretend that raw recognized requirement counts exist on replay. Existing candidate-preview authored/generated-script parity tests cover a small Box/hole/datum case only; they do not establish broader user-part parity or production admission. No test or corpus rerun for this note.
Holdout-gate cross-check: the FTC-09 report's off-sheet plan view is already addressed at the independent safety boundary by stacked PR #1840 (source 29ef1fe): view_page_containment compares each settled view to the caller's page, and focused tests cover off-page/non-finite bounds. The older FTC-09 report predates that check, so it must not be read as proof that the current gate passes the case. This fixes false safety reporting, not the off-sheet layout itself; no slow FTC-09 worker was restarted.
Additional visual review of the five preserved AP203/real-part SVG pairs from the interrupted first holdout (no CAD rerun): the previously recorded CTC02/04 concerns remain; I rendered and inspected the other three pairs at page scale. CTC01 AP203 candidate visibly relocates the long 2×R20 callout to a shorter upper-plan route and retains the same apparent views/content, but the tight right orthographic callouts still need semantic/legibility checking; this is a local visual gain, not an aggregate win. GRM03-no-PMI baseline and candidate SVGs are byte-identical (SHA-256 b7da93492e65547cd721f7bfe19f5e1829985d1a277f449fa35618891cc461be), so this is a visual tie. Single-cylinder A3/1:1 looks nearly identical in both modes, with a tiny part on a large caller-fixed sheet; that sparsity is not a recognition or safety pass. The old runner's report.json is zero bytes, so none of these observations establishes requirement parity, a quality verdict, or completion of #1831. The local rendered pairs remain under /tmp/draftwright-holdout-v1-u0R2yF; no slow worker or paired corpus was restarted.
New independent public-part holdout started once at stack head c5d87ad (no fixed15/paired rerun): NIST FTC-08 AP242, archive SHA-256 8fa78429e6d8d9b0d7681d223b6aa9ec98c3772185c55b1a0e3679b21c181911, member nist_ftc_08_asme1_ap242-e2.stp SHA-256 7b9283b4288d152fd181d76c39285d7c79fbba34fda1e14a41ac4a983f6d68b3 (4,722,264 bytes). Import-only preflight gave a ~311.607×222.707×48.260-mm compound with one MANIFOLD_SOLID_BREP record. The caller manifest SHA-256 is 772298bef0d1696a35f87a429bef870b0ee5c5471f44150ee3d6630763bf7e21: fixed A2/1:1, staggered side, exterior dimensions, SVG. Each isolated baseline/candidate worker has a 600-second bound, one worker at a time. An initial invocation failed before CAD import because the worktree runner selected system Python without build123d; the corrected invocation uses the installed venv and its baseline worker is actively using CPU. No quality, parity, cost, or safety conclusion yet. NIST permits unrestricted use of these files and requests acknowledgement: https://www.nist.gov/ctl/smart-connected-systems-division/smart-connected-manufacturing-systems-group/mbe-pmi-0 . This is not a user-supplied model or a complete large-CAD gate.
FTC-08 public-part holdout completed once and is now recorded in stacked draft PR #1856 (source c5d87ad; full report SHA-256 878da76d30143a3729c09e0aa94fff53e4355c6540aae10960a1e06f395a1dbd). Fixed A2/1:1 candidate-first chose columns; baseline/candidate SVGs are byte-identical, offline quality tied, and semantic parity passed with no missing annotations/new blocker identities/new interior dimensions. Independent safety failed six checks, including side-view page containment: x-max 659.65 mm on a 594-mm-wide sheet. Coverage is 100/131 placed, 24 dropped, 2 unsupported, 5 unverifiable. Both modes share this absolute defect; it is not a candidate regression or a verified win. Linux process times were baseline 240.41 s/candidate 211.89 s, RSS 657.47/655.16 MiB, one sample only. Source STEP contains datum/tolerance entities, but rendered PMI coverage remains unresolved. User-supplied large CAD and the broader holdout gate remain open; no fixed15 or successful worker was rerun.
Current-source FTC09 holdout correction: three-platform run https://github.com/pzfreo/draftwright/actions/runs/36330010666 at head 80becbb (same pinned FTC09 A2/1:1 manifest) is no longer the older documented tie. Linux, macOS, and Windows each completed both workers and agree: candidate-first selected columns, retained page/scale/views, but dropped baseline PMI source geometric_tolerance:0:1:4:32 and the envelope width.length dimension 190.5. Exact semantic parity therefore fails on all three; compare exit 1 is correct. This is a candidate-specific regression to diagnose from the saved reports before any slow rerun, not a shared absolute defect and not a reason to repeat the passed fixed15 work. The older FTC09 report remains valid only for its recorded source b160f2e. The broader user/large-part holdout is still open.
Follow-up to the three-platform FTC09 regression: PR #1879 at b8108cd restores the two candidate-only lost carriers using a pre-render fixed-page profile choice. One changed-source candidate worker (no baseline rerun) produced a full manifest with zero missing/added annotations, equal coverage and drops, and no introduced blockers against the saved Linux baseline artifact from run 36330010666; the SVG is byte-identical to that saved baseline for this case. This is a relative tie, not an absolute safety pass: FTC09 still has off-page views/ink and other shared completeness failures. The comparison is cross-source diagnostic evidence; exact-head supported-platform evidence remains for #1832 after merge.
The exact-source FTC09 three-platform holdout run 36332797155 at b5a9ced restores full candidate/baseline semantic parity after #1879: no missing/added annotations or introduced blocker identities on Linux, macOS, or Windows; quality and within-platform SVGs tie. Candidate-first chooses iso-growth pre-render because columns would push the plan a further 6 mm off-page. The shared absolute off-page/completeness failures remain explicit, and this public FTC09 case is not a user-supplied large-CAD holdout or broader authored-Sheet proof. #1831 remains open.
Candidate-only GRM03 direct-vs-generated-Sheet holdout on main 0f17848 (plain grm03_thumbwheel_drive_screw.step, caller A3/5:1, candidate-preview; no baseline or corpus rerun): both builds choose legacy-depth, same page/scale/views, same 18 annotation names/labels, same drops/blocker identities and empty lint. However the exact drawing_record manifests are not equal: direct detection reports coverage 15/15 placed; generated authored Sheet reports 7/15 placed and 8 unverifiable, and independent safety fails authored_dimensions/audited_coverage (direct fails recognized_carriers). The one annotation ownership difference is dim_height: direct compiler mints a bbox-envelope measurement identity without adding a recognized EnvelopeFeature; emitter intentionally declares a synthetic envelope so the authored height is nameable, giving dim_height owner feature index 8. This is not a demonstrated candidate-vs-baseline regression or a visible-ink mismatch. It is a front-door provenance/coverage mismatch that the current GRM03 generated-script parity test does not catch because it compares annotation signatures/views/page/scale/lint, not exact ownership and requirement states. The synthesized envelope is deliberate in sheet_emit.mirror_model and compiled._compile_overall_height; do not add it to detected model.features merely to make reports equal. I am tracing the eight authored unverifiable claims under the existing script-fidelity boundary (#964) before proposing a code change. #1831 remains open.
Further code-level diagnostic of the saved GRM03 scripted holdout: layout_safety._authored_dimension_gaps finds exactly two generated hole location requests (Y and Z, member 0) with request_has_no_matching_parameter. These are distinct from the eight recognition-audit unverifiable counts; the latter come from the separate recognition-owned completeness path, not from the authored request checker. The script declares both locations, and the visible annotation inventory still matches direct. This narrows the repair to authored request addressing/provenance and recognition reconciliation; it does not justify adding a synthetic recognized feature or calling the drawing complete. Existing #964 covers the wider script-fidelity contract.
PR #1883 is the focused checker correction from the GRM03 source audit. It resolves location requests through compiled exact IDs (including scheduled table cells), never static feature.parameters(), and requires each live claim. On the changed-source GRM03 generated script the safety verdict still fails, but now names the two exact missing carriers location_off_axis.location.member.0.y / .z instead of the misleading request_has_no_matching_parameter. This is honest-failure progress, not a script-parity pass; eight recognition-audit unverifiable outcomes and the envelope-owner representation difference remain.
Correction/interpretation of the GRM03 generated-Sheet holdout: the scripted drawing is a declared front door (schema 8), not a raw-recognition drawing (schema 3). Calling its public requirement_snapshot() raises ReportUnavailableError: accepted occurrence ownership is unavailable for this drawing; raw automatic recognition is required by report schema version 3. Thus its 8 recognition-audit unverifiable counts cannot be equated with 8 lost rendered annotations or required as numerically equal to the direct raw-recognition 15/15 ledger. The equal visible inventory/drops/lint remains useful, and the two authored hole-location requests truly have approved compiler IDs but no live carriers (PR #1883 reports them precisely). Future #1831 parity should compare the generated declaration/model/compiled intent and actual carriers using the existing #964 round-trip contract, while keeping raw occurrence coverage explicitly unavailable for the scripted front door. Do not manufacture recognition ownership or turn the missing authority into a candidate-only regression.
One missing offline GRM03 baseline-script comparator is now complete at the same saved source 0f17848 and caller A3/5:1. This was a single new baseline Sheet build (15.11 s including generation); the already-completed candidate Sheet worker was not rerun. Both scripted modes produced the same 18 annotation identities/types/labels/views/measurement claims/owners, with zero missing/added/changed semantic carriers, equal declared-front-door audited coverage (7/15 placed, 8 unverifiable), and no recorded drops or blocker identities. Baseline quality key is [0,0,0,0,0,-5,124740] and its views are front/side/iso; the saved candidate script chose legacy-depth, with direct-versus-script views equal. This establishes a narrow authored-script semantic tie for GRM03, not a candidate visual win, independent safety pass, or raw-recognition coverage parity. The two authored off-axis hole-location requests have approved compiler IDs but no live glyph in either script mode; #1883 improves their exact diagnostic. Larger/user CAD and broader authored workflow evidence remain open.
Versioned evidence is now in stacked PR #1884, following #1883. It records the one missing offline GRM03 baseline generated-Sheet worker against the saved same-source candidate script: semantic annotation inventory ties exactly at caller A3/5:1. It explicitly keeps declared-front-door raw occurrence coverage unavailable and the shared two location-carrier gaps visible. This is one authored-script holdout case, not closure of #1831 or #1813.
Code-path audit at current main be97376 (no CAD rerun): generated Sheet mirrors compiler addressable() off-axis hole-location requests in sheet_emit._mirrored_requests, and _compile_off_axis_hole_locations approves the Y/Z member coordinates. The renderer then filters coaxial turned-axis holes in annotations/holes.py::_locate_off_axis_holes before either phase, without attaching those exact approved IDs to the centreline or recording a suppression. The saved GRM03 holdout shows centerline_front has no measurements/satisfactions/owners in both direct and scripted manifests, while the script asks for the two location IDs. This explains the common 2-ID authored-claim gap more specifically; it is shared by baseline and candidate, so not a candidate-only regression or a reason to rerun baseline. A fix must decide semantically whether those explicit generated requests should be omitted as redundant or represented by a verified axis relation; do not simply credit an unowned centreline. #1831 remains open.
PR #1887 adds a targeted --case selector to the offline corpus runner. It validates IDs before CAD, runs only named manifest cases in manifest order, records the partial denominator, and refuses corpus-level gates on a subset. Focused tests pass; CI is pending. This prepares a bounded missing-case holdout without rerunning the five completed historical workers or the pathological flat-across-cylinder baseline. It is tooling only, not new candidate admission evidence; #1831 stays open.
Closing by the agreed issue triage, not as a claim that the full proposed AP203/user-part/authored-Sheet holdout matrix was run. We have bounded, versioned evidence (including the GRM03 generated-Sheet comparison in #1884 and the CTC01 AP203 NTS correction merged in #1888), with no identified candidate-only semantic regression in those cases. The two GRM03 location-carrier gaps remain shared baseline/candidate debt and are not certified away here. #1813 remains the rollout decision tracker; a concrete candidate-specific regression or a newly required holdout can be filed and assessed on its own merits. No expensive baseline rerun is implied by this closure.
Activity
pzfreo commentedon Sep 25, 2026
Holdout progress (not a gate pass): authored
Sheetand generated-script candidate-preview semantic comparisons both passed locally in draft PR #1835, with the same fixed A4 2:1 caller constraints. A six-case additional STEP holdout is running once from manifest/source commit74fb65ec(bounded chooser implementatione0a828d0); five baseline/preview SVG pairs are rendered, but the finalif_step_flat_across_cylinder.stepbaseline worker has exceeded 31 minutes of active CPU and no aggregate report exists yet. AP203 CTC siblings will be reported as format variants, not independent geometry wins. No default switch or safety admission follows from these partial results.pzfreo commentedon Sep 25, 2026
Partial AP203 visual holdout review, independent of the still-unfinished six-case semantic aggregate: I rendered the already-produced CTC02 and CTC04 baseline/preview SVG pairs through draftwright PDFium at 100 dpi and inspected full A2 sheets. CTC02 preview still crowds blue dimension/callout text around the upper-left orthographic view and visibly reduces the isometric relative to baseline. CTC04 preview gains a section A–A view, but retains dense stacked dimension bands around the left/lower orthographic views and introduces very long blue horizontal annotation strokes across the lower-right blank corridor; this is not a visual safety pass. These are qualitative page-level findings only; the final flat-across-cylinder worker is still active, so no holdout quality/parity aggregate is claimed. The PDFs were generated under
/tmp/draftwright-holdout-v1-u0R2yF/from the existing SVGs and should be attached as review artifacts when the holdout PR is published.pzfreo commentedon Sep 25, 2026
The first six-case run is now terminal, not merely slow: command exit 143 (SIGTERM), zero-byte aggregate, and no per-case JSON from the pre-#1836 runner. The flat-across-cylinder baseline worker had no SVG after ~51 min wall / ~34 min CPU; candidate for that case never ran. Five other baseline/preview SVG pairs survived, but cannot establish semantic parity or quality verdicts. I did not restart the run. Draft PR #1837 now versions the manifest, exact source/manifest provenance, this failure, and reviewer-accessible CTC02/04 AP203 baseline/preview page images with explicit visual limitations. The broader holdout gate remains open, including novel large user STEP.
pzfreo commentedon Sep 25, 2026
Follow-up runner safeguard is draft PR #1838 (stacked after the interrupted-run evidence): optional per-isolated-worker time bound, typed baseline/candidate timeout failures, nonzero aggregate status, and no false baseline selection or successful latency sample. It has unit coverage only; it was not used for the already-interrupted run, and it does not complete the broader holdout or authorize a rerun of the five rendered cases.
pzfreo commentedon Sep 25, 2026
A new independent public real-part holdout is in progress, without rerunning the fixed15 or the five already-rendered AP203 cases: NIST FTC-09 AP242, 5.83 MiB, one imported solid (~227 × 279 × 5.25 mm), source SHA-256
f1215fe15a78085a9fa78dd81714caf774b59071be92580b04ebfdc19f52a1bd. NIST permits unrestricted reuse and requests acknowledgement: https://www.nist.gov/ctl/smart-connected-systems-division/smart-connected-manufacturing-systems-group/mbe-pmi-0 . It is distinct FTC geometry, not a CTC AP203 sibling and not a user-supplied model. One fixed A2 1:1 candidate-first comparison has been started with a 600-second bound per isolated worker; a timeout is a failed, censored case, not a pass. Await the actual report before drawing any quality/parity conclusion.pzfreo commentedon Sep 25, 2026
New independent public real-part AP242 holdout: #1839 (stack #1816, draft pending exact-head CI). NIST FTC-09 A2/1:1 candidate-first bounded comparison completed both workers once. Parity passed and rendered pages were byte-identical, but quality tied (0 wins), independent safety failed, and the page remains crowded/clipped; no production selection. This is a 6.1-MB PMI-rich public file, not a physically huge or user-supplied model. The previous AP203 interrupted evidence and authored Sheet tests remain separate. Real/user STEP visual and safety gates stay open; no fixed15 rerun.
pzfreo commentedon Sep 25, 2026
FTC-09 first-run triage (same source/report as #1839; no rerun): candidate and baseline have identical 63 blocker identities. By class: 31 off-axis locations, 9 PMI, 9 slot dimensions, 7 callouts, 4 unsatisfiable placements, 3 circular-channel locations. Quality is also identical: 21 hard layout violations, 11 overlaps, 9 crossings. Most stark fixed-A2 failure: reported plan-view bounds are y=442.90..445.94 mm on a 420-mm-high page, i.e. the entire plan view is off-sheet; the side view is only ~3.04 mm wide. This is an absolute page-fit/legibility defect in both modes, not an introduced candidate regression or a verified gain. It needs an explicit fixed-constraint resolution or honest gated limitation before #1813 rollout.
pzfreo commentedon Sep 25, 2026
Safety-gate design note from current stack top #1848: schema-8 declared Sheet reports expose final-IR declarations/representations and lint, but deliberately have no raw schema-3 recognition requirement ledger. The current candidate gate checks declared inventory and four aspect kinds from live declaration provenance; its own limitations still mark other authored non-dimension parity and generated-script parity incomplete. A holdout or future admission check must verify those obligations from declaration identities/sidecar evidence, not pretend that raw recognized requirement counts exist on replay. Existing candidate-preview authored/generated-script parity tests cover a small Box/hole/datum case only; they do not establish broader user-part parity or production admission. No test or corpus rerun for this note.
pzfreo commentedon Sep 25, 2026
Holdout-gate cross-check: the FTC-09 report's off-sheet plan view is already addressed at the independent safety boundary by stacked PR #1840 (source 29ef1fe):
view_page_containmentcompares each settled view to the caller's page, and focused tests cover off-page/non-finite bounds. The older FTC-09 report predates that check, so it must not be read as proof that the current gate passes the case. This fixes false safety reporting, not the off-sheet layout itself; no slow FTC-09 worker was restarted.pzfreo commentedon Sep 25, 2026
Additional visual review of the five preserved AP203/real-part SVG pairs from the interrupted first holdout (no CAD rerun): the previously recorded CTC02/04 concerns remain; I rendered and inspected the other three pairs at page scale. CTC01 AP203 candidate visibly relocates the long 2×R20 callout to a shorter upper-plan route and retains the same apparent views/content, but the tight right orthographic callouts still need semantic/legibility checking; this is a local visual gain, not an aggregate win. GRM03-no-PMI baseline and candidate SVGs are byte-identical (SHA-256 b7da93492e65547cd721f7bfe19f5e1829985d1a277f449fa35618891cc461be), so this is a visual tie. Single-cylinder A3/1:1 looks nearly identical in both modes, with a tiny part on a large caller-fixed sheet; that sparsity is not a recognition or safety pass. The old runner's report.json is zero bytes, so none of these observations establishes requirement parity, a quality verdict, or completion of #1831. The local rendered pairs remain under /tmp/draftwright-holdout-v1-u0R2yF; no slow worker or paired corpus was restarted.
pzfreo commentedon Sep 25, 2026
New independent public-part holdout started once at stack head c5d87ad (no fixed15/paired rerun): NIST FTC-08 AP242, archive SHA-256 8fa78429e6d8d9b0d7681d223b6aa9ec98c3772185c55b1a0e3679b21c181911, member nist_ftc_08_asme1_ap242-e2.stp SHA-256 7b9283b4288d152fd181d76c39285d7c79fbba34fda1e14a41ac4a983f6d68b3 (4,722,264 bytes). Import-only preflight gave a ~311.607×222.707×48.260-mm compound with one MANIFOLD_SOLID_BREP record. The caller manifest SHA-256 is 772298bef0d1696a35f87a429bef870b0ee5c5471f44150ee3d6630763bf7e21: fixed A2/1:1, staggered side, exterior dimensions, SVG. Each isolated baseline/candidate worker has a 600-second bound, one worker at a time. An initial invocation failed before CAD import because the worktree runner selected system Python without build123d; the corrected invocation uses the installed venv and its baseline worker is actively using CPU. No quality, parity, cost, or safety conclusion yet. NIST permits unrestricted use of these files and requests acknowledgement: https://www.nist.gov/ctl/smart-connected-systems-division/smart-connected-manufacturing-systems-group/mbe-pmi-0 . This is not a user-supplied model or a complete large-CAD gate.
pzfreo commentedon Sep 25, 2026
FTC-08 public-part holdout completed once and is now recorded in stacked draft PR #1856 (source c5d87ad; full report SHA-256 878da76d30143a3729c09e0aa94fff53e4355c6540aae10960a1e06f395a1dbd). Fixed A2/1:1 candidate-first chose columns; baseline/candidate SVGs are byte-identical, offline quality tied, and semantic parity passed with no missing annotations/new blocker identities/new interior dimensions. Independent safety failed six checks, including side-view page containment: x-max 659.65 mm on a 594-mm-wide sheet. Coverage is 100/131 placed, 24 dropped, 2 unsupported, 5 unverifiable. Both modes share this absolute defect; it is not a candidate regression or a verified win. Linux process times were baseline 240.41 s/candidate 211.89 s, RSS 657.47/655.16 MiB, one sample only. Source STEP contains datum/tolerance entities, but rendered PMI coverage remains unresolved. User-supplied large CAD and the broader holdout gate remain open; no fixed15 or successful worker was rerun.
pzfreo commentedon Sep 27, 2026
Current-source FTC09 holdout correction: three-platform run https://github.com/pzfreo/draftwright/actions/runs/36330010666 at head 80becbb (same pinned FTC09 A2/1:1 manifest) is no longer the older documented tie. Linux, macOS, and Windows each completed both workers and agree: candidate-first selected columns, retained page/scale/views, but dropped baseline PMI source geometric_tolerance:0:1:4:32 and the envelope width.length dimension 190.5. Exact semantic parity therefore fails on all three; compare exit 1 is correct. This is a candidate-specific regression to diagnose from the saved reports before any slow rerun, not a shared absolute defect and not a reason to repeat the passed fixed15 work. The older FTC09 report remains valid only for its recorded source b160f2e. The broader user/large-part holdout is still open.
pzfreo commentedon Sep 27, 2026
Follow-up to the three-platform FTC09 regression: PR #1879 at b8108cd restores the two candidate-only lost carriers using a pre-render fixed-page profile choice. One changed-source candidate worker (no baseline rerun) produced a full manifest with zero missing/added annotations, equal coverage and drops, and no introduced blockers against the saved Linux baseline artifact from run 36330010666; the SVG is byte-identical to that saved baseline for this case. This is a relative tie, not an absolute safety pass: FTC09 still has off-page views/ink and other shared completeness failures. The comparison is cross-source diagnostic evidence; exact-head supported-platform evidence remains for #1832 after merge.
pzfreo commentedon Sep 27, 2026
The exact-source FTC09 three-platform holdout run 36332797155 at b5a9ced restores full candidate/baseline semantic parity after #1879: no missing/added annotations or introduced blocker identities on Linux, macOS, or Windows; quality and within-platform SVGs tie. Candidate-first chooses iso-growth pre-render because columns would push the plan a further 6 mm off-page. The shared absolute off-page/completeness failures remain explicit, and this public FTC09 case is not a user-supplied large-CAD holdout or broader authored-Sheet proof. #1831 remains open.
pzfreo commentedon Sep 27, 2026
Candidate-only GRM03 direct-vs-generated-Sheet holdout on main 0f17848 (plain grm03_thumbwheel_drive_screw.step, caller A3/5:1, candidate-preview; no baseline or corpus rerun): both builds choose legacy-depth, same page/scale/views, same 18 annotation names/labels, same drops/blocker identities and empty lint. However the exact drawing_record manifests are not equal: direct detection reports coverage 15/15 placed; generated authored Sheet reports 7/15 placed and 8 unverifiable, and independent safety fails authored_dimensions/audited_coverage (direct fails recognized_carriers). The one annotation ownership difference is dim_height: direct compiler mints a bbox-envelope measurement identity without adding a recognized EnvelopeFeature; emitter intentionally declares a synthetic envelope so the authored height is nameable, giving dim_height owner feature index 8. This is not a demonstrated candidate-vs-baseline regression or a visible-ink mismatch. It is a front-door provenance/coverage mismatch that the current GRM03 generated-script parity test does not catch because it compares annotation signatures/views/page/scale/lint, not exact ownership and requirement states. The synthesized envelope is deliberate in sheet_emit.mirror_model and compiled._compile_overall_height; do not add it to detected model.features merely to make reports equal. I am tracing the eight authored unverifiable claims under the existing script-fidelity boundary (#964) before proposing a code change. #1831 remains open.
pzfreo commentedon Sep 27, 2026
Further code-level diagnostic of the saved GRM03 scripted holdout: layout_safety._authored_dimension_gaps finds exactly two generated hole location requests (Y and Z, member 0) with request_has_no_matching_parameter. These are distinct from the eight recognition-audit unverifiable counts; the latter come from the separate recognition-owned completeness path, not from the authored request checker. The script declares both locations, and the visible annotation inventory still matches direct. This narrows the repair to authored request addressing/provenance and recognition reconciliation; it does not justify adding a synthetic recognized feature or calling the drawing complete. Existing #964 covers the wider script-fidelity contract.
pzfreo commentedon Sep 27, 2026
PR #1883 is the focused checker correction from the GRM03 source audit. It resolves location requests through compiled exact IDs (including scheduled table cells), never static feature.parameters(), and requires each live claim. On the changed-source GRM03 generated script the safety verdict still fails, but now names the two exact missing carriers location_off_axis.location.member.0.y / .z instead of the misleading request_has_no_matching_parameter. This is honest-failure progress, not a script-parity pass; eight recognition-audit unverifiable outcomes and the envelope-owner representation difference remain.
pzfreo commentedon Sep 27, 2026
Correction/interpretation of the GRM03 generated-Sheet holdout: the scripted drawing is a declared front door (schema 8), not a raw-recognition drawing (schema 3). Calling its public requirement_snapshot() raises ReportUnavailableError: accepted occurrence ownership is unavailable for this drawing; raw automatic recognition is required by report schema version 3. Thus its 8 recognition-audit unverifiable counts cannot be equated with 8 lost rendered annotations or required as numerically equal to the direct raw-recognition 15/15 ledger. The equal visible inventory/drops/lint remains useful, and the two authored hole-location requests truly have approved compiler IDs but no live carriers (PR #1883 reports them precisely). Future #1831 parity should compare the generated declaration/model/compiled intent and actual carriers using the existing #964 round-trip contract, while keeping raw occurrence coverage explicitly unavailable for the scripted front door. Do not manufacture recognition ownership or turn the missing authority into a candidate-only regression.
pzfreo commentedon Sep 27, 2026
One missing offline GRM03 baseline-script comparator is now complete at the same saved source 0f17848 and caller A3/5:1. This was a single new baseline Sheet build (15.11 s including generation); the already-completed candidate Sheet worker was not rerun. Both scripted modes produced the same 18 annotation identities/types/labels/views/measurement claims/owners, with zero missing/added/changed semantic carriers, equal declared-front-door audited coverage (7/15 placed, 8 unverifiable), and no recorded drops or blocker identities. Baseline quality key is [0,0,0,0,0,-5,124740] and its views are front/side/iso; the saved candidate script chose legacy-depth, with direct-versus-script views equal. This establishes a narrow authored-script semantic tie for GRM03, not a candidate visual win, independent safety pass, or raw-recognition coverage parity. The two authored off-axis hole-location requests have approved compiler IDs but no live glyph in either script mode; #1883 improves their exact diagnostic. Larger/user CAD and broader authored workflow evidence remain open.
pzfreo commentedon Sep 27, 2026
Versioned evidence is now in stacked PR #1884, following #1883. It records the one missing offline GRM03 baseline generated-Sheet worker against the saved same-source candidate script: semantic annotation inventory ties exactly at caller A3/5:1. It explicitly keeps declared-front-door raw occurrence coverage unavailable and the shared two location-carrier gaps visible. This is one authored-script holdout case, not closure of #1831 or #1813.
pzfreo commentedon Sep 27, 2026
Code-path audit at current main be97376 (no CAD rerun): generated Sheet mirrors compiler addressable() off-axis hole-location requests in sheet_emit._mirrored_requests, and _compile_off_axis_hole_locations approves the Y/Z member coordinates. The renderer then filters coaxial turned-axis holes in annotations/holes.py::_locate_off_axis_holes before either phase, without attaching those exact approved IDs to the centreline or recording a suppression. The saved GRM03 holdout shows centerline_front has no measurements/satisfactions/owners in both direct and scripted manifests, while the script asks for the two location IDs. This explains the common 2-ID authored-claim gap more specifically; it is shared by baseline and candidate, so not a candidate-only regression or a reason to rerun baseline. A fix must decide semantically whether those explicit generated requests should be omitted as redundant or represented by a verified axis relation; do not simply credit an unowned centreline. #1831 remains open.
pzfreo commentedon Sep 27, 2026
PR #1887 adds a targeted --case selector to the offline corpus runner. It validates IDs before CAD, runs only named manifest cases in manifest order, records the partial denominator, and refuses corpus-level gates on a subset. Focused tests pass; CI is pending. This prepares a bounded missing-case holdout without rerunning the five completed historical workers or the pathological flat-across-cylinder baseline. It is tooling only, not new candidate admission evidence; #1831 stays open.
pzfreo commentedon Sep 27, 2026
Closing by the agreed issue triage, not as a claim that the full proposed AP203/user-part/authored-Sheet holdout matrix was run. We have bounded, versioned evidence (including the GRM03 generated-Sheet comparison in #1884 and the CTC01 AP203 NTS correction merged in #1888), with no identified candidate-only semantic regression in those cases. The two GRM03 location-carrier gaps remain shared baseline/candidate debt and are not certified away here. #1813 remains the rollout decision tracker; a concrete candidate-specific regression or a newly required holdout can be filed and assessed on its own merits. No expensive baseline rerun is implied by this closure.