Repository navigation
Description
Part of the code-health epic. Evidence pinned to main @ 71b8c94.
Problem
- Issue-named modules: 151 of 403 test modules are
test_issue_*. They hold 63k of 156k test lines and 36% of test functions._MAX_ISSUE_MODULES = 151stops new ones but does not reduce the existing set. - Clone budget blind spot:
CLONE_BUDGET = 0passes only because_MIN_GROUP_MEMBERS = 3. At a threshold of 2, the existing scan finds 59 cross-module clone pairs (118 tests). Examples:test_add_dimension.py/test_sheet_identity_invariant.py, andtest_circular_blind_step_semantics.py/test_paired_ramp_semantics.py. - Fixture reuse: 304 of 403 modules do real builds, but there are only 2 session-scoped fixtures.
test_sheet_emit.py(4,815 lines) has 63 build call sites andtest_declare.pyhas 74. - Repeated names: 61 test function names appear in more than one file.
- STEP fixtures: two whistle-frame STEP fixtures of about 18–20k lines each were added this month; check whether both are needed.
- Possible caching bug: in one run,
_clone_groups()appeared to ignore a changed threshold. Confirm whether it is cached.
Proposal
- Turn the issue-module cap into a ratchet that goes down, for example by N per release. Fold the largest issue-named modules first (
test_issue_1143_hole_completeness.py, 2,873 lines) into behaviour modules. - Look at the 59 clone pairs. Parametrise the real clones, then decide whether to lower
_MIN_GROUP_MEMBERSto 2. - Move the common real-part builds into shared session- or module-scoped fixtures, starting with
test_sheet_emit.pyandtest_declare.py.
Progress
-
Consolidate test health and repair table failure tests (#1935) #1981 merged: the issue-named module count and enforced cap fell from 151 to 150 by moving two zone-label ink regressions into their behavior-owned lint module.
-
A class-scoped fillet drawing replaced three repeated builds; the selected three-test median fell from 2.959 to 2.319 seconds locally.
-
Seven stale baseline tests were repaired at their current seams and assertions. The full-suite audit on main
82a86b70recorded 9,727 pass, 19 fail, 3 error before these repairs; the remaining baseline and the clone-pair/fast-tier exit checks are still open. -
Extract slot rendering into focused annotation owner #1995 moved 13 signed-pad rendering tests into the behavior-owned
test_pad_rendering.pywhile extracting slots; the issue-named cap and clone-pair/full fast-tier exit checks remain open. -
Repair STEP inspection executed-module guard for IR foundation #2001 merged: the exact STEP-inspection executed-module guard now includes the foundational IR owner introduced by Extract foundational IR records into focused rank-zero owner #1993; its 49 inspection tests and CI pass.
-
Share expensive requirement-dimension test fixture (#1935) #2092 is open and awaiting CI, then merge after the current dependent stack: the The same feature is dimensioned twice on one sheet #1599 requirement-sharing cases moved into a behavior-owned module with one module-scoped whistle-frame build. The enforced issue-module cap falls from 150 to 149; the warm two-build comparison was 25.88 s before and 13.31 s after. The strengthened provenance assertion and suite-shape guard fail under targeted mutations; 72 focused tests, static checks, and independent review passed. Full fast-tier time, clone pairs, and coverage acceptance remain open.
-
Consolidate duplicated recognition-refusal test cases (#1935) #2094 is open and awaiting CI: the groove/pad recognition-refusal cases share an existing parameterized evidence helper, reducing two-member cross-module clone pairs from 53 to 52. Fixture preconditions and targeted mutations are load-bearing; focused/static checks and independent review passed. Broader clone-pair and fast-tier exit checks remain open.
-
Repair lazy STEP evaluator import guard baseline #2097 is open and awaiting CI: an unchanged-main failure in the lazy STEP evaluator import guard was repaired by listing all current engine-free evidence helpers and inspecting eager imports in module control flow and class bodies. Direct/conditional/class import mutations fail the guard; function-local imports remain allowed. Five focused tests, static checks, and final independent review passed. This repairs one fast-tier baseline failure; the full-tier and coverage acceptance remain open.
Acceptance
- The issue-module count and the full fast-tier wall time both go down measurably.
- No coverage is lost, checked with
fail_under.
2026-09-30 current status
The merged #2104 tree has 147 test_issue_* modules by rg --files tests | rg 'test_issue_.*\\.py$' | wc -l; the cap continues to ratchet down. A separately reviewed local follow-up (75166720, integrated after the next unpublished aggregate as 80f52f1b) shares one partial declared drawing across two model-seam tests. Its volume precondition failed under a one-cutout mutation, 42 focused recognition/suite/clone checks passed, and the integrated class passed 15 tests. This avoids one real build but has no demonstrated full-tier wall-time reduction yet. The issue's full fast-tier timing and no-coverage-loss acceptance remain open; do not count this local slice as merged.
The next local follow-up 991cf8a9 folds all four #1390 Sheet registration tests (nine parameter cases) into test_sheet_identity_invariant.py and ratchets the issue-module cap 147→146. Test bodies and decorators matched by AST; the destination, suite-shape, and clone-budget modules passed 195 tests, and fresh independent review found no blocking issue. This is not merged, and it does not establish the required full-tier timing reduction.
2026-09-30 fixture experiment discarded
An isolated test_sheet_emit.py class-fixture candidate (2d470371) saved one direct default plate build_drawing across two tests. The selected warm pair measured 5.41/5.38 s before and 5.09/5.08 s after; 17 parity tests, repository guards, static checks and a precondition mutation passed. Fresh independent review found the shared mutable Drawing and teardown guard added 21 test lines for only about 0.3 s on that pair. Under xdist worksteal the two tests may run on different workers and build separate fixture instances, so there is no established full fast-tier reduction. The candidate is discarded and not part of the frozen aggregate; #1935 remains open. A separate behavior-module consolidation of rectangular and round-bottom blind-slot evidence tests is being assessed to address the clone-pair and issue-named-module gaps without adding shared mutable build state.
2026-09-30 local blind-slot test consolidation
A reviewed isolated fold, integrated with the #1932 helper slice at unpublished f8be6b91 in /private/tmp/dw1939-followup-local, replaces two #1421 issue-named completeness modules with behavior-owned test_blind_slot_completeness.py. The two seven-case physical corpora and all 32 test invocations remain independent and collected; four deliberate family-specific mutations failed named tests. Test source falls 1,228→729 lines; the diagnostic cross-module two-member clone count falls 50→40, and the enforced issue-named module cap 146→144. Fresh integrated review caught stale module references and a required alphabetical capability-evidence order; both were fixed, the two affected modules passed 168/168, and exact-tip review found no P0–P3 issue. The exact-head full fast tier passed 9,938, 9 skipped, 15 xfailed with 96.03% coverage; static and 100% changed-line coverage passed; 14 CAD plus three STEP drawing/lint/decision/trace signatures match the frozen parent byte for byte. Five paired workloads showed no reproducible build regression (the noisy blind-hole two-pair 1.013 ratio became 1.001 over eight fresh pairs). Full fast-tier wall time was 440.38 s, versus 435.16 s in the earlier frozen-parent run; these single runs do not prove the required measurable full-tier reduction. #1935 stays open. The earlier one-build class-fixture experiment remains discarded and is not in this candidate. No PR/CI has started because GitHub branch writes are rejected by the current approval policy.
2026-09-30 local blind-slot semantics fold
The third dependent local unit 636a34d4 folds both #1421 issue-named semantics modules into tests/test_blind_slot_semantics.py. It keeps all 90 collected invocations and explicit family-specific physical, IR, page, declaration, rendering, and error-text expectations while removing 406 net test lines. Diagnostic two-member cross-module clone redundancy falls 40→31, and the enforced issue-named-module cap 144→142. Independent review caught one weakened family-specific error-text assertion; the final commit restored both exact patterns, and 34 invalid-IR cases passed. Exactly 90 stale pytest-split duration keys were replaced by the 90 measured new node IDs. The integrated tip passed static, 9,938 fast-tier tests, 96.02% total and 93.7% changed-line coverage, plus 17-case exact drawing/lint/decision/trace parity; fresh final aggregate review found no P0–P3 issue. A single full-tier run took 434.49 s versus the parent's earlier 440.38 s, an unpaired comparison that does not prove a measurable wall-time reduction. #1935 remains open. No PR/CI or merge because GitHub branch creation remains rejected by the session approval policy.
2026-09-30 full-tier time opportunity audit
Read-only follow-up inspected repeated mirror-corpus builds and full-suite scheduling. The module fixture eagerly builds 33 drawings (~14.71 s in a fresh local process), so a 35-test targeted -n 12 --dist worksteal run scatters its cases across workers (35 passed in 29.88 s). Exact-head full collection places those consumers together at indices 8,470–8,510 of 9,962 items, and CI loadscope groups the class. A lazy per-case fixture would therefore save no builds in the expected full tier; the candidate was not implemented. Other repeated-build opportunities were smaller or risked sharing mutable Drawing state. The committed .test_durations still contains 1,446 keys for absent files (1,329.39 of 4,666.67 recorded seconds), measured by checking each key path; refreshing that data may rebalance CI split shards but would not establish the issue's full fast-tier wall-time reduction. No code or timing-acceptance claim followed this audit.
2026-09-30 measured CI shard-duration refresh
Dependent local unit fb1a74d9 in /private/tmp/dw1939-fourth-local refreshes .test_durations from a 12-worker fast-tier run (9,938 passed, 9 skipped, 15 xfailed). The old file weighted 5,013 of 9,962 current fast cases; the new file weights all 9,962, removes 1,446 missing-file entries, and keeps existing-file timings outside this selection. On those same macOS measurements the old two-shard assignment held 2,452.94/2,023.31 seconds of test work; the refreshed assignment holds 2,239.59/2,236.66, confirmed by pytest-split collection. This is estimated worker balance, not measured Linux CI wall-time improvement or proof of #1935's separate full-tier speed acceptance. The combined unit also contains reviewed #1937 documentation only; exact-head static and fresh independent review passed. No PR, CI or merge while GitHub branch creation is rejected by the current approval policy. #1935 remains open.
2026-09-30 exact-head build-cost profile
After the four-unit local aggregate fb1a74d9 was packaged, a fresh exact-head uv run pytest --collect-only -q selected 9,962 fast cases. scripts/profile-builds --expect-collected 9962 ran with 12 local workers and wrote all 12 complete worker profiles. The instrumented suite returned nonzero (9,930 passed, 8 failed, 9 skipped, 15 xfailed) because the profiler deliberately wraps build_drawing: five tests assert sheet.build_drawing is builder.build_drawing, two assert the warning caller filename rather than the wrapper, and one asserts dependencies live under this worktree's .venv while the run borrowed the adjacent checked environment. The earlier uninstrumented exact executable-code head passed 9,938. Treat this profile as diagnostic evidence, not a green test gate.
The profile recorded 5,824 builder._build_drawing_once calls and 3,314 worker-seconds across the selected suite. The most expensive module on the critical local worker was test_pmi.py (~206 phase seconds); its read-only CTC fixture is already module-scoped, while other heavy PMI cases have distinct modes/inputs. Two annotation-box corpus tests each rebuild the same five sizing models (~30 worker-seconds apiece); a module-scoped model-only fixture could save about 30 worker-seconds and retain independent assertions, but both cases lie in the currently near-balanced first CI shard, and a 12-worker local run has only ~2.5 s ideal throughput gain, below observed run noise. This is not a demonstrated full-tier wall-time reduction, so no fixture edit was made. The invalid strict-xfail arrangement demonstration (~54 s and 22 build attempts) is being checked separately for unique ADR 2 coverage before any removal. #1935 remains open.
2026-09-30 obsolete arrangement xfail removed locally
An exact-head --runxfail execution of test_re_deriving_per_stage_instead_loses_requirements_on_an_unchanged_sheet completed both builds then failed at its intended final assertion because the perturbed and baseline lint-code sets were equal; the strict xfail did not prove its stated ADR 2 requirement loss. The preexisting source reason admitted this. The reviewed local fifth unit 99c59d07 deletes that invalid ~54.19-second duration entry and its fixture/obsolete prose, retaining the passing placement, forced-repack and requirement-loss guards. The enforced issue-module cap and clone-pair census stay 142 and 31. Focused/static checks and an independent review passed; exact-head full fast tier passed 9,938, 9 skipped, 14 xfailed, with 96.03% total coverage (earlier parent 96.02%) and 97% combined changed-line coverage against the merged-equivalent base. All 9,961 collected fast cases have duration weights; two-shard local estimates are 2,217.72/2,204.34 worker-seconds. One unpaired full-tier run was 467.87 s, slower than an earlier 434.49 s parent run, so no measurable wall-time reduction is claimed. The deletion is valid test cleanup and removes 22 instrumented build attempts, but #1935's speed acceptance remains open. It is packaged into the one-PR local aggregate; no PR/CI/merge under the continuing branch-write approval block.
2026-09-30 paired fast-tier and PMI fixture follow-up
A same-command 12-worker local fast-tier comparison showed a cold #2104-equivalent base run of 454.16 s, a warm repeat of 394.82 s, and the reviewed five-unit unpublished bundle at 393.85 s. The apparent 60-second first-pair saving was a cold-run effect; the required measurable full fast-tier wall-time reduction is not established.
A fresh read-only review found one existing-fixture reuse in tests/test_pmi.py. The isolated sixth local unit a2588358 consumes the module-scoped read-only CTC-01 pmi='annotate' drawing in the declared source-identity test instead of rebuilding the same automatic input. Under the same two-node warm selection, the test call measured 38.18 s before versus 5.04 s after (suite 72.80→38.84 s); direct builds under the old and fixture title/number options produced the same nonempty 31 source IDs. The changed head passed static, a 9,938-pass uninstrumented fast tier, and a separate 9,938-pass coverage run at 96.03% total; fresh independent review found no P0–P3 finding. The one duration key changed 54.826→5.04 s, and all 9,961 current fast cases have weights. A single uninstrumented full-tier run on the sixth unit was 385.52 s; with the warm base at 394.82 s and five-unit head at 393.85 s, this does not prove a reproducible full-tier wall-time gain. #1935 remains open. The unit is included only in the local single-gate aggregate /private/tmp/dw1939-one-gate-aggregate-v3.patch; no PR, CI or merge has occurred under the GitHub branch-write approval block.
2026-09-30 annotation-box corpus fixture experiment discarded
A fresh green uninstrumented 12-worker worksteal duration run on the six-unit local head a2588358 passed 9,938 tests in 368.51 s and found two test_annotation_box_corpus.py methods at 30.69 and 30.32 call-seconds. Each independently built the same five-model corpus. A first attempt to combine the assertions into one test was discarded after independent review found it suppressed the structure contract whenever an earlier footprint-identity assertion failed. A revised class-scoped model-only fixture retained both original test IDs, all assertions and n_steps cases, and asserted the five corpus labels before use. Fresh independent review found no correctness issue or model mutation in the exercised compose paths; targeted loadscope stored 21.0305 s for the first test and 0.00067 s for the second, and exact pytest-split collection assigned both to shard 1. Targeted worksteal instead placed the two tests on separate workers and built the fixture twice.
The full-tier acceptance still failed to show a gain: candidate worksteal passed 9,938 in 374.63 s versus the parent's 368.77 s diagnostic run; same-command uninstrumented loadscope passed 9,938 in 416.67 s on the parent versus 418.96 s on the candidate. These single pairs cannot show a reproducible improvement, and the new fixture adds shared mutable PartModel state plus eleven lines. The candidate remains isolated at /private/tmp/dw1935-annobox-fold and is excluded from the one-gate aggregate; no PR, CI, merge or acceptance claim followed. This is a measured reason for declining the duplicate-build cleanup despite the targeted saving. #1935 remains open.
2026-09-30 green worker-tail diagnosis
A fresh, uninstrumented pytest -vv -n 12 --dist worksteal --disable-warnings run on the six-unit local head a2588358 passed 9,938 tests, 9 skipped, 14 xfailed in 368.40 s. Its timestamped controller log is /private/tmp/dw1935-worker-tail.log; a parser matched all 9,961 selected result lines to 12 worker IDs. Their final completions clustered from 363.1 to 368.2 s, only 5.1 s apart. The two annotation-box corpus tests took ~30.8 and ~30.3 s on gw0, but worksteal kept all workers busy and near-balanced. Removing one five-build corpus would save ~30 worker-seconds, whose ideal 12-worker throughput effect is about 2.5 s, below the observed full-run variation; the separate fixture experiment's no-gain result is therefore expected. Other long tests (PMI 47/55 s, an angular CTC case 47 s) were distributed among workers rather than creating one dominating tail. Future #1935 speed work should target a larger aggregate reduction in build work or measured scheduling overhead, then verify a paired full tier; additional one-off shared mutable fixtures are unlikely to establish the stated wall-time acceptance. No code or duration data changed in this diagnosis, and #1935 remains open.
2026-09-30 epic-baseline time and coverage comparison
The original epic baseline 71b8c945 collected 9,616 fast cases. Its full 12-worker worksteal run took 471.93 s of pytest time (472.23 s wrapper wall) but had 26 failures and 3 errors. Rerunning the same baseline with exactly those 29 failing/error nodes deselected gave a green 9,564-pass/9-skip/14-xfail subset: 483.28 s without coverage, and 500.42 s with coverage at 95.69% (90% fail_under passed). The exact unpublished seven-unit head 7552b9d2 ran the full current 9,961-case fast selection: 9,938 passed/9 skipped/14 xfailed in 368.40–385.52 s across uninstrumented local runs, and 403.13 s with coverage at 96.02% (90% gate passed). Thus even the conservative green old subset was materially slower, despite selecting 374 fewer cases; whole-package coverage also rose 0.33 percentage points. This is local macOS evidence across the epic, not a Linux CI measurement or a gain attributable solely to #1935. The issue-named module count is 151 at baseline, 147 on merged #2104 and 142 in the unpublished bundle. The measured local acceptance is supported, but this issue stays open until the bundle is published and current-head required CI passes; the old baseline's 29 exclusions must remain visible in any completion claim.
Activity
Fold zone-label ink regression into lint coverage (#1935)
Fold zone-label ink regression into lint coverage (#1935)
pzfreo commentedon Sep 28, 2026
Full-suite baseline audited on main
82a86b70withuv run pytest -n 16 --dist worksteal(397 s): 9,727 passed, 19 failed, 3 errors, 9 skipped, 14 xfailed. The 22 failing/error node IDs exactly match the previously recorded baseline; log:/tmp/draftwright-1935-baseline-82a86b70.logon the execution machine.Cause triage: 7 stale test seams/fixtures (including 3
drawing.fit_boxreferences corrected on the local #1981 test-health branch and verified 3/3 targeted), 3 exact geometry snapshot mismatches, and 12 potential production/layout semantics requiring separate targeted review. The latter are not being relabeled as stale tests without proof. The #1981 PR will use the normal required CI gate because the known full-suite baseline remains red; this audit preserves the failure evidence while the focused test cleanup can merge.pzfreo commentedon Sep 28, 2026
Follow-up from independent #1990 review: the new
hole_locations.pyPR-tier group covers focused location, vocabulary and lint evidence tests, but future source-only changes do not yet selecttest_issue_1518_explicit_rear.pyortest_issue_1143_hole_completeness.py. The latter currently contains one of the confirmed staledrawing.fit_boxfailures, fixed on the pending #1981 branch. After that fix merges, consider those broader contracts for the tier (with runtime cost measured) and add a manifest selection guard. #1990 itself passed 98% changed-line coverage, exact 14-case live parity, required CI, and independent review.Fold zone-label ink regression into lint coverage (#1935)
Repair hole-table failure tests at current fit seam (#1935)
test: repair four stale seam fixtures (#1935)
Consolidate test health and repair table failure tests (#1935) (#1981)
Merge pull request #2052 from pzfreo/epic1939/sheet-identity-tests
Merge pull request #2055 from pzfreo/epic1939/pattern-contract-tests
pzfreo commentedon Sep 29, 2026
Next fixture slice: #2101 is open against unchanged
mainand held behind stack #2088 for merge. Eleven tests intest_issue_1143_hole_completeness.pynow share one module-scoped blind-hole build with physical and IR preconditions. Warm eleven-case wall time fell 4.29→2.82 s; the full module passed 104 tests, static/clone/suite guards passed, and a load-bearing assertion mutation failed. A fresh independent review found no P0–P3 findings and confirmed identical executed line/branch coverage for the focused cases. Required GitHub CI is pending. The broader issue-module count, clone-pair, full fast-tier timing, and full coverage acceptance remain open.Integrate reviewed code-health slices with one CI gate (#1939) (#2102)
Converge placement callback state and guard source shape (#1939) (#2103)
Refactor annotation owners and share build-local evidence (#1939) (#2104
Consolidate equivalent lint helpers and evidence tests (#1932, #1935) (…
pzfreo commentedon Sep 30, 2026
Completion evidence on merged
mainb5ba5564via #2107 (17 required CI successes, two expected skips; coverage gate green). Issue-named test modules fell from the epic baseline 151 to 142 (rg --files tests | rg '^tests/test_issue_.*\.py$' | wc -lon merged main). The diagnostic two-member cross-module groups fell from 59 at baseline to 24 (test_clone_budget._clone_groups()with_MIN_GROUP_MEMBERS=2on merged main); #2107 reduced the post-#2105 count from 31 to 24 and removed 142 net test lines while retaining test IDs. The enforced three-member clone budget remains zero. Independent audits found the remaining two-member similarities to be short or distinct API contracts, where forced parametrisation or shared mutable fixtures would add indirection without a sound gain; the threshold stays at 3.The recorded same-command covered 12-worker local fast-tier comparison is a green original-epic-baseline subset (exactly 29 historical failing/error nodes excluded): 9,564 passed in 500.42 s at 95.69% coverage, versus the full reviewed current tier before #2107: 9,938 passed in 403.13 s at 96.02% coverage. The new PR's exact-head CI/coverage checks also passed. Thus the written issue-module, wall-time and no-coverage-loss acceptance measures are met. The comparison is project-level evidence across the epic, not a speedup attributable to #1935 alone; the 29 baseline exclusions and 374 extra current passes are part of the claim. Closing #1935 with those limits explicit.