Skip to content

Ship the trace plugin in the wheel; accuracy.py fails on zero scored - #165

Merged
inth3shadows merged 4 commits into
mainfrom
fix/97-accuracy-and-wheel-plugin
Nov 12, 2026
Merged

inth3shadows merged 4 commits into
mainfrom
fix/97-accuracy-and-wheel-plugin

Conversation

@inth3shadows

Copy link
Copy Markdown
Owner

Two findings from the 2026-11-12 audit.

1. verify failed after pip install testgraph

The wheel shipped testgraph/ only, and verify needs harness/plugin/tgtrace.py at runtime. [tool.hatch.build.targets.wheel.force-include] now copies that one file to testgraph/_plugin/tgtrace.py. harness/ still ships nothing else, so the "no import contract" packaging decision holds, and harness/plugin/ stays the source of truth. plugin_dir() checks the packaged copy first, then the checkout.

From an installed wheel, run from outside any checkout (python -I), against this repo with --dry-run --base HEAD~3:

PyPI 0.1.2 : cannot find harness/plugin/tgtrace.py — it ships with the source checkout, not the installed wheel   exit=2
this branch: ! NO JOURNEY-LEVEL TEST: J2, J1, ... — ... it is not a pass.                                       exit=3

(exit 3 is the expected honest refusal, the same result a checkout gives.)

2. harness/accuracy.py printed S1 VERDICT : PASS on a run that scored nothing (#97)

min_recall starts at 1.0 and WORKTREE-FAIL / INDEX-FAIL / BLOCKED rows continue past it. Now it fails when no commit was scored or any row errored or was BLOCKED (the same bar seed_regressions.py uses), and prints commits scored: N / M.

Verification

  • New tests against unfixed accuracy.py: FAILED (failures=2) (3 tests)
  • Full suite, Python 3.12, pytest 9.0.3, TG_REQUIRE_PYTEST=1: Ran 569 tests in 35.972s / OK
  • uv build --wheel lists testgraph/_plugin/tgtrace.py

Closes #97

- verify failed after `pip install testgraph`: the wheel shipped
  testgraph/ only and the tgtrace plugin lives in harness/plugin/.
  force-include copies that one file to testgraph/_plugin/; plugin_dir
  checks the packaged copy first, then the checkout.
- harness/accuracy.py printed S1 VERDICT : PASS when every commit failed
  to check out or index (min_recall starts at 1.0 and error rows
  continue past it). Fail when nothing was scored or any row errored or
  was BLOCKED (#97).

Closes #97
#165 review:
- `uv build` builds the wheel from the sdist, which had no harness/,
  so the force-include failed and the release would not build. The
  sdist now includes harness/plugin/tgtrace.py, and a CI `build` job
  runs the same `uv build` and checks the wheel for the plugin.
- One PLUGIN_MISSING message for verify and the adapter, naming both
  the installed and the checkout location.
- accuracy.py prints MIN RECALL n/a when nothing was scored.
- USAGE/README/pyproject wording; .gitignore testgraph/_plugin/.
#165 round 2:
- plugin_dir checks harness/plugin before testgraph/_plugin, so a stray
  packaged copy in a checkout cannot shadow the file being edited (the
  .gitignore entry from the last commit only hid it; reverted).
- Test that every wheel force-include source is in the sdist include,
  so the B1 regression fails locally, not only in the CI build job.
- Test the MIN RECALL n/a line.
- Remaining "wheel ships testgraph/ alone" wording (README, registry,
  test_registries) and a pyproject comment pointing the wrong way.
#165 round 3: with the entry gone, a stray copy is swept into the sdist
and collides with the force-include. The runtime shadowing it used to
hide is fixed by the checkout-first lookup (9da5723).
@inth3shadows
inth3shadows merged commit 6e87ce8 into main Nov 12, 2026
6 checks passed
@inth3shadows
inth3shadows deleted the fix/97-accuracy-and-wheel-plugin branch November 20, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

harness/accuracy.py prints S1 VERDICT : PASS when zero commits were scored

1 participant