Skip to content

fix: one traced-run path for verify and the adapter; crash-safe trace writes - #168

Merged
inth3shadows merged 4 commits into
mainfrom
fix/154-trace-followups
Nov 15, 2026
Merged

inth3shadows merged 4 commits into
mainfrom
fix/154-trace-followups

Conversation

@inth3shadows

@inth3shadows inth3shadows commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Fixes #154

What

Every item on #154's list, plus nine defects found while reviewing it:

  • verify --json and pytest_adapter --json print a refusal as {"refused": true, "error": "..."} and still exit 2. Before, they printed prose.
  • verify and the adapter run pytest through one helper, so they refuse a broken run in the same order and with the same words: a missing trace (with pytest's stderr tail), then a truncated trace, then a trace that isn't a tgtrace object.
  • Trace writes survive crashes. tgtrace flushes and fsyncs before its atomic rename. --trace-out now writes the same way instead of with a plain open-and-dump.
  • Stream trace targets (/dev/stdout, pipes, other device paths) are refused before any test runs, with a message saying why. /dev/shm counts as a file location.
  • harness/trace.py reads a broken trace as "no trace" instead of crashing, and removes temp files left behind by killed runs.
  • Test hygiene: tests pass the work directory in, so none of them patch tempfile.TemporaryDirectory or json.dump for the whole process. The adapter's trace tests now live in test_pytest_adapter.py.

How

  • pytest_adapter.run_traced and load_trace are the shared seam. A missing JUnit report is the one place the callers still differ, on purpose. The adapter refuses, because it has nothing to join the trace against. verify reads it as no verdicts, so every selected journey is reported incomplete, never passed. verify's behaviour is otherwise unchanged.
  • One rule decides what a trace file can be: not_a_file_target. It applies abspath, then refuses anything under /dev/ or /proc/ (except /dev/shm/) and any existing non-regular file. tgtrace refuses at pytest_configure, so before any test runs. The adapter refuses --trace-out before the suite starts. harness/trace.py imports the adapter's rule. tgtrace can't import testgraph, so it carries a copy, and WriterParityTests keeps the two copies identical: 11 path cases, plus a child process whose stdout is a regular file. It also checks they use the same <path>.tmp.<pid> temp name.
  • Refused instead of raising: a truncated JUnit report, a trace whose tests/declared entries aren't tgtrace's shape (checked per entry; declared: null stays accepted, as on main), and a registry that's unreadable or not a JSON object. Each of these used to exit 1, which is the code for "tests failed".
  • Checked before the suite runs: --trace-out (directory exists and is writable, target isn't a directory). Before, a bad path only failed after a full run.
  • The harness cleanup only removes <out>.tmp.<digits> files whose process is dead. A broader pattern deleted unrelated files and the temp file of a run still in progress.
  • Tried and dropped:
    • Writing stream targets in place (round-1 fix). A second open("/dev/stdout", "w") truncates the redirected file and gets its own offset, so pytest's output overwrote the JSON: exit 0 with a corrupt trace. The /dev/ prefix rule it used also caught /dev/shm, which is ordinary tmpfs, and that broke harness/trace.py --out /dev/shm/....
    • Checking only isfile. /dev/stdout redirected to a file resolves to a regular file.

Measured (base 6e87ce8 vs this branch, Linux, Python 3.14, pytest 9.0.3)

  • Full suite on the final tree: Ran 600 tests ... OK (571 on base). Each test runs once (no inherited duplicates), and running each file directly matches discover: 47, 38, 48. The real-pytest e2e ran; nothing was skipped.
  • Red before green: the round-1 review tests failed on 2ce4041 (failures=13, errors=11). The round-2 tests failed on 756e9a3 (failures=15, errors=4).
  • Mutation check: I reverted each fix one at a time; each revert turned at least one test red: 7/7 in the first commit, 14/14 in the round-1 fixes, 12/12 in the round-2 redesign.
  • Verdict parity, checked by a reviewer: 60 real-pytest scenarios gave identical verify results on main and 756e9a3 (6 test bodies × 10 argument sets). The redesign changes no verdict path. It only adds refusals for stream targets and malformed inputs.
  • Compatibility: no change to ledger, trace or registry formats. Two inputs now refuse instead of crashing or silently corrupting: stream trace targets, and per-entry shape errors in a trace. Neither is something tgtrace writes.

Known limits

… writes (#154)

verify.run and pytest_adapter.run duplicated the temp dir, env, runner and
trace load, and refused in different orders and words. Both now go through
pytest_adapter.run_traced / load_trace. A _workdir parameter replaces the
tests' global patch of tempfile.TemporaryDirectory.

- --json refusals print {"refused": true, "error": ...} in both CLIs
- tgtrace flushes and fsyncs before os.replace; a non-regular target
  (/dev/stdout, a FIFO) is written directly
- --trace-out is written atomically (write_json_atomic)
- harness/trace.py treats an unreadable trace as no trace and removes
  stale <out>.tmp.* files before a run
- adapter trace-refusal tests move to test_pytest_adapter.py; the
  dying-write tests use an unserializable payload instead of patching json

Fixes #154
…review)

- F1/F2/F8: one stream rule in both writers (/dev/*, /proc/*, FIFOs are
  written in place) so /dev/stdout works even when redirected to a file;
  the adapter's temp is now <path>.tmp.<pid>, like tgtrace's; a parity
  test holds the two copies in step
- F3: a truncated JUnit report is refused (exit 2), not a ParseError
- F4: a trace object without tgtrace's tests/declared shape is refused;
  harness/trace.py treats it as no trace
- F5: harness/trace.py refuses a non-regular --out instead of deleting it
- F6: the stale-temp sweep takes only .tmp.<digits> of dead pids
- F7: unittest.main() moved to the end of the two test files
- F9: --trace-out is checked before the suite runs; verify refuses an
  unreadable registry instead of raising
- C1: direct test that a missing JUnit report is no verdicts, not a pass
… review round 2)

Writing /dev/stdout in place corrupted it (a second open has its own
offset, so pytest's output overwrote the JSON), and the /dev/ prefix rule
also caught /dev/shm, an ordinary tmpfs. Stream support is dropped:

- one rule, not_a_file_target: abspath, then /dev/ or /proc/ but not
  /dev/shm/, or an existing non-regular file. tgtrace refuses at
  pytest_configure; the adapter refuses --trace-out before the run;
  harness/trace.py imports the adapter's rule. The tgtrace copy is
  parity-tested, including a child whose stdout is a regular file.
- writers keep only temp + fsync + rename
- payload_shape_error checks each entry; declared: null stays accepted
- verify refuses a registry that is not a JSON object
- review test fixtures become mixins, so no inherited test runs twice
…ry (#154 review round 3)

- R3-1: pytest calls unconfigure even after configure raised; tgtrace now
  writes and stops only if tracing started (it had replaced a refused FIFO
  on 3.11 and raised on 3.12+), for both the stream and xdist refusals
- R3-2: the real-pytest tgtrace test skips without pytest (CI matrix)
- R3-3: declared ids need only be hashable; covers(1) is a bad declaration
  for verify to report, not a broken trace
- R3-4: the target rule resolves the directory with realpath, so a
  symlinked parent or //dev is still /dev
- R3-5: registry.load refuses JSON that is not an object, for every caller
- R3-6: x/, x/. and x/.. are refused; writers normalise the path once
- R3-7: a directory --trace-out says so
- R3-8/R3-9: one ROOT in harness/trace.py; tests pick writers by key
@inth3shadows
inth3shadows merged commit fc3d000 into main Nov 15, 2026
6 checks passed
@inth3shadows
inth3shadows deleted the fix/154-trace-followups branch November 16, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Trace handling follow-ups after #108: --json refusal, harness/trace.py, fsync, --trace-out, shared runner

1 participant