feat: invalidate stale downstream outputs after fork changes #33

Activity

1. utsab345

   added

   enhancementNew feature or requestNew feature or request

   testingTests and test coverageTests and test coverage

   on Oct 21, 2026
2. how2how2how2-arch commented on Oct 28, 2026

   

   how2how2how2-arch

   on Oct 28, 2026

   Contributor

   More actions

   Read the code paths this issue names, on main after #37. One correction to the framing first, then an answer to the question you asked.

   The blocker is upstream of replay. ForkEngine.fork() builds a complete ForkInfo, and then it is dropped: fork_trace() keeps only fork_point_step, fork_point_event_id and overrides_applied in ForkResult, and TraceManifest has no field to receive the rest (trace/manifest.py: extra="forbid", schema_version: Literal["0.1"]; storage.build_manifest() fills the manifest from trace fields only). So a saved fork is not merely "unlabelled by replay" — it is indistinguishable from a plain recording. The parent run id, the fork point and the override set are discarded at the moment the bundle is written, and no replay-side change can recover them.

   Two details from that path:

   • overrides_applied is a count, and the classification needs the set. "The output at step 7 was recorded under an upstream state that changed at step 2" requires knowing which steps were overridden; no arithmetic on a count produces that.
   • ForkInfo.parent_trace_path is a filesystem path, which is not portable inside a bundle. What a bundle can carry is a digest of the parent.

   Also worth noting: OverrideRegistry.apply_override() preserves id and step and replaces only output, so an overridden event is identical in identity to the event it replaced. Diffing parent against fork shows the value and nothing about why it differs.

   Proposal: persist at fork time, classify at replay time, as a pure function of the persisted data.

   1. Additive provenance under fork: in the manifest — parent_run_id, parent_trace_sha256, fork_point_step, fork_point_event_id, fork_timestamp, and overrides: [{step, event_id, target, replaced_output_sha256}]. Digest rather than value: the overridden value is already in the event; what the bundle cannot otherwise name is which outputs were replaced, and the digest of the parent's original is what makes the record checkable rather than asserted.
   2. ForkResult keeps overrides_applied: int — nothing that reads it breaks — and gains the tuple.
   3. Replay derives evidence per call from that record alone, with no new I/O:

   executed           the value was produced live (existing action == "executed")
   recorded           substituted, and no override sits at a step < this call's step
   potentially-stale  substituted, and some override sits at a step < this call's step
   unknown            the bundle is a fork but names no override set
   

   fork is None gives recorded, so every bundle written today replays exactly as it does now.

   Two axes, not three states. MatchedCall.action answers how the value was obtained; evidence answers whether the value is evidence for this path. A call can be executed while its inputs were derived from a stale value — those two facts do not fit in one enum, and collapsing them means one of the two can never be reported. Keep action a two-valued Literal and add a defaulted field beside it. (I have been bitten the other way round: one status cell holding two facts makes an unrelated timeout look like a refutation.)

   The error is one-sided, deliberately. With no dataflow, "after the earliest override" over-approximates dependency. Name the field so it says that: a value may depend on a changed upstream value, not was affected by one. The direction is the point — it can over-report stale and never under-report, which is what makes the label shippable before dependency analysis exists rather than a guess that occasionally blesses a stale output.

   The forward-compat question you have to decide, not the PR. extra="forbid" plus Literal["0.1"] means "additive and optional" is still a versioned change: an older StepFork reading a new fork bundle will refuse it instead of ignoring the new key. Either bump to 0.2 with a loader that accepts both, or accept the break explicitly. I would rather the issue state which than have a PR imply one of the two.

   Where to surface it. Silent substitution is survivable on its own. What creates the false impression is the receipt saying substituted and the diff passing — that reads as "the modified path was observed". So evidence belongs on the receipt and diff verdict: if any compared output is potentially-stale, the verdict says so and names the override steps responsible. Refusal does not have to be the default; keep the tool usable and make the claim narrow.

   Tests. I did not run the suite — git transport to github.com is failing from this machine right now, so treat the list below as a proposal, not a green run.

   • One fork that overrides step 1, then replays a substitution at step 0 and another at step 2, asserting recorded and potentially-stale respectively. A fixture with only post-override calls cannot separate those two units, and a guard whose fixtures cannot separate the units it confuses will pass while measuring the wrong scope.
   • Round-trip: fork → save → load → provenance identical; and a bundle with no fork key loads and replays with everything recorded, so backward compatibility is a test rather than a claim.
   • Plant: rewrite the persisted override set to [] in a saved fork and assert the downstream classification flips. That proves the classifier reads the provenance instead of recomputing the answer from the events.
   • The existing changed-args path must still raise ReplayMismatchError; the new axis must not absorb it.

   Not claiming: no re-execution, no dependency-aware invalidation, no "fork detects divergence". docs/fork.md keeps its limitation line as written — the label describes what a value is evidence for, and says nothing about what ran.