test: build a differential bus state-machine oracle

[wallstop]

Part of #497. Uses the evidence format from #508. Blocks runtime candidates in #501 through #505.

Question

Can a candidate bus implementation be shown trace-equivalent to the control over generated sequences of dispatch, mutation, reentrancy, reset, failure, and lifecycle operations?

Mechanism

Example tests cover known cases but miss interactions and operation ordering. A differential state-machine runner can execute one deterministic sequence against control and candidate implementations, compare externally observable state, and shrink a failure without copying production routing logic.

Contract

Observe production implementations; do not reimplement their algorithms. Compare callback order, message/context values, found result, diagnostics, exception outcome, registration counters, occupied slots, trim result, retained references, and subsequent usability.

Generate register, duplicate register, enable, disable, deregister, emit, nested emit, self-mutation, reset, trim, scene transition, host destruction, and throw operations. Preserve live activity, emission-start snapshots, deferred reset, exception cleanup, stale-handle safety, multiple-bus isolation, and all message kinds.

Factors and workloads

• MessageScenarios.AllKinds through shared parameterization.
• Sequence length, seed, type count, fan-out, priorities, active density, and route cardinality.
• Same-kind and all six cross-kind reentrancy combinations through depth 10.
• Copied, duplicate, foreign-token, out-of-order, reused, stale, and generation-wrap handles.
• Scene unload, DontDestroyOnLoad, prefab churn, disable/re-enable, quit, and host destruction.
• Empty, boundary, invalid, unexpected, and throwing operations.
• Stable, same-type churn, and unrelated-type churn.

Primary response

The number of observable trace mismatches between control and candidate.

Independent unit

One deterministic seed and its complete operation sequence. Operations inside the sequence are correlated.

Effect threshold

Behavioral acceptance is exact: zero mismatches across all preregistered seeds and canonical fixtures. One reproducible mismatch rejects a candidate regardless of throughput.

RED proof

Supply test-only faulty adapters that independently drop a callback, reorder equal-priority callbacks, continue after reset, reuse a stale handle, skip exception cleanup, misreport diagnostics, retain a dead reference, and leak a registration. Detect and shrink each mutant.

GREEN suites

Pass focused oracle tests, parameterized runtime tests, LifecycleEdgeCasesTests, ReentrantEmissionExtendedTests, allocation-matrix rows for candidate paths, LeakWatcher over whole scenarios, reclamation/trim tests, public-surface checks, and applicable EditMode, PlayMode, and Standalone legs.

Stop rule

Do not promote a candidate if a mismatch cannot be replayed and explained. Reject an oracle design that duplicates production logic, depends on sleeps or retries, swallows exceptions, or cannot shrink while preserving a valid sequence.

Immutable evidence

Retain generator version, seed manifest, original/minimized sequences, both traces, environment/profile/candidate hashes, classification, coverage, and replay command.

Dependencies

Implementation can start independently. Durable failures use #508. This task becomes a prerequisite for every runtime implementation PR in the campaign.

Completion checklist

• Define operation and observable-trace schemas.
• Add isolated control/candidate replay.
• Add deterministic generation and shrinking.
• Add every message-kind, lifecycle, and reentrancy factor.
• Prove every faulty adapter is detected.
• Integrate leak, slot, trim, and retained-reference assertions.
• Map allocation factors through the existing allocation suites without timing contamination.
• Retain seeds and minimized failures.
• Link the oracle gate from chore: correctness + perf sweep/audit #497 and perf: screen the simple untargeted dispatch hot path #501 through perf: prototype frozen, batched, queued, and Burst-compatible messaging tiers #505.

Verified foundation status (2026-09-07)

The checked items identify delivered mechanisms; they do not close the remaining factor, observation, or durable-evidence work. At source a75f054f493789707d451f523f82df2227b7dea4, the Unity 2021.3 Standalone run passed all 196 pure differential cases and all 24 native host cases. This includes detection and shrinking for the eight required fault classes. Artifact 10002799467 has archive SHA-256 8e190ebc29327ef4c7540f3ded3cb89388883edf1dffd3bfa280e5325480105d; its XML was inspected directly. This ordinary CI artifact is not the immutable campaign bundle required below.

PR #551 at commit 0a7b798ff44ae009f4235004cfe34905132e7094 adds generator version 8 independent registration handles, duplicate registrations, copied/stale aliases, reused slots, and dependency-preserving shrinking. Observation schema 2 records final typed payload/context values and actual unmatched diagnostics, including nested-call attribution and scoped logger restoration. The focused native replay/lifecycle selection passes 377 cases with zero failures, skips, or inconclusives; the ordinary runtime selection passes 1,412 cases with one expected interactive-editor optimization skip. Both runs restore the original clean scene. These local results use Unity 6000.4.6f1. The same source also passes the Unity 2021.3 Standalone job: all 286 differential, 24 native-host, 37 lifecycle, and 30 reentrant cases passed. The complete player result contains 1,406 passes, zero failures/inconclusives, and three undefined optional-integration skips. Artifact 10003892476 has archive SHA-256 d51f68c2b2807c9924c692abf1e4a2b8e5cf735651abb1537b6c0b4926293bf2; the archive digest and actual XML cases were verified. This ordinary CI artifact is not immutable campaign evidence.

Remaining work includes broader workload factors, generation wrap, generated and shrinkable native lifecycle transitions, untyped internal final values, arbitrary retained-reference coverage, allocation-factor mapping, and sealed replay bundles. The public emission API returns void: unmatched-handler diagnostics are observable, but their absence must not be mislabeled as a found=true result. PR #551 remains an increment toward this issue's full acceptance.

Explicit-handle callback replay in PR #552

PR #552, source 3e67087da13cf3d191a739d4342d204141de24c5, adds generator version 9 while retaining versions 1 through 8. Reset, throwing-callback cleanup, token disable, handler activity, and nested emission can now select explicit registration handles. Replay retains callback identity through copies, duplicates, and reused slots; stale aliases cannot arm later callbacks. Nested replay covers all message kinds and depth 10.

Dependency validation and shrinking cover these operations. Faulty adapters demonstrate skipped-action and wrong-handle cleanup detection, including a removed alias whose duplicate callback remains live. Independent review reports no remaining findings.

Local Unity 6000.4.6f1 passes 422 focused oracle, native-host, lifecycle, reentrant, and contract cases with zero failures, skips, or inconclusives. Exact CI-filter runtime runs pass 1,445 cases plus the same expected editor-only skip; the control passed 1,412. Both repeated runs restore the original clean scene and finish framework cleanup. Control/treatment means are 24.65/23.95 seconds locally. These repeated measurements use the ordinary CI filter and preserve all existing cases.

All PR checks pass, including all four Unity versions and all three modes. The downloaded native artifacts match their published SHA-256 digests. Every existing case and outcome is preserved, with the unsupported-version case moving from 9 to 10 now that 9 is supported. Each runtime mode passes all 319 differential cases, up from 286, and retains all 12 contract cases. Each version reports 941 EditMode, 1,443 PlayMode, and 1,439 Standalone passes with the same 11/3/3 expected skips and zero failures or inconclusives. The tested merge tree matches the PR source tree. For the Unity 2021.3 Standalone artifact 10006012973, archive SHA-256 is 1b219ede0f07c22f63ea513de960236e09cb8551c25187d503fa022fe4ac499c.

Observed Unity test-step time across the twelve scopes is 787 seconds before and 772 seconds after. The 2022 comparison uses the preceding PR551 run on the same runner; its complete source tree equals the main baseline. Runner assignments vary for other versions, so the aggregate reduction is not a causal speedup claim. The same-runner 6000.5 test suites are 2.66 seconds faster, although their surrounding build/startup steps cost more in this run. The ordinary CI workload retains its filters and gains coverage. Generated native lifecycle transitions, generation wrap, broader workload factors, untyped internal final values, arbitrary retained references, allocation mapping, and sealed cross-backend evidence remain open.

Reset-generation boundaries in PR #554

PR #554, source 5afd1aabc2470d8bcf721f66a9a9f28a10d2e818, adds 12 parameterized cases across all message kinds. They seed isolated real buses at signed-wrap and zero-wrap boundaries, then exercise reset during dispatch, copied and stale handles, replacement registrations, disable/re-enable recovery, a second reset, and forced trim. A real adapter fault that refuses the boundary reset is detected and reduced to a valid deletion-minimal sequence. A checked-arithmetic mutation of the production increment fails the leak guard.

Both final native Unity MCP runs pass 1,457 ordinary runtime cases with one unchanged expected skip, zero failures/inconclusives, matching run/cleanup identity, and clean restored scenes. All 1,446 previous leaf outcomes are retained as a multiset, with 12 additional passes. Independent adversarial review is clear.

The same PR keeps every comparison, callback, registration, and emission in 24 priority/interceptor assertion loops, but constructs NUnit diagnostics only when integer values differ. Baseline durations are 24.3663277 and 23.8647054 seconds; final durations are 6.469511 and 5.3421594 seconds. Means are 24.11551655 and 5.9058352 seconds. NUnit assertion calls decrease; behavioral comparisons and coverage remain intact. These are local Mono observations. The Windows CI evidence below measures the final source separately.

Generated native scene/host lifecycle factors, generation-bearing override leases, untyped final internal values, and durable cross-backend campaign evidence remain open. This does not close the full oracle issue.

Checklist scope audit

The checked schema mechanism refers to generator version 9 and observation schema 2. It does not mean every required observation or factor exists. PR #554 passes all four editors in EditMode, PlayMode, and Standalone. Independent raw-artifact review verifies every prior fullname/result/label outcome and multiplicity. Each of the eight runtime legs adds exactly the same 12 expected reset-boundary passes; no case is removed or changes status. All mode and cleanup steps pass. Historical PR #551/#552 statements that generation wrap remained open describe their state at delivery. PR #554 adds reset-generation boundaries; generation-bearing override leases and the other remaining observations/lifecycle factors remain open. No full oracle or durable cross-backend acceptance is claimed.

Final PR #554 native acceptance

The four-version correctness run passes at source 5afd1aabc2470d8bcf721f66a9a9f28a10d2e818. Actual checkout logs and Git trees bind the tested merge to that source. All 24 baseline/candidate correctness ZIP digests and complete outcome multisets were independently verified. The 24 guarded integer-comparison sites and eight containing tests remain intact.

Unity	Baseline Unity execution seconds	Candidate seconds	Runner comparison
2021.3	205	150	DAD-MACHINE on both
2022.3	202	154	DAD-MACHINE on both
6000.3	224	158	DAD-MACHINE on both
6000.5	247	137	DAD-MACHINE to ELI-MACHINE

The matched-runner correctness subtotal is 631 to 462 seconds. The full observed correctness subtotal is 878 to 599 seconds. These are sequential elapsed-work observations; the hardware change in the final pair prevents attributing its whole difference to this change. Queue time is excluded, and nested XML suite durations are not added to Unity-step time. Shipping was skipped in all these ordinary correctness jobs. Full campaign lifecycle coverage and durable cross-backend evidence remain open.

Global override lease replay in PR #555

PR #555, source fc3927cf63f7061b01b8808cf3aa2a6b1fdcc930, adds generator version 10 while preserving versions 1 through 9. Generated actions acquire, copy, dispose, and replace real global bus leases. Replay observes actual bus identity and preserves primary emitter fault injection. Validation and shrinking preserve lease dependencies, stale copies, slot reuse, and out-of-order cleanup.

Parameterized native tests cover generation zero, signed wrap, and final issuance before exhaustion. Nine emitter mutation regressions fail before the adapter correction and pass afterward. All 500 focused native cases pass, including memory reclamation and cleanup-on-failure coverage. Two ordinary Runtime runs each pass 1,495 cases with the same expected skip, zero failures/inconclusives, matching cleanup identities, and clean restored scenes.

The exact prior outcome multiset is preserved after two reviewed replacements: unsupported generator 10 becomes 11, and the original exhausted-generation case becomes the equivalent -2L row. The suite gains 38 passing cases. Disposable scopes preserve cleanup order, exceptions, inputs, and assertions. Independent review is clear.

The source-scan optimization preserves all 38 matches across 229 files. Local timings vary; the final four-version native matrix and complete CI timing acceptance are pending.

This completes the bounded generation-bearing override lease factor. Generated native scene/host transitions, broader workload factors, untyped internal final values, retained-reference and allocation mapping, and sealed cross-backend campaign evidence remain open. Observation schema 2 remains current; the full oracle issue stays open.

[wallstop]

PR #524 adds the first deterministic trace foundation in fdfe4cd. It generates, validates, replays, compares, and deletion-shrinks Register/Remove/Enable/Disable/Emit sequences. Each replay uses isolated production MessageBus/token APIs. Reports include the seed, generator version, complete operation sequence, first mismatch, callbacks, state, and exception observations.

Forty cases compile against actual Unity references; 28 pure cases pass independently. The 12 real-bus cases include equivalence, isolation, and a genuinely faulty emission adapter whose missing callbacks must be detected and shrunk. Those native cases have not run locally and now await exact-head CI.

This is not the complete oracle gate. Reset/trim, lifecycle, reentrancy, cross-kind depth, retained-reference observations, all eight faulty adapters, and durable campaign evidence remain open.

[wallstop]

PR #524 adds the callback-time reset increment in 1be8edc.

EmitWithReset arms one selected token's actual callback to reset its isolated production bus. A test-only faulty emitter postpones that real reset until emission returns. Control invokes only the first callback; the faulty emitter invokes both. Complete final state and exception traces match, isolating the callback-timing defect. Each message kind shrinks to exactly two registrations plus one reset emission and replays the same mismatch; removing the faulty adapter restores equality.

Coverage also checks post-reset silence, new registration, stale cleanup, staged disable/re-enable, another bus remaining usable, disabled/unmatched triggers, and a throwing reset action. Trigger state clears before reset and in finally. Generator version 2 is explicit; version-1 generation and minimized-trace provenance remain supported.

The expanded trace fixture has 56 cases. Independent managed verification passes 1,800 parameterized executions plus 20 mechanics cases. Fresh Unity 6000.4.6f1 passes the trace fixture within a 323-case runtime/lifecycle/reclamation/public-surface batch, twice, with zero skips. Editor contracts and the explicit allocation matrix pass 139/139. Independent review reports zero findings; current and Unity-2021-compatible C# builds pass.

This proves two faulty adapters, not all eight. The remaining six, broader lifecycle/reentrancy factors and observables, and immutable evidence integration remain open. New-head CI is pending. Full/minimized traces are retained locally in .artifacts/session262-reset-oracle-probe/repeat.log; this is not an immutable remote evidence claim.

[wallstop]

PR #528 extends the oracle in 0b50c38. Version-three inputs add stale-handle removal, throwing emission and diagnostics changes while preserving version-one/two values and generation. The shared observer now captures real token metadata, call counts, history and retained boxed-message references.

Six independent operation adapters reorder callbacks, reuse a stale handle, skip callback-owned exception cleanup, misreport configured diagnostics, retain an old diagnostic message, or leak a registration. These mutate actual production operations; they do not replace the observed trace. Every mutant is detected and shrunk to a replayable dependency-minimal sequence for all three message kinds. Combined with the prior dropped-callback/reset mutants, all eight named defect classes have bounded demonstrations.

The 85 oracle cases pass against actual current Runtime in a separate managed harness. Test-only source copies remove outer Unity preprocessor guards; no routing implementation or Unity stubs are supplied. The 12 new boxed-payload regressions bring that proof to 97/97 in 99 ms. Unity-symbol compilation also passes; native execution remains pending.

This does not close the full issue. Exception coverage is callback-owned cleanup, and liveness covers diagnostic-history references. Complete scene/prefab/quit/reentrancy/depth/type/handle factors, arbitrary retained-reference probes, allocation integration and durable failure bundles still need work.

[wallstop]

PR #534 adds version-five differential replay for same-kind and all six cross-kind nested dispatch pairs at depths 1 and 10, plus callback-time disable and re-enable usability. Versions 1 through 4 retain their recorded generation. The root cause of this coverage gap was that the synchronous operation schema could express a throwing or resetting emit, but not nested routing or callback-time disable; separate lifecycle fixtures did not exercise adapter replay state.

The new adapters perform real bus operations. Independent omitted-nested/omitted-disable faults are detected and shrunk to valid sequences. Native RED was 134 passed with exactly six expected mutant failures; fresh GREEN was 147 passed. Review then required nested exception cleanup coverage. Eighteen additional cases throw from a real nested callback and prove emission-scope restoration, no live dispatch, no rearmed trigger, and subsequent plain/nested usability. Independent copies missing either trigger clearing or depth unwinding fail all 18 cases.

The final native batch includes the oracle, LifecycleEdgeCasesTests, and ReentrantEmissionExtendedTests: 232 passed, zero failures/skips, then 232 passed again without another refresh (0.652/0.390 seconds, macOS Unity 6000.4.6f1). This is local Editor/Mono evidence. Standalone and the four-version CI matrix are pending on #534.

Keep this umbrella open: coroutine scene/host lifecycle replay, additional handle and workload factors, broader observables, and durable campaign evidence remain incomplete. Canonical lifecycle tests are regression coverage, not a substitute for those oracle operations.

[wallstop]

PR #539 advances foreign-token ownership coverage with generator version 6. Versions 1-5 keep their pinned replay prefixes. RemoveForeign carries an explicit handle owner and calls the real token API with a copied handle. Canonical cases cover live, disabled/re-enabled, stale, and reused registrations across all message kinds. An independent faulty adapter removes the destination's real registration, and the oracle detects the resulting state mismatch.

Native RED rejected missing replay support in all three kinds. Native testing also caught a new test assumption: a reused Register/Remove/Register dependency cycle has five operations under the shrinker's documented one-deletion-minimal rule, although paired deletions can reach a global minimum of three. The test now verifies each possible single deletion; a separate simple seed still requires exactly Register/Register/RemoveForeign. No shrinker or production behavior was weakened.

Final full runtime Unity runs pass 1,432 tests twice, in 25.66 and 21.02 seconds, with one existing skip. No new frame waits, assemblies, benchmark windows, timing exemptions or workflow changes were added. Static generation checks validate 1,000 v6 seeds; independent review has no remaining findings. Current-head CI is pending.

This does not close #509. Scene/host lifecycle factors, copied/out-of-order/generation-wrap handles, wider workloads and durable campaign evidence remain open.

[wallstop]

The foreign-handle factor is accepted in PR #539: generator v6 records handle ownership and RemoveForeign, while versions 1-5 retain their earlier prefixes. All four Unity versions pass 176 differential cases in both PlayMode and standalone. Faulty-adapter detection and valid single-deletion-minimal shrinking pass. This advances the workstream; remaining scene/host lifecycle factors and durable campaign requirements stay open.

Final acceptance and timing evidence verifies all four Unity versions, all 12 correctness archives, all performance archives, exact baseline skips, and green PR checks at 9f656f277e9082d5215edfc92dca1b53eebe2ff0. The PR is ready and remains unmerged.

[wallstop]

#547 advances the live-handler lifecycle part of this workstream. Generator version seven adds explicit handler activity and callback-time activity changes, retaining real MessageHandler objects independently of token.Enabled. The model and adapters agree on later-handler visibility within the same emit. Ignored-toggle mutants, skipped-trigger cleanup, and shrinking validity are covered; versions one through six retain their seeded streams.

The combined focused lifecycle/oracle batch passed 242 cases. Two full native runtime runs passed 1,485 cases. Broader scene/workload factors and campaign acceptance remain open.

[wallstop]

PR #551 adds a native scene-unload protocol at commit 7e6dc3e5. The same operation sequence runs against isolated production adapters for all three message kinds. A callback requests real asynchronous scene unload; the protocol then verifies persistent-owner delivery, ordinary-owner destruction, registration diagnostics cleanup, stale cleanup, disable/re-enable, trim, and later usability. A test-only adapter that omits DontDestroyOnLoad must produce a callback mismatch on the first post-unload emit.

Local Unity 6000.4.6f1 validation through Pipeline passed all 24 differential host cases plus 37 lifecycle cases. The ordinary PlayMode assembly passed 1,322 cases, zero failures/inconclusive, with one expected interactive-editor optimization skip. A subsequent persistent-domain focused run passed the same 61 cases. Every run reached matching GUID/path cleanup records and restored the original saved scene.

This is an explicit native scene-unload-v1 boundary over version-seven operations. It does not claim generated or shrinkable scene transitions, broader workload/handle-factor completion, or durable campaign acceptance. Those requirements remain open. PR CI is pending.