Unified repeatability (Rerun / Recall / Replay) across Governance, Playbook and consuming apps

Summary

Deliver one coherent repeatability capability across three deliverables with strict boundaries:

1. Goodtocode.Agents.Governance (policy): what repeatability means and how it is recorded.
2. Goodtocode.Agents.Playbook (mechanism): how a Collect / Evaluate / Record (CER) playbook run can be resumed from prior state.
3. Consuming apps (execution): agent-framework-quick-start (simple) and Censic Crucible (complex, pipeline level) compose the two.

Hard constraint: Governance and Playbook never reference each other, directly or through a shared package. They converge only in the consuming app through plain values (bools, strings, Guids, and the consumer's own types). No NuGet reference hell, no duplicated rules.

Background and verified current state

• Playbook owns PlaybookReplayMode (Rerun, Recall, Replay), PlaybookReplayContext of TEvidence and TFinding (mode, sourceExecutionId, priorEvidence, priorFinding, with Validate()), and a PlaybookExecutor.ExecuteAsync overload that accepts the replay context. Verified by reflection: Replay needs only prior evidence; Recall needs prior evidence and prior finding.
• Governance v2.0.5 has no ReplayMode or SourceExecutionRef. Only the unreleased main branch has them.
• Crucible works around this with a stopgap: PolicyProfileVersion suffixed with the mode and source execution id, and a run reference added to EvidenceRefs.
• Quick-start has a bug: PlaybookWorkflowGraphExecutor does not default metadata to Rerun, and its graph path cannot skip stages. Its Recall re-runs Record; Crucible requires Recall to be a pure read.
• Crucible implemented pipeline-level repeatability with normalized persistence (PlaybookRunEntity, PlaybookEvidenceItemEntity, PlaybookFindingItemEntity, ExecutionEntity.ReplayMode and SourceExecutionId), per-kit evidence codecs, and an application service that calls the Playbook executor directly.
• Both consumers hand-roll the mode-to-stage logic. Nothing guarantees they agree.

Goals

• One vocabulary for the three modes with one definition of what each does, owned by exactly one package.
• Libraries stay independent: no package, project or type reference between Governance and Playbook.
• Simple consumers adopt it in a few lines. Complex consumers (multi-playbook pipelines, multi-tenant, normalized persistence) can extend every seam.
• SOLID, .NET centric, interface first, DI friendly, async end to end, nullable enabled, no reflection or JSON bags in contracts.
• Learn from LangGraph and LangChain, and improve on them (see Design principles).

Non-goals

• No persistence implementation in either library (ports only).
• No pipeline orchestration in Playbook or Governance (stays in the app).
• No UI in the libraries.
• No before/after comparison view (separate future work).

Design principles (what we borrow, what we improve)

Borrowed from LangGraph and LangChain:

• Checkpointing as a port: state is saved at stage boundaries behind a store interface, keyed by a thread/run identity (LangGraph checkpointer).
• Resume from a checkpoint rather than special-case replay code (LangGraph time travel).
• Config passed through the call, not global state (LangChain RunnableConfig).
• Small composable abstractions and middleware-style extensibility.

Improvements for .NET:

• Strongly typed generics instead of dictionary state: evidence and finding are the consumer's own types.
• Policy is separated from mechanism. LangGraph mixes replay semantics and execution; here Governance decides, Playbook executes, the app wires.
• Governance is a first-class concern: every repeat is recorded with mode and lineage for audit and defensibility, which neither Python library provides.
• Conformance tests are shipped so consumers cannot drift.
• DI extension methods and options validation (ValidateOnStart) instead of ambient config.

Architecture: the seam is plain data

Governance (policy) emits a stage plan. Playbook (mechanism) consumes a stage request. The app maps one to the other. The only things crossing the seam are bools, strings, Guids and consumer types.

Flow in the consuming app:

1. Ask the Governance policy for a ReplayStagePlan for the requested mode (and validate the source).
2. Load seeded evidence/finding through the app's store (port defined by Playbook).
3. Map the plan to a Playbook ExecutionRequest (skip flags plus seeds).
4. Run the playbook executor.
5. Read the Playbook result (which stages ran, lineage) and pass mode, source reference and stages-ran to the Governance recorder.
6. Persist new state through the app's store.

The mapper is roughly 15 lines. It is documented and covered by the conformance kit. We deliberately do not ship a third bridge package.

Part 1: Goodtocode.Agents.Governance (policy only)

Must not mention playbooks, evidence types, CER, or Playbook package types.

Contracts:

• RepeatMode enum: Rerun = 0 (default), Recall = 1, Replay = 2. Serialization-stable values.
• RepeatRequest (immutable value object): Mode, SourceExecutionId (string or Guid). Validation: Rerun forbids a source; Recall and Replay require one. Typed exceptions, no magic strings.
• ReplayStagePlan (immutable): Collect, Evaluate, Record booleans plus the mode. Default table:
  • Rerun: collect, evaluate, record.
  • Replay: skip collect, evaluate, record.
  • Recall: skip collect, skip evaluate, skip record (pure read). Re-recording on Recall is an explicit opt-in policy option (RecallRecords = false by default) so the quick-start behavior stays expressible.
• IRepeatPolicy: Plan(RepeatRequest) returns ReplayStagePlan. Default implementation is table driven and replaceable (open/closed).
• IRepeatSourceResolver: ResolveAsync(request, scope) returns a validated source reference. Rules a consumer implements: tenant scoped, source must be completed successfully, source must be replay capable. Governance defines the contract and standard failure categories (not found, conflict, forbidden).
• IRepeatRecorder: records mode, source reference, and stages-run on the governance record/event. Replaces the PolicyProfileVersion suffix and EvidenceRefs stopgap with first-class ReplayMode and SourceExecutionRef fields. Defaults (Rerun, null source) keep existing serialized records valid.
• RepeatGuard: named guard methods that throw typed exceptions the app can map to 400, 404, 409.
• AddGovernanceRepeatability() DI extension with options (RepeatabilityOptions: RecallRecords, validated on start).
• Conformance kit: abstract test classes (for example RepeatPolicyContractTests, RepeatRecorderContractTests) that any implementation or app mapper runs to prove it honors the stage table.
• Docs: modes, stage table, recording rules, migration off the stopgap.

Governance pillar mapping: repeatability becomes enforceable and auditable (what mode, from which source, which stages actually ran), reinforcing defensibility and auditability.

Part 2: Goodtocode.Agents.Playbook (CER mechanism only)

Must not mention governance, policy, or modes. Owns how a CER run starts from prior state.

Contracts:

• PlaybookExecutionRequest of TEvidence and TFinding (input): SkipCollect, SkipEvaluate, SkipRecord booleans, optional SeedEvidence, optional SeedFinding. Validation lives here as mechanical invariants (for example SkipCollect requires SeedEvidence; SkipEvaluate requires SeedFinding). These are structural rules about CER, not repeat policy.
• PlaybookExecutionResult additions: StagesExecuted (collect, evaluate, record flags), plus the produced Evidence and Finding.
• IPlaybookExecutor: one overload taking the request. The existing replay overload and PlaybookReplayContext are marked obsolete (one release), implemented as thin adapters over the new request, and PlaybookReplayMode is deprecated.
• Checkpoint port (LangGraph checkpointer analogue): IPlaybookCheckpointStore of TEvidence and TFinding with load/save by run identity and stage. Playbook defines the port only. Typed evidence and finding, no JSON in the contract. Consumers implement it (normalized rows, EF, blob, memory).
• In-memory reference store for tests and quick-start samples.
• Graph and non-graph executors share the same request semantics, so a stage skipped in one path is skipped in the other (fixes the quick-start graph limitation at the mechanism level).
• Docs: CER stage model, seeding and skipping, store port, obsolete migration.

Part 3: consuming apps (execution)

agent-framework-quick-start (simple consumer)

• Reference both packages. Implement the small mapper and an in-memory or EF store.
• Fix PlaybookWorkflowGraphExecutor to default to Rerun and to honor the request semantics.
• Ensure every runner generates an ExecutionId and persists it.
• Run the Governance conformance kit against the mapper.
• UI: Rerun, Recall, Replay controls on the playbook page with short guidance text and an explicit label showing the mode and source on the result.
• Recall semantics: either keep re-record (RecallRecords = true) and document, or adopt the pure read default.

Censic Crucible (complex consumer)

Already implemented at pipeline level; this issue converges it onto the libraries:

• Replace ExecutionReplayMode with the Governance RepeatMode at the application boundary (Domain stays free of package types; map at Application).
• Replace PlaybookRepeatabilityService mode switches with: IRepeatPolicy plan, IRepeatSourceResolver implementation (tenant filter, instance canonical key, latest succeeded), mapper to PlaybookExecutionRequest.
• Implement IPlaybookCheckpointStore through the existing normalized entities and per-kit evidence codecs (PlaybookRunEntity, PlaybookEvidenceItemEntity, PlaybookFindingItemEntity). No JSON bags.
• Remove the PolicyProfileVersion suffix and EvidenceRefs workaround; record through IRepeatRecorder.
• Pipeline-level orchestration stays here: source resolution across instances, Recall short circuit returning the source execution, Replay creating a new linked execution, Chronicle and Timeline rebuild, non-replayable playbook guard (conflict on Replay, Recall still works).
• Convert ExecutionEntity.MarkReplayOf InvalidOperationException into the conflict exception category.
• Hydrate RunParameters on Recall (currently empty).
• Run the Governance conformance kit and the existing Reqnroll and KING tests.

Ownership matrix

Phasing and sequencing

Governance and Playbook are independent and can ship in parallel. Neither blocks the other.

1. Governance: contracts, default policy, recorder, guard, DI, conformance kit, docs. Minor version bump.
2. Playbook: request/result/checkpoint port, obsolete adapters, in-memory store, docs. Minor version bump.
3. Quick-start adopts both (simple reference implementation).
4. Crucible adopts both, removes the stopgap and the local mode enum.
5. Deprecations removed in the next major release of Playbook.

Compatibility

• All additions are additive. Defaults are Rerun and no source.
• Old PlaybookReplayMode and replay overload keep working for one release via adapters.
• Existing serialized governance records remain valid.

Acceptance criteria

Governance

• RepeatMode, RepeatRequest, ReplayStagePlan, IRepeatPolicy, IRepeatSourceResolver, IRepeatRecorder, RepeatGuard shipped.
• First-class ReplayMode and SourceExecutionRef on records/events, defaults Rerun and null.
• Recall is a pure read by default; RecallRecords opt-in documented and tested.
• AddGovernanceRepeatability() with validated options.
• Conformance kit published and tested.
• Zero references to Playbook types.

Playbook

• PlaybookExecutionRequest with skip flags and seeds, structural validation tests.
• Result reports stages executed.
• Checkpoint store port and in-memory implementation.
• Graph and non-graph paths honor identical skip semantics (tested).
• Old replay API adapted and marked obsolete.
• Zero references to Governance types.

Apps

• Quick-start: mapper, graph executor fix, ExecutionId on every run, UI controls, conformance kit passing.
• Crucible: stopgap removed, store implemented on normalized entities, source resolver implemented, mapper covered by the conformance kit, MarkReplayOf exception fixed, RunParameters hydrated on Recall.
• Crucible Reqnroll scenarios and the KING end-to-end test still pass.

Cross-cutting

• Dependency check in CI proves neither package references the other.
• Docs updated in all three repos: modes, stage table, seam diagram, migration notes.
• Release notes for both packages.

Risks

• Mapper drift between apps: mitigated by the conformance kit and documented reference mapper.
• A new CER stage added to Playbook requires app mappers to update: mitigated by the stage-plan contract tests and a result flag that fails loudly on unmapped stages.
• Recall semantics divergence (re-record vs pure read): resolved by an explicit option with a documented default.

Open questions

• Should Replay compare the regenerated finding with the source finding and record a match or mismatch flag in governance?
• Is SourceExecutionRef a Guid, string, or Curi in the Governance contract? (Recommend string or Guid to stay agnostic; Crucible wraps as Curi.)
• Should we add a Fork mode later (LangGraph time travel with modified state) as a fourth RepeatMode, or keep it out of scope?
• Should the checkpoint port support partial stage checkpoints (after Collect, after Evaluate) from day one? (Recommend yes: one save per stage boundary.)