Implement RFC-023 substrate-native key-conflict fencing

[aaltshuler]

Summary

• activate RFC-023 in internal schema v6: every graph table has exact non-null physical id as Lance's unenforced PK, and every production strict insert/upsert uses the sealed exact-id filter-bearing adapter
• add the durable omnigraph.insert_absence=v1 certificate and the provenance-proven BranchMerge publisher, with complete-history validation, source/target incarnation fences, bounded transaction chains, and safe ordered-diff fallback
• remove keyed Append as a production graph-write route; add typed KeyConflict, ResourceLimitExceeded, effect-aware retry/recovery behavior, and the corresponding CLI/HTTP/OpenAPI contracts
• add genuine v5↔v6 rebuild/refusal coverage, duplicate-repair guidance, extensive concurrency/recovery/format tests, and the RFC-023 decision benchmark harness

Correctness boundaries

• Mutation/Load remains one keyed transaction per table and refuses inputs above 8,192 rows or 32 MiB before recovery arm
• BranchMerge uses row/byte-bounded chunks, at most 1,024 logical data transactions per table, and a 1,026-version exact-recovery ceiling
• the proven all-new route performs zero target key preflights, target merge joins, ordered diffs, or committed Appends; it still commits exact-id filtered Lance Update transactions with zero transparent retries
• missing, cleaned, unknown, or malformed proof returns to the general ordered diff
• the certificate is intentionally non-cryptographic; raw Lance graph writers remain unsupported, and destructive recovery retains the documented single-writer-process boundary

Acceptance evidence

The predeclared five-pair production series passed both fixed gates:

Size	Production/comparator medians	Ratio	Max paired RSS overhead
10K × 256	31 / 8 ms	3.875×	24,297,472 bytes
100K × 256	136 / 35 ms	3.886×	32,604,160 bytes

Both are below the 5× latency and 64 MiB RSS-overhead gates. All 20 aggregate records, 60 setup/operation/verification child phases, exact-content checks, and route assertions passed. The exact seeds, order, tree, binary digest, raw arrays, and preserved negative diagnostics are recorded in RFC-023 §11.4.

Validation

• cargo test --workspace --locked
• cargo fmt --all -- --check
• scripts/check-agents-md.sh
• full merge_fast_forward and forbidden_apis suites
• focused source/target incarnation ABA, first-touch, recovery-direction, and keyed-conflict failpoint coverage
• independent final certificate/publisher safety review: no actionable correctness findings

Greptile Summary

This PR implements substrate-native key-conflict fencing for graph writes. The main changes are:

• Internal schema v6 with exact non-null physical id keys.
• Strict insert and upsert adapters with bounded write plans.
• Certificate-backed branch merges with ordered-diff fallback.
• Recovery handling for conflicts and multi-transaction merges.
• Structured HTTP errors for key conflicts and resource limits.
• Rebuild, concurrency, recovery, and performance coverage.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.
• Keyed writes validate physical identity before committing.
• Partial and ambiguous effects remain covered by durable recovery.
• Branch merges recheck source and target authority before publication.

Important Files Changed

Filename	Overview
crates/omnigraph/src/exec/staging.rs	Adds bounded strict-insert and upsert staging with effect-aware conflict handling.
crates/omnigraph/src/table_store.rs	Adds exact-id keyed-write adapters, validation, and insert-absence certificates.
crates/omnigraph/src/exec/merge.rs	Adds certificate-backed merge publication, incarnation checks, fallback logic, and bounded transaction planning.
crates/omnigraph/src/db/manifest/recovery.rs	Extends recovery for keyed writes and multi-transaction branch merges.
crates/omnigraph-compiler/src/catalog/mod.rs	Orders composite key members by stable property identity.
crates/omnigraph-server/src/lib.rs	Maps key conflicts and resource-limit failures to structured HTTP responses.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant C as Writer
    participant S as Staging
    participant R as Recovery intent
    participant L as Lance table
    participant M as Manifest

    C->>S: Prepare bounded keyed write
    S->>L: Validate exact id key
    S->>R: Arm recovery intent
    S->>L: Commit filtered transaction(s)
    alt Effect-free conflict
        S->>R: Retire intent
        S-->>C: KeyConflict
    else Ambiguous or partial effect
        S-->>C: RecoveryRequired
    else Effects confirmed
        S->>M: Publish graph state
        S->>R: Remove intent
        S-->>C: Success
    end

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant C as Writer
    participant S as Staging
    participant R as Recovery intent
    participant L as Lance table
    participant M as Manifest

    C->>S: Prepare bounded keyed write
    S->>L: Validate exact id key
    S->>R: Arm recovery intent
    S->>L: Commit filtered transaction(s)
    alt Effect-free conflict
        S->>R: Retire intent
        S-->>C: KeyConflict
    else Ambiguous or partial effect
        S-->>C: RecoveryRequired
    else Effects confirmed
        S->>M: Publish graph state
        S->>R: Remove intent
        S-->>C: Success
    end

Loading

_{Reviews (2): Last reviewed commit: "chore: remove unrelated rustfmt churn" | Re-trigger Greptile}

Context used:

• Context used - AGENTS.md (source)
• Context used - CLAUDE.md (source)