feat(observation): add v3 relation-scoped index evidence

[seonghobae]

Stacked on #45 (codex/pr6-v3-representation). This remains the Draft Source Observation successor. Ordinary-forward commits are adopted; exact-head execution evidence never transfers across head movement.

Current authority — 2026-09-13

• Base: feat(observation): add domain-separated v3 successor schema representation #45 exact 6b2a8f555725dc79f60432afbc492d6005290a4a.
• Current source/docs head: 89f53ff08c9a1510b04b0207c2f6b10c1af0ed6e.
• Protected/default ConceptWeave main@f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425 remains the protected coordinate at this snapshot.
• Lifecycle: OPEN / Draft / mechanically mergeable. Mergeability is not acceptance evidence.

Retained production repairs

The branch retains the ordinary-forward Source Observation repairs for exclusion-operator provenance, PERIOD referenced-key completeness, temporal backing-index ordered/static shape, explicit unusable lifecycle rejection, lifecycle-complete PK/UNIQUE supporting-index evidence, and source-authoritative column collation. Generic standalone index lifecycle remains optional; index facts never invent PK/UNIQUE or conperiod truth.

Review 5187787381 additionally found retained temporal/index fixtures that failed before reaching their claimed assertions after the stricter index-admission contract landed. The fixture-only repair series through d704bf9254d0f4c98401052ee275f0984039d62a restores exact key semantics/lifecycle evidence without weakening production rules.

Column collation P1 — source repaired, acceptance superseded by new RED

Review 5187855669 on predecessor exact fcb75659c4c7ffc046c4c7187789ef3b5d53715d found that ColumnObservationV3 did not preserve pg_attribute.attcollation and therefore could not govern PostgreSQL 18 FK collation consistency. Source-level RED 8006b24fd4409bc092d80640084a64892190ea4a preceded production repair.

The ordinary-forward repair remains present:

• 3e495eac9e345db40e3009fe7579e6422a46447a adds the isolated ColumnCollationObservation family;
• 664894780f2643c00b86165b0b75757ee1d607db repairs borrowed determinism access before integration;
• 3d8a7fb7c8f00ebbc7f6994f2b83274a6f3d3bc0 integrates the public aggregate constructor/state/accessor/digest boundary while preserving the legacy family-unobserved digest path;
• 8a4b7a1b59353289d3cc33f80d852db9116f7096 adds completeness, duplicate-coordinate, conflicting-determinism, permutation-invariance, retained-evidence, and one-sided-nondeterministic FK edge contracts;
• 6fb0c2b673bec4784751cf2e2e90da55fc47458c aligns doctoring with the implemented bounded/cross-boundary boundary.

When the family is observed it covers every bounded relation column. Each column is explicitly uncollatable (attcollation=0) or carries exact qualified pg_collation identity plus collisdeterministic; repeated use of one exact collation coordinate cannot contradict its determinism. Input order is canonicalized and the family extends identity under conceptweave.postgres_schema_snapshot.v3.column_collations.v1. The original v3 constructor still means “column-collation family unobserved” and keeps its prior digest.

Column identity P1 — behavioral RED active

Review 5188215648 on exact predecessor 122dfa983ef1c2a9a1f6a68882eb12461243155d found that the successor column/public aggregate contract does not preserve PostgreSQL pg_attribute.attidentity. PostgreSQL 18 defines the empty value as not-identity, a as GENERATED ALWAYS, and d as GENERATED BY DEFAULT; ALWAYS and BY DEFAULT have different explicit-insert/override behavior. Type, nullable, default text, sequence naming, PK/UNIQUE, and index shape are not declaration authority.

Test-first commit 85f2eb0cdd80b6de6983895f598efe5e463da887 adds column_identity_contract.rs. It requires:

• GENERATED ALWAYS and GENERATED BY DEFAULT for one exact column to produce different governed identities;
• explicitly observed not-identity to remain distinct from an unobserved family;
• a claimed family to cover every bounded relation column;
• duplicate exact column coordinates to fail closed; and
• input-order permutations of identical complete evidence to retain one digest.

Doctoring 9bcc5e66c0eb6e0234ebcdb82ea8d5b5896b7138 records direct catalog authority, compatibility and inheritance/partition boundaries, rejected inference paths, and the separate future owner contract for linked identity-sequence options. Baseline successor 89f53ff08c9a1510b04b0207c2f6b10c1af0ed6e makes this RED state code-current.

Production does not yet provide ColumnIdentityObservation, new_with_column_identities, the observed-family accessor, or its domain-separated digest. The minimal repair must keep the existing v3 constructor/digest unchanged when the family is unobserved, use explicit not-identity/generated-always/generated-by-default states, validate exact bounded column completeness/duplicates, canonicalize input order, and keep sequence options out until separately observed rather than inferred.

Acceptance boundary

Current state is COLUMN_IDENTITY_RED_ACTIVE. Exact 89f53ff08c9a1510b04b0207c2f6b10c1af0ed6e is deliberately not source GREEN, native/Product GREEN, Ready, merge-authorized, published, or released.

After the identity-family production repair, one unchanged exact #46 successor must pass repository-pinned Rust 1.98 cargo fmt --all --check, strict workspace/all-target Clippy with warnings denied, column_identity_contract, the repaired column-collation contract plus all retained temporal/type/index contracts, workspace/doc tests, release build, owned production docstring/test/edge-case coverage, and applicable Product/security/dependency/review terminal evidence. The available execution host has no cargo/rustc; current branch pushes have not produced pull-request-triggered workflow runs, so native evidence cannot be substituted locally.

Do not use Draft/Ready toggles, no-op/manual retriggers, force-push, destructive rebase, self-approval, review dismissal, synthetic status, copied central workflows, or gate weakening.

Central owner path

Central workflow ownership remains outside ConceptWeave. Fresh owner state is ContextualWisdomLab/.github#2114@e7c58c04ed7e59c23cbe4a5f38d4c522ae712712, OPEN/non-Draft/mergeable, based on protected .github/main@fb17ef556f94f673234aa557254ae52779e9a7b0. SAST Semgrep 34716210489, Python Security 34716210535, Security Scan 34716210462, Runtime Quality 34716210506, and CodeQL PR 34716210555 are terminal GREEN. The same head still has OpenCode CHANGES_REQUESTED because Required Noema Review failed and the Strix review path was cancelled; no qualifying independent approval exists. No central evidence transfers to #46.

Only after one unchanged #46 successor repairs the active RED and reaches terminal exact-head GREEN should its complete delta be adopted ordinary/non-force into #45, followed by fresh parent acceptance and then #6. PostgreSQL transport and semantic publication remain later gates.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

P1 representation invariant: IndexObservation currently stores the key/INCLUDE role twice — by outer collection (key_attributes vs include_attributes) and by IndexAttributeKind — but IndexObservation::new does not require those two representations to agree. The current test even constructs an Include attribute inside key_attributes and a Key attribute inside include_attributes and accepts it as a valid fixture. PostgreSQL 18 pg_index defines indnkeyatts as the key-column count and indkey as indnatts entries with key columns before non-key included columns; therefore one observation must have one unambiguous ordinal boundary. A contradictory value object can otherwise produce a stable v3 digest/receipt for evidence that cannot faithfully correspond to the source catalog. Add a behavioral RED that rejects role/collection disagreement (and preferably a non-contiguous key→INCLUDE ordinal layout), then make construction fail closed before digest/snapshot creation. Preserve expression key attributes; do not infer from search_path. This is independent of the still-open #45 relation-kind coordinate finding.

[seonghobae]

Exact-current follow-up: review 5174516003 now has a committed behavioral RED contract at this head. index_layout_contract.rs requires IndexObservation::new to reject (a) collection/IndexAttributeKind disagreement and (b) non-contiguous key→INCLUDE ordinals, while accepting a contiguous key then INCLUDE layout. The production constructor in representation_v3.rs still only sorts the two vectors and therefore does not satisfy either RED. No hosted PR workflow has materialized for this head; combined status exposes CodeRabbit only, so this is intentionally RED/non-GREEN. Next causal repair is the constructor invariant plus correction of the inherited role_swapped test that currently treats contradictory evidence as a valid digest variant. Keep expression key support and keep #45's relation-kind coordinate finding separate.

[seonghobae]

Second exact-current representation finding: PostgreSQL 18 does not support expressions as INCLUDE/non-key payload columns, but the current VO permits IndexAttributeObservation::expression(..., IndexAttributeKind::Include, ...) and IndexObservation::new accepts it in include_attributes. Exact head a1eb5b242d7ecb4d2e4af51c7e69c286e204c34f adds that behavioral RED alongside the role/ordinal REDs. Repair the same index_attribute_layout boundary so INCLUDE entries are contiguous non-key column attributes only, while expressions remain valid key attributes (zero indkey source positions represented structurally, not by permitting expression INCLUDE). Do not claim GREEN until the production constructor and inherited contradictory digest fixture are repaired and fresh Rust 1.98/hosted evidence exists.

[seonghobae]

P1 docs-to-code authority finding: docs/product-technical-gap-baseline.md on this exact branch still snapshots 2026-09-10 and names obsolete active coordinates (#35 22709ec..., Source Observation #6 18d882..., central .github/main f578d8d...). Live authority is #35 9bb82f..., #6 287165d..., #45 6b2a8f..., this #46 a1eb5b24..., and central protected main cb0872c9.... The baseline also still describes first-class index representation as a future P0 rather than the current #46 implemented-but-RED state. Refresh the baseline in the same causal repair lineage after the index invariant fix; do not call this PR code-current or representation-complete while the canonical baseline contradicts live source/stack state.

[seonghobae]

P1 — current IndexObservation::validate_attribute_layout correctly repairs role/collection disagreement, non-contiguous ordinals, and expression INCLUDE, but it still admits key_attributes.is_empty(). That allows both an entirely empty index and an INCLUDE-only observation to enter v3 snapshot identity. PostgreSQL 18 CREATE INDEX requires at least one key index_elem in the parenthesized key list; INCLUDE is optional payload after that key list, not a substitute for it. See https://www.postgresql.org/docs/18/sql-createindex.html. Add a behavioral RED for empty and INCLUDE-only layouts, then minimally fail closed at the same index_attribute_layout constructor boundary when there is no key attribute. Preserve valid expression keys and current key→INCLUDE ordering semantics. Exact-head native/Product acceptance must be regenerated after the repair.

[seonghobae]

Exact-current follow-up: the PostgreSQL key-cardinality finding is source-addressed by 808faa922c466a586ea3bc8bf9f928f049ec3b95 (behavioral RED contract) -> 6dd59aef9a69e08aae6e7ede54ed1e204a634d9f (minimal constructor/rustdoc repair). 033ae72b95e3ef490c47ec84c191b83632601b10 then currentizes docs/product-technical-gap-baseline.md to the live #35/#6/#45/#46 and central .github/main authority without changing production behavior. Static inspection now has one coherent IndexObservation::new admission invariant: nonempty key set, role/collection agreement, contiguous one-based key→INCLUDE positions, expression keys permitted, INCLUDE columns only. This is source/docs GREEN only. Do not transfer historical Rust/Product evidence: exact 033ae72... still requires repository-pinned Rust 1.98 fmt, strict all-target Clippy, workspace tests, rustdoc/doc tests, release/coverage and applicable hosted Product/security/review checks before Ready/merge/adoption.

[seonghobae]

P1 semantic-identity finding on exact b2806d0b2d02d1a635d5a4850a88082c99c38fda.

The v3 index VO is still not lossless for PostgreSQL key semantics. IndexAttributeObservation currently preserves only position, Key/INCLUDE role, and column-vs-expression source. PostgreSQL 18 carries additional material per-key facts in pg_index: indcollation (key collation), indclass (operator class), and indoption (access-method-specific per-key flag bits). CREATE INDEX also exposes per-key COLLATE, operator class/parameters, ASC|DESC, and NULLS FIRST|LAST; these alter index behavior/use and are not equivalent metadata.

Because the digest currently encodes only the present attribute fields, two otherwise-identical v3 observations that differ solely in key collation/opclass/order/null placement can collapse to the same governed identity unless the optional reconstructed index_definition happens to be populated. A lossy optional text blob cannot substitute for first-class facts when this branch claims representation-complete index evidence.

Required RED before repair: construct two valid index observations over the same key/source that differ in one structured key semantic and prove distinct v3 digests; also prove missing/unresolvable qualified collation/opclass evidence fails closed where applicable. Minimal repair should extend key attributes (not INCLUDE payloads) with explicit canonical evidence for collation, opclass(+ parameters if captured), and access-method option semantics, preserving exact source coordinates without OID-as-identity or search_path inference. Keep server-reconstructed pg_get_indexdef as provenance text, not the sole semantic identity carrier.

Primary authority: PostgreSQL 18 pg_index (indcollation, indclass, indoption) and CREATE INDEX syntax/semantics. Do not attach transport or call the representation complete until this is RED→GREEN and exact-head accepted.

[seonghobae]

Exact-current follow-up on 189bfdc642b5ad14230bb32c6002db44d2865e4e: the baseline is now code-current for review 5175066205, but this is documentation/authority repair only. The per-key semantic-identity P1 is still open in production source: current IndexAttributeObservation has no first-class carrier for PostgreSQL 18 indcollation, indclass, or indoption, so do not call this head representation-complete or GREEN. Next source action remains RED first, then minimal structured VO/digest repair, then exact-head Rust/Product acceptance. Keep Draft; no predecessor execution evidence transfers across this docs head movement.

[seonghobae]

P1 refinement on exact 189bfdc642b5ad14230bb32c6002db44d2865e4e: the open per-key semantic gap is not only missing fields. PostgreSQL 18 defines indcollation, indclass, and indoption as arrays of exactly indnkeyatts entries, so they belong only to key positions; INCLUDE positions explicitly do not participate in index semantics. indoption is also interpreted by the index access method. The current v3 model keeps access_method optional and with_access_method accepts arbitrary text, so adding raw option bits without binding them to a nonblank observed access method would still leave governed identity semantically ambiguous.

RED/repair must therefore prove all of the following together: (1) one semantic record exists for every key position and for no INCLUDE position; (2) changing qualified collation, qualified operator class, or exact access-method option bits changes structured v3 identity even when pg_get_indexdef is absent; (3) blank/unqualified operator-class or collation coordinates fail closed; (4) a relation-attached index cannot enter a snapshot without a nonblank observed access method and complete per-key semantic evidence. Do not decode indoption into B-tree-specific ASC/DESC/NULLS flags in the generic domain layer; preserve the exact per-key bit pattern under its access-method binding and let a PostgreSQL adapter or access-method-specific layer interpret it. This keeps OIDs/search_path out of identity and preserves INCLUDE as payload-only.

[seonghobae]

Exact-current review on f24821ed91d9069fab3fbb073a5b7b1bb0a7d1e1: the source-level RED introduced at ebac140d... remains intentionally unsatisfied. This successor only currentizes the product/technical baseline; it does not add QualifiedOperatorClassName, IndexKeySemantics, with_key_semantics, relation/snapshot admission completeness, or v3 digest framing for per-key semantics. Fresh current-head evidence is still 0 pull-request workflow runs and CodeRabbit success only, so there is no Rust/Product GREEN to transfer. Keep Draft. Next causal source delta is the minimal production VO/admission/digest repair against the RED, followed by exact-head Rust 1.98 and applicable hosted acceptance.

[seonghobae]

Exact-current fixture-integrity checkpoint on fcb75659c4c7ffc046c4c7187789ef3b5d53715d: review 5187787381 was valid. The ordinary-forward successor from cdb50a49... through d704bf92... is 10 commits ahead / 0 behind and changes only nine retained contract-test files; production source is unchanged. Coherent temporal/timing/backing-index controls now satisfy the current relation/index admission contract with one IndexKeySemantics record per key plus explicit ready/valid/live=true where the index is promoted as constraint support; negative controls retain their intended missing/false evidence. constraint_period_type_contract.rs now reaches its type-kind assertions with complete exclusion-operator evidence, and the stale period-fixture import was removed. Baseline fcb75659... records the repair. This head is NOT execution GREEN: pull-request-triggered workflow runs are still 0, combined status is only CodeRabbit success, and this execution host has no Rust toolchain. Keep Draft; next admissible transition is unchanged-head Rust 1.98 + hosted Product/security/dependency/review acceptance, repairing only actual failures ordinary-forward.

[seonghobae]

P1 Source Observation semantic-identity gap on exact fcb75659c4c7ffc046c4c7187789ef3b5d53715d: v3 has a qualified collation value object for domains/index-key semantics, but ColumnObservationV3 itself carries only name/ordinal/display type/qualified type/nullability/comment. PostgreSQL 18 stores each column's defined/effective collation in pg_attribute.attcollation; that fact is material even outside indexes. Two otherwise-identical valid schemas whose collatable columns use different deterministic collations therefore collapse at the column-observation layer today. The gap is also correctness-relevant for foreign keys: PostgreSQL 18 requires each collatable referencing/referenced pair to use collations that are either both deterministic or exactly the same, so current governed FK/PERIOD evidence cannot verify that catalog rule.

Primary authority: PostgreSQL 18 pg_attribute.attcollation (https://www.postgresql.org/docs/18/catalog-pg-attribute.html), pg_collation.collisdeterministic (https://www.postgresql.org/docs/18/catalog-pg-collation.html), and CREATE TABLE foreign-key semantics (https://www.postgresql.org/docs/18/sql-createtable.html).

Do not patch this by deriving collation from data_type, a domain default, index indcollation, search_path, locale text, or OIDs. The source-authoritative repair needs an exact qualified column-collation binding and deterministic-collation evidence, while keeping OIDs adapter-local and preserving the existing v3 compatibility digest boundary through a domain-separated observed family (or an equivalent compatibility-safe successor layer). RED should first prove (1) same column/type with two exact collations is distinct governed evidence, (2) unobserved vs explicitly uncollatable is distinct, and (3) FK/PERIOD admission rejects a pair where either side is nondeterministic and the exact collation coordinates differ, while allowing both-deterministic and exact-same nondeterministic controls. Keep this PR Draft and reset exact-head acceptance after any repair.

[seonghobae]

Exact-current review on 1430947606da7392a9d167a618b07c26c5e84dae: column-collation P1 remains deliberately RED-active. Finding 5187855669, source-level RED 8006b24f..., primary-source doctoring 313f27c..., and code-current baseline 14309476... are aligned. No production ColumnCollationObservation/observed-family constructor, domain-separated digest framing, completeness/canonicalization, determinism consistency, or FK pair validation exists yet, so this head is not source GREEN. Current exact head has zero pull-request-triggered workflow runs and only CodeRabbit success; predecessor execution evidence cannot transfer. Keep Draft. The next causal delta is the compatibility-safe observed column-collation family and direct FK column-pair rule, without deriving truth from types/domains/indexes/search_path/OIDs; after any source movement, reacquire the full Rust 1.98 + hosted acceptance set before #45 adoption.

[seonghobae]

Exact-head repair review for 122dfa983ef1c2a9a1f6a68882eb12461243155d (COMMENT, not approval). The column-collation RED is now implemented at a compatibility-safe successor boundary rather than by mutating frozen ColumnObservationV3: ColumnCollationObservation records explicit uncollatable state or exact qualified collation plus collisdeterministic; the observed family is complete for bounded relation columns, canonicalizes order, rejects duplicate coordinates and contradictory determinism for one collation coordinate, and extends the predecessor digest under its own domain. Bounded FK pairs apply PostgreSQL 18's both-deterministic-or-exact-same rule; remote referenced relations are not assigned invented collation truth. Edge contracts now cover completeness, duplicate coordinates, conflicting determinism, permutation invariance, retained evidence, and one-sided nondeterminism. Static diff review found no intentional weakening of retained type/index/timing/PERIOD rules. Execution acceptance is still absent: this exact head has not produced PR-triggered Rust/Product runs, and this host has no Rust toolchain. Keep Draft; do not transfer predecessor evidence or adopt into #45/#6 until one unchanged exact head is terminal GREEN.

[seonghobae]

P1 Source Observation finding on exact 122dfa983ef1c2a9a1f6a68882eb12461243155d: ColumnObservationV3 and the public v3 aggregate preserve qualified type/null/comment plus optional collation, but no source-authoritative pg_attribute.attidentity family is present. PostgreSQL 18 defines attidentity=''|'a'|'d' as not-identity / GENERATED ALWAYS / GENERATED BY DEFAULT. ALWAYS versus BY DEFAULT changes INSERT/OVERRIDING behavior and is therefore material source schema semantics; identity is not implied by type, nullable, default text, sequence naming, PK/UNIQUE, or index shape. Current v3 can therefore collapse otherwise-identical schemas that differ only in identity generation mode. Repair must be compatibility-safe: leave the existing constructor/digest unchanged when the family is unobserved, add a complete bounded column-identity observed family with explicit not-identity/always/by-default states and its own digest domain, reject duplicate/missing column coordinates, preserve input-order invariance, and keep linked identity-sequence options for a separately versioned evidence family rather than guessing them. RED first; no Ready/merge until one exact successor passes retained Rust/Product gates.

[seonghobae]

Exact-current Source Observation review for 89f53ff08c9a1510b04b0207c2f6b10c1af0ed6e (COMMENT, not approval). Current lineage is 5188215648 -> 85f2eb0c... -> 9bcc5e66... -> 89f53ff.... The new column_identity_contract.rs is deliberately source/compile RED because production still has no ColumnIdentityObservation, new_with_column_identities, observed-family accessor, or domain-separated identity layer. The finding remains valid: pg_attribute.attidentity is direct declaration authority and the a/d modes are materially different. Minimal repair is the optional complete bounded column-identity family only; do not alter the legacy unobserved v3 digest, infer identity from defaults/sequences/PK/index shape, or pull sequence configuration into this family. Retained column-collation/index/lifecycle/temporal contracts stay intact. Keep Draft until repair and one unchanged exact-head Rust 1.98 + hosted acceptance are GREEN.