-
Notifications
You must be signed in to change notification settings - Fork 10
feat(metamodel): schema versioning core with per-type governance #83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
4fecea7
feat(metamodel): schema versioning core with per-type governance
jimador e087a80
docs(metamodel): voice pass on versioning docs and KDoc
jimador a6d827b
Add declared aliases and stamp provenance to metamodel versioning
jimador d8bd318
Remove stamp provenance until a caller records it
jimador 0a011f5
Define the stamping contract for dice.metamodel.version
jimador 1226c14
Retire the word seam from versioning docs and test names
jimador abbd3da
Answer schema attribution through run lineage
jimador 2fcca07
Mark the versioning surface experimental
jimador 726a51d
Name no consumer in the changelog header
jimador 3c2fcda
Answer the code review on the metamodel stamp
jimador e423a0a
Answer the agent findings on the metamodel stamp
jimador File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # Changelog | ||
|
|
||
| Notable changes to DICE. Each entry states its compatibility impact on consumers | ||
| (anything tracking `0.2.0-SNAPSHOT`): **additive** (safe to | ||
| pick up), **behavioral** (same API, different runtime behavior — read the note), | ||
| or **breaking** (consumer change required; the entry links the migration notes | ||
| and the consumer PRs that deliver it). | ||
|
|
||
| ## Unreleased | ||
|
|
||
| ### Added | ||
|
|
||
| - `dice-metamodel` module, first slice of schema versioning: `MetamodelVersion` | ||
| content-hash stamping with per-type governance selection, the declared-schema | ||
| opt-in seam, and the `MetamodelVersionStore` contract. Pure JVM. | ||
| **Compatibility: additive.** New module; no existing API touched. | ||
|
|
||
| - Declared renames in `dice-metamodel`. **EXPERIMENTAL** (shape may change | ||
| before 1.0): `SchemaAliases`, `PropertySignature.aliases`, and | ||
| `MetamodelVersion.entityTypeAliases`. A declaration states the names a type or | ||
| property used to go by, so a later comparison pairs a rename instead of | ||
| reading it as a removal and an addition. | ||
| **Compatibility: additive.** `contentHash` is unchanged for any schema that | ||
| declares no aliases — the new hash blocks serialize only when non-empty, and | ||
| the pinned golden digest is asserted unchanged, including for a stamp rebuilt | ||
| through the public constructor. The new constructor parameters carry | ||
| `@JvmOverloads`, so the existing `PropertySignature(String, Kind, String, | ||
| Cardinality)` and `MetamodelVersion(String, List, Map, Map, List)` descriptors | ||
| survive. `SchemaAliases` reaches `MetamodelVersion.from` and | ||
| `DeclaredSchema.from` through separate overloads that take it as a required | ||
| parameter, which leaves the shipped one- and two-argument forms and their | ||
| Kotlin `from$default` synthetics byte-identical; widening those functions with | ||
| a third defaulted parameter would have replaced | ||
| `DeclaredSchema.Companion.from$default(Companion, DataDictionary, | ||
| GovernedTypeSelector, int, Object)` with a wider descriptor and broken any | ||
| caller already compiled against it. `MetamodelJavaCompatTest` calls every | ||
| static form. The changed Kotlin synthetic constructor, `copy`, `copy$default` | ||
| and `componentN` signatures on `PropertySignature` are the accepted boundary: | ||
| Kotlin callers recompile, and no consumer holds a compiled reference to them. | ||
|
|
||
| - Schema attribution mechanism: per-proposition version is answered through the | ||
| run that produced the proposition (PRODUCED_BY_RUN). The run record carries | ||
| the declared schema's content hash, resolved by the extraction coordinator from | ||
| the host's DeclaredSchemaSource. The `DiceMetadataKeys.METAMODEL_VERSION` | ||
| metadata key is removed; lineage answers per-proposition attribution. | ||
| **Compatibility: breaking.** The key is no longer available; code holding it | ||
| must migrate to extraction-run queries. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
| <parent> | ||
| <groupId>com.embabel.dice</groupId> | ||
| <artifactId>dice-parent</artifactId> | ||
| <version>0.2.0-SNAPSHOT</version> | ||
| </parent> | ||
| <artifactId>dice-metamodel</artifactId> | ||
| <packaging>jar</packaging> | ||
| <name>Dice Metamodel</name> | ||
| <description>Schema versioning for DICE knowledge graphs: content-hash stamping, the declared-schema contract, and the version store contract</description> | ||
|
|
||
| <dependencies> | ||
| <!-- | ||
| Embabel agent core types (DataDictionary, DomainType); provided, supplied by the | ||
| consuming app. This module deliberately does not depend on dice core — it stamps a | ||
| schema and nothing more, so it never touches the Proposition model. | ||
| --> | ||
| <dependency> | ||
| <groupId>com.embabel.agent</groupId> | ||
| <artifactId>embabel-agent-api</artifactId> | ||
| <scope>provided</scope> | ||
| </dependency> | ||
|
|
||
| <!-- | ||
| https://mvnrepository.com/artifact/org.jetbrains/annotations | ||
| For Experimental annotation | ||
| --> | ||
| <dependency> | ||
| <groupId>org.jetbrains</groupId> | ||
| <artifactId>annotations</artifactId> | ||
| <scope>provided</scope> | ||
| </dependency> | ||
|
|
||
| <!-- Test --> | ||
| <dependency> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-test</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
| <!-- | ||
| Default interface methods available for Java consumers. The test-compile | ||
| execution also picks up src/test/java, where MetamodelJavaCompatTest calls the | ||
| JVM constructor and factory descriptors a compiled Java consumer is linked | ||
| against. | ||
| --> | ||
| <groupId>org.jetbrains.kotlin</groupId> | ||
| <artifactId>kotlin-maven-plugin</artifactId> | ||
| <configuration> | ||
| <args> | ||
| <arg>-Xjvm-default=all</arg> | ||
| </args> | ||
| </configuration> | ||
| <executions> | ||
| <execution> | ||
| <id>compile</id> | ||
| <phase>compile</phase> | ||
| <goals> | ||
| <goal>compile</goal> | ||
| </goals> | ||
| <configuration> | ||
| <sourceDirs> | ||
| <source>src/main/java</source> | ||
| <source>src/main/kotlin</source> | ||
| </sourceDirs> | ||
| </configuration> | ||
| </execution> | ||
| <execution> | ||
| <id>test-compile</id> | ||
| <phase>test-compile</phase> | ||
| <goals> | ||
| <goal>test-compile</goal> | ||
| </goals> | ||
| <configuration> | ||
| <sourceDirs> | ||
| <source>src/test/java</source> | ||
| <source>src/test/kotlin</source> | ||
| </sourceDirs> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
|
|
||
| </project> |
128 changes: 128 additions & 0 deletions
128
dice-metamodel/src/main/kotlin/com/embabel/dice/metamodel/DeclaredSchemaSource.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| /* | ||
| * Copyright 2024-2026 Embabel Pty Ltd. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package com.embabel.dice.metamodel | ||
|
|
||
| import com.embabel.agent.core.DataDictionary | ||
| import org.jetbrains.annotations.ApiStatus | ||
|
|
||
| /** | ||
| * The schema as declared: the stamped [version] plus the bare relationship type names it allows. | ||
| * | ||
| * The bare names travel alongside the stamp rather than being recovered from it, because | ||
| * [MetamodelVersion.relationshipNames] holds rendered `From-[name]->To` descriptors and | ||
| * reverse-parsing one is ambiguous: these names come from free-text and LLM extraction, and can | ||
| * themselves contain a `-[...]->`-shaped substring. Whoever builds a declaration holds the | ||
| * un-rendered names before anything is stamped, so it passes them straight through. Comparing | ||
| * declared relationships against what a graph holds needs those bare names, and that comparison is | ||
| * the next slice. | ||
| * | ||
| * @property version The stamped declared schema. | ||
| * @property relationshipTypeNames The bare relationship type names [version] allows. | ||
| */ | ||
| @ApiStatus.Experimental | ||
| class DeclaredSchema( | ||
| val version: MetamodelVersion, | ||
| relationshipTypeNames: Set<String>, | ||
| ) { | ||
|
|
||
| /** Copied into a JVM-immutable set so the declaration can't drift from its stamped [version]. */ | ||
| val relationshipTypeNames: Set<String> = java.util.Set.copyOf(relationshipTypeNames) | ||
|
jimador marked this conversation as resolved.
|
||
|
|
||
| override fun equals(other: Any?): Boolean = | ||
| other is DeclaredSchema && | ||
| version == other.version && | ||
| relationshipTypeNames == other.relationshipTypeNames | ||
|
|
||
| override fun hashCode(): Int = 31 * version.hashCode() + relationshipTypeNames.hashCode() | ||
|
|
||
| override fun toString(): String = | ||
| "DeclaredSchema(version=$version, relationshipTypeNames=$relationshipTypeNames)" | ||
|
|
||
| companion object { | ||
|
|
||
| /** | ||
| * Declare the governed part of [dataDictionary]: stamp it and carry through the bare | ||
| * relationship names the same governed types declare. | ||
| * | ||
| * Use this rather than building the two halves separately. Picking a governed subset for | ||
| * the stamp while taking relationship names from the whole dictionary would declare | ||
| * relationships the stamp never covered, and the mismatch would only surface much later, | ||
| * as phantom disagreement in a drift check. | ||
| * | ||
| * @param dataDictionary The schema to declare. | ||
| * @param selector Which types are under governance. Defaults to all of them. | ||
| * @return The declaration. | ||
| */ | ||
| @JvmStatic | ||
| @JvmOverloads | ||
| fun from( | ||
| dataDictionary: DataDictionary, | ||
| selector: GovernedTypeSelector = GovernedTypeSelector.ALL, | ||
| ): DeclaredSchema = from(dataDictionary, selector, SchemaAliases.NONE) | ||
|
|
||
| /** | ||
| * Declare the governed part of [dataDictionary], carrying the former names [aliases] | ||
| * declares for its types and properties. | ||
| * | ||
| * This is how a rename gets recorded as one. The stamp carries the former names, and a | ||
| * later comparison pairs the old name with the new one rather than reading the change as a | ||
| * type or property disappearing. | ||
| * | ||
| * [aliases] has no default, and the shorter form above stays a separate function. Adding a | ||
| * third defaulted parameter to it would replace its synthetic `from$default` descriptor | ||
| * with a wider one, which is a link error for any caller already compiled against it. | ||
| * | ||
| * @param dataDictionary The schema to declare. | ||
| * @param selector Which types are under governance. [GovernedTypeSelector.ALL] governs all | ||
| * of them. | ||
| * @param aliases Former names for the schema's types and properties. [SchemaAliases.NONE] | ||
| * declares none. Experimental: shape may change before 1.0. | ||
| * @return The declaration. | ||
| * @throws IllegalArgumentException when a declared type name appears in another type's | ||
| * alias set, or when aliases are declared for a property name the governed types hold | ||
| * more than one signature for. | ||
| */ | ||
| @JvmStatic | ||
| fun from( | ||
| dataDictionary: DataDictionary, | ||
| selector: GovernedTypeSelector, | ||
| aliases: SchemaAliases, | ||
| ): DeclaredSchema = DeclaredSchema( | ||
| version = MetamodelVersion.from(dataDictionary, selector, aliases), | ||
| relationshipTypeNames = MetamodelVersion.governedRelationshipTypeNames(dataDictionary, selector), | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Supplies the schema an application has declared. | ||
| * | ||
| * This interface is the versioning opt-in: with no declared schema, nothing is stamped. The Spring | ||
| * wiring that arrives in a later slice activates only when a `DeclaredSchemaSource` bean is | ||
| * present, so an application that hasn't decided what it governs is left alone. | ||
| * | ||
| * A declared schema can come from anywhere. A consuming app implements this over whatever it | ||
| * already uses to define its types (a `DataDictionary`, a config file, a registry...) and wires it | ||
| * as a bean. There is no default implementation, because there is no default declared schema. | ||
| */ | ||
| @ApiStatus.Experimental | ||
| fun interface DeclaredSchemaSource { | ||
|
|
||
| /** | ||
| * @return the current [DeclaredSchema]. | ||
| */ | ||
| fun declare(): DeclaredSchema | ||
| } | ||
59 changes: 59 additions & 0 deletions
59
dice-metamodel/src/main/kotlin/com/embabel/dice/metamodel/GovernedTypeSelector.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| /* | ||
| * Copyright 2024-2026 Embabel Pty Ltd. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package com.embabel.dice.metamodel | ||
|
|
||
| import com.embabel.agent.core.DomainType | ||
| import org.jetbrains.annotations.ApiStatus | ||
|
|
||
| /** | ||
| * Decides which domain types a [MetamodelVersion] stamp covers. | ||
| * | ||
| * A DICE domain is rarely all one thing. Part of it is closed-world: the types you have committed | ||
| * to, whose shape you want to notice changing. The rest is open-world: exploratory types that | ||
| * extraction proposes, that come and go, and that would otherwise break a version comparison every | ||
| * time an LLM invents one. Governance is therefore per type and opt-in, in the spirit of | ||
| * Hibernate's `@Version`. | ||
| * | ||
| * A selector is a predicate over types, so the usual form is a lambda over names you already hold: | ||
| * | ||
| * ```kotlin | ||
| * val governed = setOf("Person", "Company") | ||
| * val version = MetamodelVersion.from(dataDictionary, GovernedTypeSelector { it.name in governed }) | ||
| * ``` | ||
| * | ||
| * Selecting a subset changes which types the stamp covers, and leaves the encoding alone. Adding an | ||
| * ungoverned type to the dictionary leaves the content hash as it was, while touching a governed | ||
| * one changes it. | ||
| */ | ||
| @ApiStatus.Experimental | ||
| fun interface GovernedTypeSelector { | ||
|
jimador marked this conversation as resolved.
|
||
|
|
||
| /** | ||
| * @param type A domain type from the dictionary being stamped. | ||
| * @return `true` when this type is under version governance and belongs in the stamp. | ||
| */ | ||
| fun governs(type: DomainType): Boolean | ||
|
|
||
| companion object { | ||
|
|
||
| /** | ||
| * Governs every type in the dictionary. This is the whole-schema stamp, and what | ||
| * [MetamodelVersion.from] uses when no selector is given. | ||
| */ | ||
| @JvmField | ||
| val ALL: GovernedTypeSelector = GovernedTypeSelector { true } | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.