Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ DICE (Domain-Integrated Context Engineering) is a proposition-first knowledge su
| `dice-storage-autoconfigure` | Spring Boot auto-configuration that wires the right backend based on `embabel.dice.store.type`, schedules the decay tick, and provides auto-configuration for the multi-signal duplicate collector (properties prefix `embabel.dice.collector`) |
| `dice-report` | Output projectors over propositions: rationale (why a fact is believed, with evidence), structured report, and surprising-link discovery |
| `dice-ingestion` | Ingestion SPI (artifacts → chunks) with a content-hash dedup ledger so the same source isn't extracted twice |
| `dice-metamodel` | Schema versioning: `MetamodelVersion` content-hash stamps over the governed types of a `DataDictionary`, `GovernedTypeSelector`, the `DeclaredSchemaSource` opt-in, and the `MetamodelVersionStore` contract. Pure JVM, with no dependency on `dice` |
| `dice-integration-tests` | Test-only: the cross-feature end-to-end canonical-flow harness |

## Build & test
Expand Down
47 changes: 47 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Changelog

Notable changes to DICE. Each entry states its compatibility impact on consumers
(anything tracking `0.2.0-SNAPSHOT`): **additive** (safe to
pick up), **behavioral** (same API, different runtime behavior — read the note),
or **breaking** (consumer change required; the entry links the migration notes
and the consumer PRs that deliver it).

## Unreleased

### Added

- `dice-metamodel` module, first slice of schema versioning: `MetamodelVersion`
content-hash stamping with per-type governance selection, the declared-schema
opt-in seam, and the `MetamodelVersionStore` contract. Pure JVM.
**Compatibility: additive.** New module; no existing API touched.

- Declared renames in `dice-metamodel`. **EXPERIMENTAL** (shape may change
before 1.0): `SchemaAliases`, `PropertySignature.aliases`, and
`MetamodelVersion.entityTypeAliases`. A declaration states the names a type or
property used to go by, so a later comparison pairs a rename instead of
reading it as a removal and an addition.
**Compatibility: additive.** `contentHash` is unchanged for any schema that
declares no aliases — the new hash blocks serialize only when non-empty, and
the pinned golden digest is asserted unchanged, including for a stamp rebuilt
through the public constructor. The new constructor parameters carry
`@JvmOverloads`, so the existing `PropertySignature(String, Kind, String,
Cardinality)` and `MetamodelVersion(String, List, Map, Map, List)` descriptors
survive. `SchemaAliases` reaches `MetamodelVersion.from` and
`DeclaredSchema.from` through separate overloads that take it as a required
parameter, which leaves the shipped one- and two-argument forms and their
Kotlin `from$default` synthetics byte-identical; widening those functions with
a third defaulted parameter would have replaced
`DeclaredSchema.Companion.from$default(Companion, DataDictionary,
GovernedTypeSelector, int, Object)` with a wider descriptor and broken any
caller already compiled against it. `MetamodelJavaCompatTest` calls every
static form. The changed Kotlin synthetic constructor, `copy`, `copy$default`
and `componentN` signatures on `PropertySignature` are the accepted boundary:
Kotlin callers recompile, and no consumer holds a compiled reference to them.

- Schema attribution mechanism: per-proposition version is answered through the
run that produced the proposition (PRODUCED_BY_RUN). The run record carries
the declared schema's content hash, resolved by the extraction coordinator from
the host's DeclaredSchemaSource. The `DiceMetadataKeys.METAMODEL_VERSION`
Comment thread
jimador marked this conversation as resolved.
metadata key is removed; lineage answers per-proposition attribution.
**Compatibility: breaking.** The key is no longer available; code holding it
must migrate to extraction-run queries.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ recover by reading a single class — see the design notes in [`docs/design/`](d
- [Durable storage](docs/design/durable-storage.md) — backend selection, defense-in-depth dedup,
two-phase save, materialised effective confidence, schema-as-beans, and the decay tick.
- [Events](docs/design/events.md) — the domain-event model the store and pipeline emit.
- [Metamodel versioning](docs/design/metamodel-versioning.md) — content-hash schema stamps, per-type
governance, the declared-schema opt-in, and version history.

## Real-World Example: Impromptu

Expand Down
93 changes: 93 additions & 0 deletions dice-metamodel/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.embabel.dice</groupId>
<artifactId>dice-parent</artifactId>
<version>0.2.0-SNAPSHOT</version>
</parent>
<artifactId>dice-metamodel</artifactId>
<packaging>jar</packaging>
<name>Dice Metamodel</name>
<description>Schema versioning for DICE knowledge graphs: content-hash stamping, the declared-schema contract, and the version store contract</description>

<dependencies>
<!--
Embabel agent core types (DataDictionary, DomainType); provided, supplied by the
consuming app. This module deliberately does not depend on dice core — it stamps a
schema and nothing more, so it never touches the Proposition model.
-->
<dependency>
<groupId>com.embabel.agent</groupId>
<artifactId>embabel-agent-api</artifactId>
<scope>provided</scope>
</dependency>

<!--
https://mvnrepository.com/artifact/org.jetbrains/annotations
For Experimental annotation
-->
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<scope>provided</scope>
</dependency>

<!-- Test -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<!--
Default interface methods available for Java consumers. The test-compile
execution also picks up src/test/java, where MetamodelJavaCompatTest calls the
JVM constructor and factory descriptors a compiled Java consumer is linked
against.
-->
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<configuration>
<args>
<arg>-Xjvm-default=all</arg>
</args>
</configuration>
<executions>
<execution>
<id>compile</id>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
<configuration>
<sourceDirs>
<source>src/main/java</source>
<source>src/main/kotlin</source>
</sourceDirs>
</configuration>
</execution>
<execution>
<id>test-compile</id>
<phase>test-compile</phase>
<goals>
<goal>test-compile</goal>
</goals>
<configuration>
<sourceDirs>
<source>src/test/java</source>
<source>src/test/kotlin</source>
</sourceDirs>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
/*
* Copyright 2024-2026 Embabel Pty Ltd.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.embabel.dice.metamodel

import com.embabel.agent.core.DataDictionary
import org.jetbrains.annotations.ApiStatus

/**
* The schema as declared: the stamped [version] plus the bare relationship type names it allows.
*
* The bare names travel alongside the stamp rather than being recovered from it, because
* [MetamodelVersion.relationshipNames] holds rendered `From-[name]->To` descriptors and
* reverse-parsing one is ambiguous: these names come from free-text and LLM extraction, and can
* themselves contain a `-[...]->`-shaped substring. Whoever builds a declaration holds the
* un-rendered names before anything is stamped, so it passes them straight through. Comparing
* declared relationships against what a graph holds needs those bare names, and that comparison is
* the next slice.
*
* @property version The stamped declared schema.
* @property relationshipTypeNames The bare relationship type names [version] allows.
*/
@ApiStatus.Experimental
class DeclaredSchema(
val version: MetamodelVersion,
relationshipTypeNames: Set<String>,
) {

/** Copied into a JVM-immutable set so the declaration can't drift from its stamped [version]. */
val relationshipTypeNames: Set<String> = java.util.Set.copyOf(relationshipTypeNames)
Comment thread
jimador marked this conversation as resolved.

override fun equals(other: Any?): Boolean =
other is DeclaredSchema &&
version == other.version &&
relationshipTypeNames == other.relationshipTypeNames

override fun hashCode(): Int = 31 * version.hashCode() + relationshipTypeNames.hashCode()

override fun toString(): String =
"DeclaredSchema(version=$version, relationshipTypeNames=$relationshipTypeNames)"

companion object {

/**
* Declare the governed part of [dataDictionary]: stamp it and carry through the bare
* relationship names the same governed types declare.
*
* Use this rather than building the two halves separately. Picking a governed subset for
* the stamp while taking relationship names from the whole dictionary would declare
* relationships the stamp never covered, and the mismatch would only surface much later,
* as phantom disagreement in a drift check.
*
* @param dataDictionary The schema to declare.
* @param selector Which types are under governance. Defaults to all of them.
* @return The declaration.
*/
@JvmStatic
@JvmOverloads
fun from(
dataDictionary: DataDictionary,
selector: GovernedTypeSelector = GovernedTypeSelector.ALL,
): DeclaredSchema = from(dataDictionary, selector, SchemaAliases.NONE)

/**
* Declare the governed part of [dataDictionary], carrying the former names [aliases]
* declares for its types and properties.
*
* This is how a rename gets recorded as one. The stamp carries the former names, and a
* later comparison pairs the old name with the new one rather than reading the change as a
* type or property disappearing.
*
* [aliases] has no default, and the shorter form above stays a separate function. Adding a
* third defaulted parameter to it would replace its synthetic `from$default` descriptor
* with a wider one, which is a link error for any caller already compiled against it.
*
* @param dataDictionary The schema to declare.
* @param selector Which types are under governance. [GovernedTypeSelector.ALL] governs all
* of them.
* @param aliases Former names for the schema's types and properties. [SchemaAliases.NONE]
* declares none. Experimental: shape may change before 1.0.
* @return The declaration.
* @throws IllegalArgumentException when a declared type name appears in another type's
* alias set, or when aliases are declared for a property name the governed types hold
* more than one signature for.
*/
@JvmStatic
fun from(
dataDictionary: DataDictionary,
selector: GovernedTypeSelector,
aliases: SchemaAliases,
): DeclaredSchema = DeclaredSchema(
version = MetamodelVersion.from(dataDictionary, selector, aliases),
relationshipTypeNames = MetamodelVersion.governedRelationshipTypeNames(dataDictionary, selector),
)
}
}

/**
* Supplies the schema an application has declared.
*
* This interface is the versioning opt-in: with no declared schema, nothing is stamped. The Spring
* wiring that arrives in a later slice activates only when a `DeclaredSchemaSource` bean is
* present, so an application that hasn't decided what it governs is left alone.
*
* A declared schema can come from anywhere. A consuming app implements this over whatever it
* already uses to define its types (a `DataDictionary`, a config file, a registry...) and wires it
* as a bean. There is no default implementation, because there is no default declared schema.
*/
@ApiStatus.Experimental
fun interface DeclaredSchemaSource {

/**
* @return the current [DeclaredSchema].
*/
fun declare(): DeclaredSchema
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
* Copyright 2024-2026 Embabel Pty Ltd.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.embabel.dice.metamodel

import com.embabel.agent.core.DomainType
import org.jetbrains.annotations.ApiStatus

/**
* Decides which domain types a [MetamodelVersion] stamp covers.
*
* A DICE domain is rarely all one thing. Part of it is closed-world: the types you have committed
* to, whose shape you want to notice changing. The rest is open-world: exploratory types that
* extraction proposes, that come and go, and that would otherwise break a version comparison every
* time an LLM invents one. Governance is therefore per type and opt-in, in the spirit of
* Hibernate's `@Version`.
*
* A selector is a predicate over types, so the usual form is a lambda over names you already hold:
*
* ```kotlin
* val governed = setOf("Person", "Company")
* val version = MetamodelVersion.from(dataDictionary, GovernedTypeSelector { it.name in governed })
* ```
*
* Selecting a subset changes which types the stamp covers, and leaves the encoding alone. Adding an
* ungoverned type to the dictionary leaves the content hash as it was, while touching a governed
* one changes it.
*/
@ApiStatus.Experimental
fun interface GovernedTypeSelector {
Comment thread
jimador marked this conversation as resolved.

/**
* @param type A domain type from the dictionary being stamped.
* @return `true` when this type is under version governance and belongs in the stamp.
*/
fun governs(type: DomainType): Boolean

companion object {

/**
* Governs every type in the dictionary. This is the whole-schema stamp, and what
* [MetamodelVersion.from] uses when no selector is given.
*/
@JvmField
val ALL: GovernedTypeSelector = GovernedTypeSelector { true }
}
}
Loading
Loading