Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,34 @@ and the consumer PRs that deliver it).
metadata key is removed; lineage answers per-proposition attribution.
**Compatibility: breaking.** The key is no longer available; code holding it
must migrate to extraction-run queries.
- Drivine/Neo4j-backed `MetamodelVersionStore` in `dice-storage`
(`DrivineMetamodelVersionStore`): stamps persist as `(:MetamodelVersion)` nodes,
MERGEd on the natural key `(schemaName, contentHash)`, so a re-stamp updates in
place. `latestVersion`, `versionHistory` and `findVersion` all resolve in Cypher.
History is ordered by a persisted per-schema sequence, taken off a
`(:MetamodelSchemaCounter)` node in the same statement that creates the version;
`savedAt` and `savedAtEpochMillis` are informational, and nothing sorts on them.
An idempotent re-save leaves the counter and the sequence alone. Concurrent saves
of one version leave one node. Hosts must declare three uniqueness constraints:
`MetamodelVersion(schemaName, contentHash)`, `MetamodelSchemaCounter(schemaName)`,
and `MetamodelVersion(schemaName, sequence)`.
Declared aliases persist at both levels: the version-level `entityTypeAliases` map
as its own node property, and a property signature's former names as a fifth
`aliases` field inside the stored signature. Both are written only when they hold
something, so an alias-free stamp writes exactly the properties this mapper wrote
before aliases existed, and a node from that older build reads back as a stamp
declaring none. Aliases feed `contentHash`, and the mapper recomputes the hash from
the persisted fields, so a stamp that failed to store them would be unreadable for
good — pinned by an integration test that writes a row in the old four-field shape
through raw Cypher and reads it back, one that round-trips a stamp carrying both
alias kinds, and one that removes the stored alias map and asserts the integrity
check rejects the row.
`savedAt` and `savedAtEpochMillis` keep their existing behavior: set on create,
untouched by a re-save. `dice-metamodel` gains `InMemoryMetamodelVersionStore`, the
reference implementation of the store contract, promoted from a private class in
that module's own tests. `AbstractMetamodelVersionStoreContractTest` runs one suite
against both stores.
**Compatibility: additive.** New classes, and a new `dice-storage` → `dice-metamodel`
module dependency; no existing API touched. Stored nodes stay readable: every
property that existed before keeps its name, meaning, and encoding, and the two
new alias fields are absent when nothing declares them.
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
* Copyright 2024-2026 Embabel Pty Ltd.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.embabel.dice.metamodel
Comment thread
jimador marked this conversation as resolved.

/**
* Reference [MetamodelVersionStore] that keeps stamps in a list.
*
* It is the executable statement of what the contract means, so a durable backend can be held to
* the same suite of tests. It also lets a host stamp and compare schemas before it has a database,
* which is most of what the first tier of versioning is for.
*
* Nothing here survives the JVM, and two instances know nothing about each other.
*/
class InMemoryMetamodelVersionStore : MetamodelVersionStore {

private val saved = mutableListOf<MetamodelVersion>()

/**
* Upsert on `(schemaName, contentHash)`. A stamp that is already there keeps its place in the
* write order, so re-saving an old version doesn't make it the latest; the incoming stamp
* replaces it in place.
*
* Everything runs under the list's own lock, so two threads saving the same version can't
* interleave the search for an existing stamp with the write that lands the new one.
*/
override fun saveVersion(version: MetamodelVersion) {
synchronized(saved) {
val existingIndex = saved.indexOfFirst {
it.schemaName == version.schemaName && it.contentHash == version.contentHash
}
if (existingIndex < 0) saved += version else saved[existingIndex] = version
}
}

override fun latestVersion(schemaName: String): MetamodelVersion? =
synchronized(saved) { saved.lastOrNull { it.schemaName == schemaName } }

override fun versionHistory(schemaName: String): List<MetamodelVersion> =
synchronized(saved) { saved.filter { it.schemaName == schemaName }.reversed() }
}
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,17 @@ interface MetamodelVersionStore {
* Save a version stamp, keyed on `(schemaName, contentHash)`. Saving the same version twice
* leaves one stored version.
*
* Everything about a stored stamp is content the key already determines, so a re-save
* overwrites it with an identical value.
*
* Whatever an implementation records as the moment of the save keeps its existing value on a
* re-save, along with the stamp's place in the write order: a re-saved old stamp does not
* become the latest.
*
* An implementation may fail a save with its backend's own concurrency exception when two
* writers race to save the same schema at once. Since the write is idempotent, the caller can
* simply retry it.
*
* @param version The version to save.
*/
fun saveVersion(version: MetamodelVersion)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,36 +24,21 @@ import org.junit.jupiter.api.Test
* Covers the one piece of behaviour the contract itself ships: the default [findVersion], which a
* backend is free to override with a keyed lookup. A store implementation gets its own tests
* wherever it lives.
*
* The upsert rules [MetamodelVersionStore.saveVersion] states are checked by
* `AbstractMetamodelVersionStoreContractTest`, which runs the same suite against
* [InMemoryMetamodelVersionStore] and the graph-backed store.
*/
class MetamodelVersionStoreTest {

/** Minimal store honouring the contract: upsert on (schemaName, contentHash), newest first. */
private class InMemoryVersionStore : MetamodelVersionStore {

private val saved = mutableListOf<MetamodelVersion>()

override fun saveVersion(version: MetamodelVersion) {
// Idempotent: re-saving an existing version keeps its original position in write order.
if (saved.none { it.schemaName == version.schemaName && it.contentHash == version.contentHash }) {
saved.add(version)
}
}

override fun latestVersion(schemaName: String): MetamodelVersion? =
versionHistory(schemaName).firstOrNull()

override fun versionHistory(schemaName: String): List<MetamodelVersion> =
saved.filter { it.schemaName == schemaName }.reversed()
}

private fun version(schemaName: String, vararg typeNames: String): MetamodelVersion =
MetamodelVersion.from(
DataDictionary.fromDomainTypes(schemaName, typeNames.map { DynamicType(name = it) }),
)

@Test
fun `findVersion returns the stamp with that hash`() {
val store = InMemoryVersionStore()
val store = InMemoryMetamodelVersionStore()
val first = version("app", "Person")
val second = version("app", "Person", "Company")
store.saveVersion(first)
Expand All @@ -67,7 +52,7 @@ class MetamodelVersionStoreTest {
fun `findVersion is scoped to the schema name`() {
// Two schemas can hold structurally identical versions, because the hash excludes the
// name, so the lookup has to match on both halves of the key.
val store = InMemoryVersionStore()
val store = InMemoryMetamodelVersionStore()
val mine = version("mine", "Person")
store.saveVersion(mine)

Expand All @@ -77,15 +62,15 @@ class MetamodelVersionStoreTest {

@Test
fun `findVersion returns null for an unknown hash`() {
val store = InMemoryVersionStore()
val store = InMemoryMetamodelVersionStore()
store.saveVersion(version("app", "Person"))

assertNull(store.findVersion("app", "not-a-hash"))
}

@Test
fun `re-saving a version leaves one record, not two`() {
val store = InMemoryVersionStore()
val store = InMemoryMetamodelVersionStore()
val v = version("app", "Person")
store.saveVersion(v)
store.saveVersion(v)
Expand All @@ -96,7 +81,7 @@ class MetamodelVersionStoreTest {

@Test
fun `an empty store has no latest version and an empty history`() {
val store = InMemoryVersionStore()
val store = InMemoryMetamodelVersionStore()

assertNull(store.latestVersion("app"))
assertEquals(emptyList<MetamodelVersion>(), store.versionHistory("app"))
Expand Down
6 changes: 6 additions & 0 deletions dice-storage/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@
<artifactId>dice</artifactId>
</dependency>

<!-- Dice metamodel: the MetamodelVersionStore contract and the MetamodelVersion stamp it persists -->
<dependency>
<groupId>com.embabel.dice</groupId>
<artifactId>dice-metamodel</artifactId>
</dependency>

<!-- Embabel agent types referenced by the repository/mapper (SimilarityResult, Cluster, etc.) -->
<dependency>
<groupId>com.embabel.agent</groupId>
Expand Down
Loading
Loading