Skip to content

Commit c841acf

Browse files
committed
🧪 Add a seeded differential corpus with known-drift gates
Generate deterministic DDL DML expression aggregate window and transaction cases and compare every case against both external oracles Pin the reproducible round differences as known drift and keep the remaining mismatches in the capability matrix without weakening assertions or changing the facade
1 parent e8f68ee commit c841acf

7 files changed

Lines changed: 714 additions & 9 deletions

File tree

‎docs/sql-capability-matrix.md‎

Lines changed: 34 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -201,11 +201,39 @@ asserted as structured rejections by `unsupported foreign key shapes keep struct
201201

202202
| Item | This engine | Both oracles | Coordinates |
203203
|------|-------------|--------------|-------------|
204-
| `round(1.2345, 3)` | `1.235` | `1.234` | `engine.rs:4960-4985` (scale/round path at 4974-4981) |
205-
| `round(123.456, -1)` | `120` | `123` (SQLite clamps `n` to `>= 0`) | `engine.rs:4960-4985` (scale/round path at 4974-4981) |
204+
| `round(1.2345, 3)` | `1.235` | `1.234` | `engine.rs:5540-5565` (scale/round path at 5554-5561) |
205+
| `round(123.456, -1)` | `120` | `123` (SQLite clamps `n` to `>= 0`) | `engine.rs:5540-5565` (scale/round path at 5554-5561) |
206+
| `round(2.675, 2)` | `2.68` | `2.67` | `engine.rs:5540-5565` (scale/round path at 5554-5561) |
206207

207-
The gate asserts only the ties (`round(±2.5)`) and the two-digit case, both of which match. The two rows above
208-
need a fix in `round_cell` before they can be asserted.
208+
The gate pins these three as known differences in `known differential drift pins round ties and negative
209+
precision`, asserting both this engine's current value and both oracles' value. The tie forms (`round(±2.5)`)
210+
and `round(1.2345, 2)` match both oracles and are asserted without any drift.
211+
212+
## Differential corpus
213+
214+
`npm run corpus -- --seed 20261106 --count 200` generates the corpus and runs every case on the same fixture
215+
through this engine and both live oracles. Seed and count fully determine the result; the digest for this run is
216+
`e3d6c1d1baf823086965286e4725e7921b79e6406ec29e72a96d03b08c1456b1`.
217+
218+
Distribution: DDL 40, DML 40, expressions 40, aggregates 30, windows 30, transactions 20. Result: 139 passed,
219+
51 mismatched, 10 discarded because `better-sqlite3` and CPython `sqlite3` disagreed with each other.
220+
221+
The only value-level differences are the `round` rows above. The other 45 mismatches are structured rejections
222+
against two successful oracles, all already listed in the cluster tables:
223+
224+
| Cluster | Cases | Shapes |
225+
|---------|-------|--------|
226+
| DDL | 16 | expression index 4, trigger 3, temp table 2, partial index 2, `WITHOUT ROWID` 2, `DROP COLUMN` 1, view 1, `STRICT` 1 |
227+
| DML | 10 | `INSERT OR REPLACE` 3, `INSERT ... SELECT` 3, UPSERT 2, `REPLACE INTO` 2 |
228+
| Expressions | 5 | `IS DISTINCT FROM` 2, modulo 1, `GLOB` 1, `NULLS FIRST` 1 |
229+
| Aggregates | 4 | `group_concat` with `ORDER BY` 1, `FILTER` 1, `count(DISTINCT)` 1, multi-argument `min`/`max` 1 |
230+
| Windows | 6 | `GROUPS` frame 3, `EXCLUDE` 3 |
231+
| Transactions | 4 | `SAVEPOINT` 4 |
232+
233+
The 10 discarded cases are all transaction statements. CPython's default isolation keeps an implicit transaction
234+
open after the fixture inserts, while `better-sqlite3` stays in autocommit, so `BEGIN` / `COMMIT` / `ROLLBACK`
235+
disagree between the two oracles and are never compared with this engine. `SAVEPOINT` and its release forms agree
236+
between the oracles, which is why the transaction mismatches above are `SAVEPOINT` rejections.
209237

210238
## How this was measured
211239

@@ -214,7 +242,7 @@ need a fix in `round_cell` before they can be asserted.
214242
2. Each case ran in a fresh in-memory engine; the same setup and statement ran in `better-sqlite3` and in
215243
CPython `sqlite3`; rows came back as JSON and were compared value by value. CPython keeps foreign keys
216244
disabled by default, so the oracle turns them on explicitly for every foreign key case.
217-
3. The eight gate cases added by this change use the live `better-sqlite3` oracle (`withOracle`) exactly like
218-
the existing window and CTE cases.
245+
3. The gate cases added by this change and the known-drift test use the live `better-sqlite3` oracle
246+
(`withOracle`) exactly like the existing window and CTE cases.
219247

220248
`readme.md` still describes the early `SELECT`-only subset; this file is the current surface description.

‎package.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,8 @@
1212
"test:py": "python -m unittest discover -s projects/sqlite.py/packages/sqlite/tests -t projects/sqlite.py/packages/sqlite/tests",
1313
"examples:ts": "node --experimental-strip-types projects/sqlite.ts/examples/roundtrip.ts",
1414
"examples:py": "python projects/sqlite.py/examples/roundtrip.py",
15-
"test:gate": "node scripts/check-test-floor.mjs"
15+
"test:gate": "node scripts/check-test-floor.mjs",
16+
"corpus": "node projects/sqlite.ts/packages/sqlite-harness/bin/cli.mjs corpus"
1617
},
1718
"packageManager": "pnpm@10.18.3",
1819
"engines": {

‎projects/sqlite.ts/packages/sqlite-harness/package.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,8 @@
1313
"bench": "node bin/cli.mjs bench",
1414
"conformance": "node bin/cli.mjs conformance",
1515
"fixtures": "node bin/cli.mjs fixtures",
16-
"oracle": "node bin/cli.mjs oracle"
16+
"oracle": "node bin/cli.mjs oracle",
17+
"corpus": "node bin/cli.mjs corpus"
1718
},
1819
"devDependencies": {
1920
"@yydb/sqlite": "workspace:*",

0 commit comments

Comments
 (0)