diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 6f3de15a97..a9642a908e 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -21,6 +21,8 @@ jobs: - name: Checkout uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: + # The run-budget report compares changed files with the merge-base. + fetch-depth: 0 persist-credentials: false - name: Setup Tools uses: tanstack/config/.github/setup@e4b48f16568324f76f467aa4c2aac2f05db632c3 @@ -28,6 +30,8 @@ jobs: run: pnpm run lint:check - name: Check oracle lint without inline directives run: pnpm run lint:oracles + - name: Report lowered oracle run budgets + run: node scripts/check-oracle-run-counts.mjs --format=github - name: Test oracle lint rules run: pnpm run test:lint-rules minified-db: diff --git a/AGENTS.md b/AGENTS.md index 458b57c2aa..49fee10c50 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -134,6 +134,11 @@ after each edit and at the end of each turn. A hook sees only the edits that the harness's own edit tools make. After an edit through a shell command, or in a harness without these hooks, run `pnpm lint:changed` before you finish. +The end-of-turn hook and `pnpm lint:changed` also report every oracle run +budget that the branch lowered, and CI prints it as a warning. A lowered +budget can be deliberate. Restore it, or explain the reason in the pull +request. + Fix a report. Do not turn the rule off. The hooks, `pnpm lint:changed`, the pre-commit hook, and CI ignore `eslint-disable` comments for these rules. A true exception needs a maintainer to change the rule in a reviewed pull diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 872abef797..7c220c1a0c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -57,7 +57,9 @@ If you have been assigned to fix an issue or develop a new feature, please follo your branch changes. It does not replace `pnpm lint`. If you work with a coding agent, the repository's hooks return oracle lint reports to it as it edits. Codex and Gemini CLI run project hooks only after you - trust them in `/hooks`; review them again when they change. + trust them in `/hooks`; review them again when they change. In a linked Git + worktree, Codex reads project hooks from the main checkout, not from the + worktree, so the hooks run there only after the main checkout has them. - Implement your changes and tests in the relevant package or example. - For a production change that adds, removes, or changes observable behavior, diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index a085dd75d9..6e930c7841 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -410,7 +410,7 @@ does not award oracle credit for a filename alone. | [#1833](https://github.com/TanStack/db/pull/1833) | Explicit oracle | `packages/db/tests/query/pagination-oracle.property.test.ts` owns inherited collection collation, actual `item2`/`item10` order, exact request options, hostile lexical/numeric controls, and both scan and auto-index paths. It runs in `@tanstack/db`'s `test:oracles` campaign. | | [#1834](https://github.com/TanStack/db/pull/1834) | Explicit oracle | `packages/db/tests/query/cold-join-reconciliation-oracle.test.ts` owns join/predicate equality equivalence across the established value domains, binary/string and nullish controls, replacement histories, raw lazy demand, and scan/auto-index paths. It runs in `@tanstack/db`'s `test:oracles` campaign. | | [#1835](https://github.com/TanStack/db/pull/1835) | Explicit oracle | The existing `packages/db/tests/collection-state-retention-oracle.property.test.ts` and `packages/db/tests/optimistic-transaction-oracle.property.test.ts` owners cover separate collection-state and transaction-history laws. Both were already registered in `@tanstack/db`'s `test:oracles` campaign; focused storage/local-only tests remain collateral. | -| [#1837](https://github.com/TanStack/db/pull/1837) | Explicit oracle | The offline scheduler, leadership replay, and serializer owners cover selective replay retirement, durable per-ID settlement, stale-read fencing, lifecycle recovery, native scalar encoding, and prior wire compatibility. They run in the package test campaign; generated owners expose `OFFLINE_ORACLE_{SEED,PATH,RUNS}` or the scheduler's `TANSTACK_DB_OFFLINE_ORACLE_*` replay interface. | +| [#1837](https://github.com/TanStack/db/pull/1837) | Explicit oracle | The offline scheduler, leadership replay, and serializer owners cover selective replay retirement, durable per-ID settlement, stale-read fencing, lifecycle recovery, native scalar encoding, and prior wire compatibility. They run in the package test campaign. Every generated campaign in these owners uses the shared `TANSTACK_DB_ORACLE_{SEED,PATH,PROPERTY}` replay interface, with one property per campaign (`offline.fifo-retry`, `offline.leadership.*`, `offline.serializer.*`, `offline.settlement.*`). | | [#1842](https://github.com/TanStack/db/pull/1842) | No shipped-law case | The PR changed only focused observer tests and introduced no production behavior. `packages/db/tests/live-query-observer.test.ts` remains the correct evidence; no synthetic oracle or campaign claim is added. | [PR #1816](https://github.com/TanStack/db/pull/1816) preserves existing witnesses, @@ -2296,11 +2296,12 @@ properties have separate environment variables; inspect their test headers. Do not assume the core multiplier reaches them. The Node expression-index oracle runs matching fixed-seed and seedless-random -campaigns by default. Supplying both replay values selects only the requested -seed and shrink path: +campaigns by default. A path replay of its property selects only the +requested seed and shrink path: ```sh -TANSTACK_DB_WS5A_SEED=1659005 TANSTACK_DB_WS5A_PATH=0 \ +TANSTACK_DB_ORACLE_SEED=1659005 TANSTACK_DB_ORACLE_PATH=0 \ +TANSTACK_DB_ORACLE_PROPERTY=sqlite-persistence.expression-index \ pnpm --filter @tanstack/node-db-sqlite-persistence test:oracles ``` diff --git a/docs/contributing/oracle-reviews/2026-09-30-guide-portfolio.md b/docs/contributing/oracle-reviews/2026-09-30-guide-portfolio.md index af65dccb7f..2146271cba 100644 --- a/docs/contributing/oracle-reviews/2026-09-30-guide-portfolio.md +++ b/docs/contributing/oracle-reviews/2026-09-30-guide-portfolio.md @@ -115,7 +115,7 @@ Revised means changed in executable revision 7e4266bb; retained means the assign | packages/db/tests/live-query-observer-history.property.test.ts | revised | | packages/db/tests/mutation-handler-type-oracle.test-d.ts | revised | | packages/db/tests/optimistic-transaction-oracle.property.test.ts | revised | -| packages/db/tests/oracle-replay.fixture.test.ts | retained | +| packages/db/tests/replay-guard-calibration.fixture.test.ts | retained | | packages/db/tests/oracle-replay.test.ts | revised | | packages/db/tests/paced-mutations-oracle.test.ts | retained | | packages/db/tests/query/cold-join-reconciliation-oracle.test.ts | revised | diff --git a/docs/contributing/oracle-rubric.md b/docs/contributing/oracle-rubric.md index 4b3fc0f6dc..b595083dbc 100644 --- a/docs/contributing/oracle-rubric.md +++ b/docs/contributing/oracle-rubric.md @@ -702,8 +702,12 @@ example or fixed campaign. [`fifo-retry-oracle.property.test.ts`](https://github.com/TanStack/db/blob/main/packages/offline-transactions/tests/fifo-retry-oracle.property.test.ts) shows this shape. Its fixed run preserves one scheduler campaign. Its second -campaign uses a random seed by default and accepts `OFFLINE_ORACLE_SEED` with -`OFFLINE_ORACLE_PATH` for replay. +campaign uses a random seed by default. To replay its recorded failure, set +`TANSTACK_DB_ORACLE_PROPERTY=offline.fifo-retry` with +`TANSTACK_DB_ORACLE_SEED` and `TANSTACK_DB_ORACLE_PATH`. The fixed run stands +aside, and the random campaign runs the recorded path once. The offline +owners' other properties use the same variables and name their own properties, +such as `offline.leadership.stale-read` and `offline.settlement.retry-decision`. Do not use a larger random run count as a substitute for structural reach. Pinned examples force rare boundaries. Bounded enumeration proves small finite diff --git a/package.json b/package.json index 85af542017..191e3f1ca3 100644 --- a/package.json +++ b/package.json @@ -21,6 +21,7 @@ "lint:check": "pnpm --filter \"./packages/**\" lint --fix=false", "lint:changed": "node scripts/agent-lint-feedback.mjs", "lint:oracles": "node scripts/agent-lint-feedback.mjs --all", + "lint:run-budgets": "node scripts/check-oracle-run-counts.mjs", "test:lint-rules": "node --test \"scripts/eslint-plugin-oracle/*.test.js\"", "lint-all": "eslint . --fix", "prepare": "husky", diff --git a/packages/browser-db-sqlite-persistence/tests/opfs-page-lifecycle-oracle.test.ts b/packages/browser-db-sqlite-persistence/tests/opfs-page-lifecycle-oracle.test.ts index c53447830b..73ee1dece2 100644 --- a/packages/browser-db-sqlite-persistence/tests/opfs-page-lifecycle-oracle.test.ts +++ b/packages/browser-db-sqlite-persistence/tests/opfs-page-lifecycle-oracle.test.ts @@ -1,5 +1,10 @@ import { fc, test as fcTest } from '@fast-check/vitest' import { afterEach, describe, expect, it, vi } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' import { openBrowserWASQLiteOPFSDatabase } from '../src/opfs-database' import type { BrowserOPFSWorkerErrorCode, @@ -932,66 +937,6 @@ afterEach(async () => { vi.useRealTimers() }) -const lifecycleSeed = Number( - process.env.TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_SEED ?? 1844, -) -const lifecycleRuns = Number( - process.env.TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_RUNS ?? 48, -) -const lifecyclePath = process.env.TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_PATH -const mixedSeed = Number(process.env.TANSTACK_DB_OPFS_MIXED_ORACLE_SEED ?? 1845) -const mixedRuns = Number(process.env.TANSTACK_DB_OPFS_MIXED_ORACLE_RUNS ?? 32) -const mixedPath = process.env.TANSTACK_DB_OPFS_MIXED_ORACLE_PATH -const failureSeed = Number( - process.env.TANSTACK_DB_OPFS_FAILURE_ORACLE_SEED ?? 1846, -) -const failureRuns = Number( - process.env.TANSTACK_DB_OPFS_FAILURE_ORACLE_RUNS ?? 18, -) -const failurePath = process.env.TANSTACK_DB_OPFS_FAILURE_ORACLE_PATH -const terminalEventSeed = Number( - process.env.TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_SEED ?? 1847, -) -const terminalEventRuns = Number( - process.env.TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_RUNS ?? 24, -) -const terminalEventPath = - process.env.TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_PATH -const closeSeed = Number(process.env.TANSTACK_DB_OPFS_CLOSE_ORACLE_SEED ?? 1848) -const closeRuns = Number(process.env.TANSTACK_DB_OPFS_CLOSE_ORACLE_RUNS ?? 32) -const closePath = process.env.TANSTACK_DB_OPFS_CLOSE_ORACLE_PATH -const replayRequested = [ - process.env.TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_SEED, - lifecyclePath, - process.env.TANSTACK_DB_OPFS_MIXED_ORACLE_SEED, - mixedPath, - process.env.TANSTACK_DB_OPFS_FAILURE_ORACLE_SEED, - failurePath, - process.env.TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_SEED, - terminalEventPath, - process.env.TANSTACK_DB_OPFS_CLOSE_ORACLE_SEED, - closePath, -].some((value) => value !== undefined) - -type OracleCampaign = { - label: `fixed` | `random` | `replay` - seed?: number - path?: string -} - -function oracleCampaigns( - seed: number, - replaySeedSupplied: boolean, - path: string | undefined, - replayRequestedForFile: boolean, -): Array { - if (replaySeedSupplied || path !== undefined) { - return [{ label: `replay`, seed, ...(path === undefined ? {} : { path }) }] - } - if (replayRequestedForFile) return [] - return [{ label: `fixed`, seed }, { label: `random` }] -} - const pagehideHistory = fc.record({ pending: fc.constantFrom(`init`, `execute`, `close`), delivery: fc.constantFrom( @@ -1189,10 +1134,12 @@ bfcache admission, durability, native handle release, or wa-sqlite #88. The Chromium Web Lock fixture checks native queued-lock release separately. Challenge/replay: fixed examples exhaust the cheap structural products. Each generated lane runs a fixed and an unseeded campaign with the same grammar and -run budget. Verbose FastCheck output retains original/reduced traces. Setting -SEED or PATH selects only the direct replay for that lane. Use its -TANSTACK_DB_OPFS_*_ORACLE_{SEED,RUNS,PATH} controls with this file's -package-local Vitest command. +run budget. Verbose FastCheck output retains original/reduced traces. A path +replay selects only the direct replay of its lane's property, named by +TANSTACK_DB_ORACLE_PROPERTY: opfs.page-lifecycle, opfs.mixed-execute, +opfs.terminal-event, opfs.close, or opfs.failure-cleanup. A seed without a +path reseeds the unseeded campaigns. Run it with this file's package-local +Vitest command. */ describe(`OPFS page lifecycle oracle`, () => { it.each([ @@ -1397,93 +1344,89 @@ describe(`OPFS page lifecycle oracle`, () => { expectDisposed(page, worker) }) - for (const campaign of oracleCampaigns( - lifecycleSeed, - process.env.TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_SEED !== undefined, - lifecyclePath, - replayRequested, - )) { - fcTest.prop([pagehideHistory], { - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - numRuns: lifecycleRuns, - ...(campaign.path === undefined ? {} : { path: campaign.path }), - examples: pagehideExamples, - verbose: true, - })( - `matches the independent model across generated histories (${campaign.label})`, - runPagehideHistory, - ) - } - - for (const campaign of oracleCampaigns( - mixedSeed, - process.env.TANSTACK_DB_OPFS_MIXED_ORACLE_SEED !== undefined, - mixedPath, - replayRequested, - )) { - fcTest.prop([mixedExecuteHistory], { - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - numRuns: mixedRuns, - ...(campaign.path === undefined ? {} : { path: campaign.path }), - examples: mixedExamples, - verbose: true, - })( - `preserves completed siblings while aborting pending siblings (${campaign.label})`, - runMixedExecuteHistory, - ) - } - - for (const campaign of oracleCampaigns( - terminalEventSeed, - process.env.TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_SEED !== undefined, - terminalEventPath, - replayRequested, - )) { - fcTest.prop([terminalEventHistory], { - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - numRuns: terminalEventRuns, - ...(campaign.path === undefined ? {} : { path: campaign.path }), - examples: terminalEventExamples, - verbose: true, - })( - `rejects pending work across Worker terminal event paths (${campaign.label})`, - runTerminalEventHistory, - ) - } - - for (const campaign of oracleCampaigns( - closeSeed, - process.env.TANSTACK_DB_OPFS_CLOSE_ORACLE_SEED !== undefined, - closePath, - replayRequested, - )) { - fcTest.prop([closeHistory], { - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - numRuns: closeRuns, - ...(campaign.path === undefined ? {} : { path: campaign.path }), - examples: closeExamples, - verbose: true, - })( - `settles pending work and refuses new work across explicit close (${campaign.label})`, - runCloseHistory, - ) - } - - for (const campaign of oracleCampaigns( - failureSeed, - process.env.TANSTACK_DB_OPFS_FAILURE_ORACLE_SEED !== undefined, - failurePath, - replayRequested, - )) { - fcTest.prop([failureHistory], { - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - numRuns: failureRuns, - ...(campaign.path === undefined ? {} : { path: campaign.path }), - examples: failureExamples, - verbose: true, - })( - `matches terminal cleanup and error mapping across worker failures (${campaign.label})`, - runFailureHistory, - ) - } + oracleReplayTest(fcTest).prop([pagehideHistory], { + seed: 1844, + numRuns: oracleRuns(48), + examples: pagehideExamples, + verbose: true, + })( + `matches the independent model across generated histories (fixed)`, + runPagehideHistory, + ) + oracleReplayTest(fcTest, `opfs.page-lifecycle`).prop([pagehideHistory], { + ...oraclePropertyOptions(48, `opfs.page-lifecycle`), + examples: pagehideExamples, + verbose: true, + })( + `matches the independent model across generated histories (random)`, + runPagehideHistory, + ) + oracleReplayTest(fcTest).prop([mixedExecuteHistory], { + seed: 1845, + numRuns: oracleRuns(32), + examples: mixedExamples, + verbose: true, + })( + `preserves completed siblings while aborting pending siblings (fixed)`, + runMixedExecuteHistory, + ) + oracleReplayTest(fcTest, `opfs.mixed-execute`).prop([mixedExecuteHistory], { + ...oraclePropertyOptions(32, `opfs.mixed-execute`), + examples: mixedExamples, + verbose: true, + })( + `preserves completed siblings while aborting pending siblings (random)`, + runMixedExecuteHistory, + ) + oracleReplayTest(fcTest).prop([terminalEventHistory], { + seed: 1847, + numRuns: oracleRuns(24), + examples: terminalEventExamples, + verbose: true, + })( + `rejects pending work across Worker terminal event paths (fixed)`, + runTerminalEventHistory, + ) + oracleReplayTest(fcTest, `opfs.terminal-event`).prop([terminalEventHistory], { + ...oraclePropertyOptions(24, `opfs.terminal-event`), + examples: terminalEventExamples, + verbose: true, + })( + `rejects pending work across Worker terminal event paths (random)`, + runTerminalEventHistory, + ) + oracleReplayTest(fcTest).prop([closeHistory], { + seed: 1848, + numRuns: oracleRuns(32), + examples: closeExamples, + verbose: true, + })( + `settles pending work and refuses new work across explicit close (fixed)`, + runCloseHistory, + ) + oracleReplayTest(fcTest, `opfs.close`).prop([closeHistory], { + ...oraclePropertyOptions(32, `opfs.close`), + examples: closeExamples, + verbose: true, + })( + `settles pending work and refuses new work across explicit close (random)`, + runCloseHistory, + ) + oracleReplayTest(fcTest).prop([failureHistory], { + seed: 1846, + numRuns: oracleRuns(18), + examples: failureExamples, + verbose: true, + })( + `matches terminal cleanup and error mapping across worker failures (fixed)`, + runFailureHistory, + ) + oracleReplayTest(fcTest, `opfs.failure-cleanup`).prop([failureHistory], { + ...oraclePropertyOptions(18, `opfs.failure-cleanup`), + examples: failureExamples, + verbose: true, + })( + `matches terminal cleanup and error mapping across worker failures (random)`, + runFailureHistory, + ) }) diff --git a/packages/browser-db-sqlite-persistence/tests/opfs-request-order-oracle.test.ts b/packages/browser-db-sqlite-persistence/tests/opfs-request-order-oracle.test.ts index 71d5ca9274..d4e7a9e314 100644 --- a/packages/browser-db-sqlite-persistence/tests/opfs-request-order-oracle.test.ts +++ b/packages/browser-db-sqlite-persistence/tests/opfs-request-order-oracle.test.ts @@ -13,6 +13,11 @@ import { readFileSync } from 'node:fs' import { createRequire } from 'node:module' import { fc, test as fcTest } from '@fast-check/vitest' import { afterAll, beforeAll, describe, expect, vi } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' import { openBrowserWASQLiteOPFSDatabase } from '../src/opfs-database' import { findRequestOrderViolation, @@ -261,36 +266,29 @@ const requestOrderHistory = fc }), ) -const requestOrderSeed = Number( - process.env.TANSTACK_DB_OPFS_REQUEST_ORDER_ORACLE_SEED ?? 1849, -) -const requestOrderRuns = Number( - process.env.TANSTACK_DB_OPFS_REQUEST_ORDER_ORACLE_RUNS ?? 24, -) -const requestOrderPath = process.env.TANSTACK_DB_OPFS_REQUEST_ORDER_ORACLE_PATH -const replayRequested = - process.env.TANSTACK_DB_OPFS_REQUEST_ORDER_ORACLE_SEED !== undefined || - requestOrderPath !== undefined +const requestOrderProperty = `opfs.request-order` describe(`OPFS request order oracle (Node driver)`, () => { - const campaigns = replayRequested - ? [{ label: `replay`, seed: requestOrderSeed, path: requestOrderPath }] - : [ - { label: `fixed`, seed: requestOrderSeed, path: undefined }, - { label: `random`, seed: undefined, path: undefined }, - ] - for (const campaign of campaigns) { - fcTest.prop([requestOrderHistory], { - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - numRuns: requestOrderRuns, - ...(campaign.path === undefined ? {} : { path: campaign.path }), - examples: requestOrderExamples.map((history): [RequestOrderHistory] => [ - history, - ]), - verbose: true, - })( - `answers requests in order and closes after them (${campaign.label})`, - runRequestOrderHistory, - ) - } + // A path replay of this property selects only its random campaign. + oracleReplayTest(fcTest).prop([requestOrderHistory], { + seed: 1849, + numRuns: oracleRuns(24), + examples: requestOrderExamples.map((history): [RequestOrderHistory] => [ + history, + ]), + verbose: true, + })( + `answers requests in order and closes after them (fixed)`, + runRequestOrderHistory, + ) + oracleReplayTest(fcTest, requestOrderProperty).prop([requestOrderHistory], { + ...oraclePropertyOptions(24, requestOrderProperty), + examples: requestOrderExamples.map((history): [RequestOrderHistory] => [ + history, + ]), + verbose: true, + })( + `answers requests in order and closes after them (random)`, + runRequestOrderHistory, + ) }) diff --git a/packages/browser-db-sqlite-persistence/tests/opfs-worker-diagnostics-oracle.test.ts b/packages/browser-db-sqlite-persistence/tests/opfs-worker-diagnostics-oracle.test.ts index 32b3844169..6e2d240b3d 100644 --- a/packages/browser-db-sqlite-persistence/tests/opfs-worker-diagnostics-oracle.test.ts +++ b/packages/browser-db-sqlite-persistence/tests/opfs-worker-diagnostics-oracle.test.ts @@ -3,8 +3,8 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { oraclePropertyOptions, oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../../db/tests/oracle-config' import type { BrowserOPFSWorkerRequest, @@ -150,30 +150,9 @@ const diagnosticInput = fc.record({ const diagnosticProperty = `opfs-worker.diagnostics` const diagnosticRuns = 40 -// A path replay runs only the selected property's recorded history, once. A -// seed without a path reseeds the random campaign and keeps the fixed one. -const replay = readOracleRunConfig() -const campaigns = - replay.replayPath === undefined - ? [ - { - name: `fixed`, - options: { seed: 1846, numRuns: oracleRuns(diagnosticRuns) }, - }, - { - name: `random`, - options: oraclePropertyOptions(diagnosticRuns, diagnosticProperty), - }, - ] - : replay.replayProperty === diagnosticProperty - ? [ - { - name: `replay`, - options: oraclePropertyOptions(diagnosticRuns, diagnosticProperty), - }, - ] - : [] - +// A path replay of this property runs only the random campaign, from its +// recorded history, once. A seed without a path reseeds the random campaign +// and keeps the fixed one. const diagnosticExamples: Array<[DiagnosticInput]> = [ [ { @@ -291,14 +270,22 @@ describe(`OPFS worker diagnostics oracle`, () => { }, ) - it.each(campaigns)( - `matches the independent diagnostic formatter ($name campaign)`, - async ({ options }) => { - await fc.assert(fc.asyncProperty(diagnosticInput, checkDiagnostic), { - ...options, - verbose: true, - }) - }, + const checkDiagnostics = (options: { numRuns: number; seed?: number }) => + fc.assert(fc.asyncProperty(diagnosticInput, checkDiagnostic), { + ...options, + verbose: true, + }) + oracleReplayTest(it)( + `matches the independent diagnostic formatter (fixed campaign)`, + () => checkDiagnostics({ seed: 1846, numRuns: oracleRuns(diagnosticRuns) }), + 15_000, + ) + oracleReplayTest(it, diagnosticProperty)( + `matches the independent diagnostic formatter (random campaign)`, + () => + checkDiagnostics( + oraclePropertyOptions(diagnosticRuns, diagnosticProperty), + ), 15_000, ) }) diff --git a/packages/browser-db-sqlite-persistence/tests/shared-driver-fairness-oracle.test.ts b/packages/browser-db-sqlite-persistence/tests/shared-driver-fairness-oracle.test.ts index 0043b955e6..dccdecafd4 100644 --- a/packages/browser-db-sqlite-persistence/tests/shared-driver-fairness-oracle.test.ts +++ b/packages/browser-db-sqlite-persistence/tests/shared-driver-fairness-oracle.test.ts @@ -25,6 +25,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import fc from 'fast-check' import { describe, expect, it } from 'vitest' +import { oracleRuns } from '../../db/tests/oracle-config' import { SHARED_DRIVER_FAIRNESS_BOUND, createPersistFirstFaultObservation, @@ -629,7 +630,7 @@ describe(`shared BrowserWASQLiteDriver fairness oracle`, () => { const executionsBefore = generatedPropertyExecutions const failuresBefore = generatedFailureRecords.length const result = await fc.check(generatedFairnessProperty, { - numRuns: GENERATED_RUNS, + numRuns: oracleRuns(GENERATED_RUNS), verbose: 2, ...(mode.seed === undefined ? {} : { seed: mode.seed }), ...(mode.path === undefined ? {} : { path: mode.path }), diff --git a/packages/db-ivm/tests/hash-failure-retry-oracle.property.test.ts b/packages/db-ivm/tests/hash-failure-retry-oracle.property.test.ts index 183e785fd6..08e2e61508 100644 --- a/packages/db-ivm/tests/hash-failure-retry-oracle.property.test.ts +++ b/packages/db-ivm/tests/hash-failure-retry-oracle.property.test.ts @@ -1,6 +1,11 @@ import { describe, expect, it } from 'vitest' import { fc } from '@fast-check/vitest' import { hash } from '../src/hashing/hash' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' /** * A failed structural traversal must publish no reusable hash state. @@ -42,26 +47,9 @@ const retryHistoryArbitrary = fc.record({ value: fc.integer({ min: -20, max: 20 }), }) -const replaySeedText = process.env.TANSTACK_DB_IVM_HASH_RETRY_SEED -const replayPath = process.env.TANSTACK_DB_IVM_HASH_RETRY_PATH -if (replayPath !== undefined && replaySeedText === undefined) { - throw new Error(`TANSTACK_DB_IVM_HASH_RETRY_PATH requires a seed`) -} -const replaySeed = - replaySeedText === undefined ? undefined : Number(replaySeedText) -if ( - replaySeedText !== undefined && - (replaySeedText.trim() === `` || !Number.isSafeInteger(replaySeed)) -) { - throw new Error(`TANSTACK_DB_IVM_HASH_RETRY_SEED must be an integer`) -} -const campaigns = - replaySeed === undefined - ? [ - { name: `fixed seed 205206`, seed: 205206, path: undefined }, - { name: `random seed`, seed: undefined, path: undefined }, - ] - : [{ name: `replay`, seed: replaySeed, path: replayPath }] +// The fixed campaign is stable. The random campaign is a fresh seed, or the +// recorded seed and path of this property when replayed. +const retryProperty = `ivm.hash-failure-retry` function expectFailureCut( error: unknown, @@ -129,20 +117,28 @@ describe(`getter failure does not publish partial structural hashes`, () => { expectRetryHistory({ completed: 3, tail: 0, failures: 3, value: 20 }) }) - for (const { name, seed, path } of campaigns) { - it(`preserves failure causes and retry cuts (${name})`, () => { + oracleReplayTest(it)( + `preserves failure causes and retry cuts (fixed seed 205206)`, + () => { fc.assert( fc.property(retryHistoryArbitrary, (history) => expectRetryHistory(history), ), - { - numRuns: 100, - ...(seed === undefined ? {} : { seed }), - ...(path === undefined ? {} : { path }), - }, + { numRuns: oracleRuns(100), seed: 205206 }, ) - }) - } + }, + ) + oracleReplayTest(it, retryProperty)( + `preserves failure causes and retry cuts (random seed)`, + () => { + fc.assert( + fc.property(retryHistoryArbitrary, (history) => + expectRetryHistory(history), + ), + oraclePropertyOptions(100, retryProperty), + ) + }, + ) it(`rejects cached-sibling and substituted-error observations`, () => { const error = new Error(`getter rejected`) diff --git a/packages/db-ivm/tests/hash-graph-oracle.property.test.ts b/packages/db-ivm/tests/hash-graph-oracle.property.test.ts index a6c23c45ee..da44da110a 100644 --- a/packages/db-ivm/tests/hash-graph-oracle.property.test.ts +++ b/packages/db-ivm/tests/hash-graph-oracle.property.test.ts @@ -1,6 +1,11 @@ import { describe, expect, it } from 'vitest' import { fc } from '@fast-check/vitest' import { hash } from '../src/hashing/hash' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' /** * Structural hash follows the reachable value graph, not object identity. @@ -20,28 +25,7 @@ import { hash } from '../src/hashing/hash' */ const fixedSeed = 1657019 -const replaySeedText = process.env.TANSTACK_DB_IVM_HASH_GRAPH_SEED -const replayPath = process.env.TANSTACK_DB_IVM_HASH_GRAPH_PATH -const replaySeed = - replaySeedText === undefined ? undefined : Number(replaySeedText) -if ( - replaySeedText !== undefined && - (replaySeedText.trim() === `` || !Number.isSafeInteger(replaySeed)) -) { - throw new Error(`TANSTACK_DB_IVM_HASH_GRAPH_SEED must be an integer`) -} -if (replayPath !== undefined && replaySeed === undefined) { - throw new Error( - `TANSTACK_DB_IVM_HASH_GRAPH_PATH requires TANSTACK_DB_IVM_HASH_GRAPH_SEED`, - ) -} -const campaigns = - replaySeed === undefined - ? [ - { name: `fixed`, seed: fixedSeed, path: undefined }, - { name: `random`, seed: undefined, path: undefined }, - ] - : [{ name: `replay`, seed: replaySeed, path: replayPath }] +const graphProperty = `ivm.hash-graph` // This adjacency list exists only in the model; production receives a rooted // JavaScript value. Kahn's algorithm checks reachability without using the @@ -175,20 +159,28 @@ describe(`structural hash graph boundary`, () => { expectGraphHash(edges) }) - for (const campaign of campaigns) { - it(`matches reachable graph cycles and shared DAGs (${campaign.name})`, () => { + oracleReplayTest(it)( + `matches reachable graph cycles and shared DAGs (fixed)`, + () => { fc.assert( fc.property(graphArbitrary, (edges) => { expectGraphHash(edges) }), - { - numRuns: 300, - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - ...(campaign.path === undefined ? {} : { path: campaign.path }), - }, + { numRuns: oracleRuns(300), seed: fixedSeed }, ) - }) - } + }, + ) + oracleReplayTest(it, graphProperty)( + `matches reachable graph cycles and shared DAGs (random)`, + () => { + fc.assert( + fc.property(graphArbitrary, (edges) => { + expectGraphHash(edges) + }), + oraclePropertyOptions(300, graphProperty), + ) + }, + ) it(`rejects cycle acceptance while accepting its acyclic near-neighbor`, () => { expect(() => expectGraphHash([[1], [0]], () => 0)).toThrow() diff --git a/packages/db-ivm/tests/hash-identity-oracle.property.test.ts b/packages/db-ivm/tests/hash-identity-oracle.property.test.ts index d64786f9f1..45300a1d8e 100644 --- a/packages/db-ivm/tests/hash-identity-oracle.property.test.ts +++ b/packages/db-ivm/tests/hash-identity-oracle.property.test.ts @@ -1,13 +1,18 @@ import { runInNewContext } from 'node:vm' import { Temporal } from 'temporal-polyfill' -import { describe, expect, it } from 'vitest' -import { fc } from '@fast-check/vitest' +import { describe, expect, it as vitestIt } from 'vitest' +import { fc, test as fcTest } from '@fast-check/vitest' import { MultiSet } from '../src/multiset.js' import { equalHashValues, hash, registerOpaqueHash, } from '../src/hashing/hash.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' import { captureHashSession } from './hash-session' /** @@ -786,56 +791,47 @@ function expectCyclicPair({ spec, changed, seeds }: CyclicPair): void { } // --------------------------------------------------------------------------- -// Campaigns. `TANSTACK_DB_IVM_HASH_IDENTITY_SEED` and -// `TANSTACK_DB_IVM_HASH_IDENTITY_PATH` select a direct replay. +// Campaigns. Each generated property has a fixed campaign and a random +// campaign over the same grammar and check. A path replay of one property +// runs its recorded history once and every other test stands aside. A seed +// alone reseeds the random campaigns. -const replaySeed = process.env.TANSTACK_DB_IVM_HASH_IDENTITY_SEED -const replayPath = process.env.TANSTACK_DB_IVM_HASH_IDENTITY_PATH const FIXED_SEED = 2026930 -const campaigns = - replaySeed === undefined && replayPath === undefined - ? [ - { name: String(FIXED_SEED), seed: FIXED_SEED as number | undefined }, - { name: `random`, seed: undefined }, - ] - : [ - { - name: `replay`, - seed: replaySeed === undefined ? undefined : Number(replaySeed), - }, - ] +const PAIR_IDENTITY = `ivm.hash-identity` +const CYCLIC_IDENTITY = `ivm.hash-identity-cycles` -function runOptions(seed: number | undefined) { - if (replayPath !== undefined && replaySeed === undefined) - throw new Error(`TANSTACK_DB_IVM_HASH_IDENTITY_PATH requires a seed`) - if ( - replaySeed !== undefined && - (replaySeed.trim() === `` || - typeof seed !== `number` || - !Number.isSafeInteger(seed)) - ) - throw new Error(`TANSTACK_DB_IVM_HASH_IDENTITY_SEED must be an integer`) - return { - numRuns: 300, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - } -} +const it = oracleReplayTest(vitestIt) describe(`hash identity oracle`, () => { - for (const { name, seed } of campaigns) { - it(`matches the identity model across generated value pairs (${name})`, () => { - fc.assert( - fc.property(pairArb, fc.nat(), fc.nat(), (pair, left, right) => - expectPair(pair, [left, right]), - ), - runOptions(seed), - ) - }) - it(`matches constructed verdicts across generated cyclic values (${name})`, () => { - fc.assert(fc.property(cyclicArb, expectCyclicPair), runOptions(seed)) + it(`matches the identity model across generated value pairs (${FIXED_SEED})`, () => { + fc.assert( + fc.property(pairArb, fc.nat(), fc.nat(), (pair, left, right) => + expectPair(pair, [left, right]), + ), + { numRuns: oracleRuns(300), seed: FIXED_SEED }, + ) + }) + oracleReplayTest(fcTest, PAIR_IDENTITY).prop( + [pairArb, fc.nat(), fc.nat()], + oraclePropertyOptions(300, PAIR_IDENTITY), + )( + `matches the identity model across generated value pairs (random)`, + (pair, left, right) => expectPair(pair, [left, right]), + ) + + it(`matches constructed verdicts across generated cyclic values (${FIXED_SEED})`, () => { + fc.assert(fc.property(cyclicArb, expectCyclicPair), { + numRuns: oracleRuns(300), + seed: FIXED_SEED, }) - } + }) + oracleReplayTest(fcTest, CYCLIC_IDENTITY).prop( + [cyclicArb], + oraclePropertyOptions(300, CYCLIC_IDENTITY), + )( + `matches constructed verdicts across generated cyclic values (random)`, + expectCyclicPair, + ) // Positive execution witness: the fixed campaign reaches every value kind // and every mutation, with both verdicts. diff --git a/packages/db-ivm/tests/hash-mixed-graph-oracle.property.test.ts b/packages/db-ivm/tests/hash-mixed-graph-oracle.property.test.ts index b95bca5a9b..8125e933e6 100644 --- a/packages/db-ivm/tests/hash-mixed-graph-oracle.property.test.ts +++ b/packages/db-ivm/tests/hash-mixed-graph-oracle.property.test.ts @@ -1,6 +1,12 @@ import { describe, expect, it } from 'vitest' import { fc } from '@fast-check/vitest' import { hash } from '../src/hashing/hash' +import { + oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' /** * Mixed-carrier graphs extend the established structural-hash graph law across @@ -71,28 +77,7 @@ const graphArbitrary = fc ), ) -const replaySeedText = process.env.TANSTACK_DB_IVM_MIXED_GRAPH_SEED -const replayPath = process.env.TANSTACK_DB_IVM_MIXED_GRAPH_PATH -const replayFault = process.env.TANSTACK_DB_IVM_MIXED_GRAPH_FAULT -if (replayPath !== undefined && replaySeedText === undefined) - throw new Error(`Mixed graph replay path requires a seed`) -if (replayFault !== undefined && replayFault !== `accept-cycles`) - throw new Error(`Unknown mixed graph replay fault`) -const replaySeed = - replaySeedText === undefined ? undefined : Number(replaySeedText) -if ( - replaySeedText !== undefined && - (replaySeedText.trim() === `` || !Number.isSafeInteger(replaySeed)) -) - throw new Error(`Mixed graph replay seed must be an integer`) - -const campaigns = - replaySeed === undefined - ? [ - { name: `fixed seed 205203`, seed: 205203, path: undefined }, - { name: `random`, seed: undefined, path: undefined }, - ] - : [{ name: `replay`, seed: replaySeed, path: replayPath }] +const mixedGraphPropertyId = `ivm.hash-mixed-graph` // Reachable indegrees are a topology authority, not the hasher's recursion // stack. Payloads and carrier allocation never enter this decision. @@ -264,23 +249,35 @@ describe(`mixed-carrier structural graph laws`, () => { expect(() => expectMixedGraph(diamond, rejectsAll)).toThrow() }) - for (const campaign of campaigns) { - it(`matches independent labels and mixed-carrier topology (${campaign.name})`, () => { - let checked = 0 - // The fault option is a test-only wrong-result control for CLI replay. - const hashValue = replayFault === `accept-cycles` ? () => 0 : hash - const property = mixedGraphProperty(hashValue, () => checked++) - fc.assert(property, { - numRuns: 150, - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - ...(campaign.path === undefined ? {} : { path: campaign.path }), - }) - // A green normal campaign must reach the graph check for every run. - if (campaign.path === undefined) expect(checked).toBe(150) - else expect(checked).toBeGreaterThan(0) - }) + // A calibration replay of this property runs its wrong-result control. + function expectMixedGraphCampaign( + parameters: ReturnType, + ): void { + let checked = 0 + const hashValue = + oracleReplayRequested() && + process.env.TANSTACK_DB_ORACLE_CALIBRATION === `accept-cycles` + ? () => 0 + : hash + const property = mixedGraphProperty(hashValue, () => checked++) + fc.assert(property, parameters) + // A green normal campaign must reach the graph check for every run. + if (parameters.path === undefined) expect(checked).toBe(parameters.numRuns) + else expect(checked).toBeGreaterThan(0) } + oracleReplayTest(it)( + `matches independent labels and mixed-carrier topology (fixed seed 205203)`, + () => expectMixedGraphCampaign({ numRuns: oracleRuns(150), seed: 205203 }), + ) + oracleReplayTest(it, mixedGraphPropertyId)( + `matches independent labels and mixed-carrier topology (random)`, + () => + expectMixedGraphCampaign( + oraclePropertyOptions(150, mixedGraphPropertyId), + ), + ) + it(`replays a generated cycle-acceptance mismatch at the same law`, () => { const acceptsCycles = (): number => 0 const failed = fc.check(mixedGraphProperty(acceptsCycles), { diff --git a/packages/db-ivm/tests/hash-oracle.property.test.ts b/packages/db-ivm/tests/hash-oracle.property.test.ts index bb6c0db903..a637ea9440 100644 --- a/packages/db-ivm/tests/hash-oracle.property.test.ts +++ b/packages/db-ivm/tests/hash-oracle.property.test.ts @@ -1,6 +1,11 @@ import { describe, expect, it } from 'vitest' import { fc } from '@fast-check/vitest' import { hash } from '../src/hashing/hash' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' import type { Arbitrary } from 'fast-check' /** @@ -78,47 +83,24 @@ function expectPermutedHashes( } // The reusable value laws run with identical generators and checks in a -// stable campaign and a fresh campaign. Select a single test name with Vitest -// when replaying, for example: -// TANSTACK_DB_IVM_HASH_SEED=123 TANSTACK_DB_IVM_HASH_PATH=0:1 \ +// fixed campaign and a random campaign. A path replay of `ivm.hash-value` +// selects only its random campaign. Replay it with Vitest, for example: +// TANSTACK_DB_ORACLE_SEED=123 TANSTACK_DB_ORACLE_PATH=0:1 \ +// TANSTACK_DB_ORACLE_PROPERTY=ivm.hash-value \ // pnpm exec vitest --run tests/hash-oracle.property.test.ts -t 'cloned arrays have same hash' -const valueReplaySeedText = process.env.TANSTACK_DB_IVM_HASH_SEED -const valueReplayPath = process.env.TANSTACK_DB_IVM_HASH_PATH -const valueReplaySeed = - valueReplaySeedText === undefined ? undefined : Number(valueReplaySeedText) -const valueCampaigns = - valueReplaySeedText === undefined && valueReplayPath === undefined - ? [ - { name: `fixed`, seed: 1657021 }, - { name: `random`, seed: undefined }, - ] - : [{ name: `replay`, seed: valueReplaySeed }] +const valueProperty = `ivm.hash-value` function hashProp]>(arbitraries: { [K in keyof Ts]: Arbitrary }) { return (name: string, check: (...values: Ts) => void): void => { const property = fc.property(...arbitraries, check) - for (const { name: campaign, seed } of valueCampaigns) { - it(`${name} (${campaign})`, () => { - if (valueReplayPath !== undefined && valueReplaySeedText === undefined) - throw new Error(`TANSTACK_DB_IVM_HASH_PATH requires a seed`) - if ( - valueReplaySeedText !== undefined && - (valueReplaySeedText.trim() === `` || - typeof seed !== `number` || - !Number.isSafeInteger(seed) || - seed < -2147483648 || - seed > 2147483647) - ) - throw new Error(`TANSTACK_DB_IVM_HASH_SEED must be a 32-bit integer`) - fc.assert(property, { - numRuns: 100, - ...(seed === undefined ? {} : { seed }), - ...(valueReplayPath === undefined ? {} : { path: valueReplayPath }), - }) - }) - } + oracleReplayTest(it)(`${name} (fixed)`, () => { + fc.assert(property, { numRuns: oracleRuns(100), seed: 1657021 }) + }) + oracleReplayTest(it, valueProperty)(`${name} (random)`, () => { + fc.assert(property, oraclePropertyOptions(100, valueProperty)) + }) } } diff --git a/packages/db-ivm/tests/incrementalization-law-oracle.property.test.ts b/packages/db-ivm/tests/incrementalization-law-oracle.property.test.ts index 6831dc4281..d503291b6a 100644 --- a/packages/db-ivm/tests/incrementalization-law-oracle.property.test.ts +++ b/packages/db-ivm/tests/incrementalization-law-oracle.property.test.ts @@ -15,6 +15,11 @@ import { import { StreamBuilder } from '../src/d2.js' import { DifferenceStreamWriter, LinearUnaryOperator } from '../src/graph.js' import { MultiSet } from '../src/multiset.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' import { assertBinaryIncrementalization, assertUnaryIncrementalization, @@ -178,47 +183,23 @@ type GroupedOutput = [ ] const FIXED_SEED = 1741 -type GeneratedCampaign = { - name: `fixed` | `random` | `replay` - seed: number | undefined - path: string | undefined -} - -const replaySeedText = process.env.TANSTACK_DB_IVM_ORACLE_SEED -const replaySeed = - replaySeedText === undefined ? undefined : Number(replaySeedText) -const RUNS = Number(process.env.TANSTACK_DB_IVM_ORACLE_RUNS ?? 100) -const REPLAY_PATH = process.env.TANSTACK_DB_IVM_ORACLE_PATH -if ( - replaySeedText !== undefined && - (replaySeedText.trim() === `` || !Number.isSafeInteger(replaySeed)) -) { - throw new Error(`TANSTACK_DB_IVM_ORACLE_SEED must be an integer`) -} -if (!Number.isSafeInteger(RUNS) || RUNS <= 0) { - throw new Error(`TANSTACK_DB_IVM_ORACLE_RUNS must be a positive integer`) -} -if (REPLAY_PATH !== undefined && replaySeed === undefined) { - throw new Error( - `TANSTACK_DB_IVM_ORACLE_PATH requires TANSTACK_DB_IVM_ORACLE_SEED`, - ) -} - -const generatedCampaigns: Array = - replaySeed === undefined - ? [ - { name: `fixed`, seed: FIXED_SEED, path: undefined }, - { name: `random`, seed: undefined, path: undefined }, - ] - : [{ name: `replay`, seed: replaySeed, path: REPLAY_PATH }] +const LAW_PROPERTY = `ivm.incrementalization-law` +const RUNS = oracleRuns(100) -function campaignParameters(campaign: (typeof generatedCampaigns)[number]) { - return { - numRuns: RUNS, - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - ...(campaign.path === undefined ? {} : { path: campaign.path }), - } -} +// The fixed campaign stands aside during a path replay. The random campaign is +// a fresh seed, or the recorded seed and path of this property when replayed. +const generatedCampaigns = [ + { + name: `fixed`, + test: oracleReplayTest(it), + parameters: { numRuns: RUNS, seed: FIXED_SEED }, + }, + { + name: `random`, + test: oracleReplayTest(it, LAW_PROPERTY), + parameters: oraclePropertyOptions(100, LAW_PROPERTY), + }, +] /** * Reach of one generated campaign. The checker delivers every logical batch @@ -285,8 +266,8 @@ const uniqueRowSplitDomain = { * index, which these laws drop before comparing. Publication timing and * SQL meaning are separate contracts. * - * Normal runs pair a stable campaign with a new random seed. Replay variables - * instead select only the recorded seed and shrink path. + * Normal runs pair a stable campaign with a new random seed. A replay of this + * property selects only its recorded seed and shrink path. */ const weightedWorld = weightedStateArbitrary( @@ -584,267 +565,270 @@ function firstTwoPerParity(input: Weighted): Weighted { } describe(`DBSP incrementalization laws`, () => { - it.each(generatedCampaigns)( - `checks consolidate, reduce, and groupBy against full recomputation ($name campaign)`, - async (campaign) => { - const reach = new CampaignReach() - await fc.assert( - fc.asyncProperty( - fc.array(weightedWorld, { minLength: 2, maxLength: 7 }), - async (worlds) => { - // Yield between complete histories so stress runs can report progress. - // Graph delivery and every observation within a history stay synchronous. - await setImmediate() - const batches = transitions(worlds) - reach.unary({ - name: `consolidate`, - initial: worlds[0]!, - batches, - inputPolicy: keyedPolicy, - outputPolicy: keyedPolicy, - build: (input) => input.pipe(consolidate()), - evaluate: keyedIdentity, - }) - reach.unary({ - name: `reduce`, - initial: worlds[0]!, - batches, - inputPolicy: keyedPolicy, - outputPolicy: keyedPolicy, - build: (input) => - input.pipe( - reduce((values) => { - if (values.length === 0) return [] - return [ - [ - values.reduce( - (sum, [value, weight]) => sum + value * weight, - 0, - ), - 1, - ], - ] - }), + for (const campaign of generatedCampaigns) + campaign.test( + `checks consolidate, reduce, and groupBy against full recomputation (${campaign.name} campaign)`, + async () => { + const reach = new CampaignReach() + await fc.assert( + fc.asyncProperty( + fc.array(weightedWorld, { minLength: 2, maxLength: 7 }), + async (worlds) => { + // Yield between complete histories so stress runs can report progress. + // Graph delivery and every observation within a history stay synchronous. + await setImmediate() + const batches = transitions(worlds) + reach.unary({ + name: `consolidate`, + initial: worlds[0]!, + batches, + inputPolicy: keyedPolicy, + outputPolicy: keyedPolicy, + build: (input) => input.pipe(consolidate()), + evaluate: keyedIdentity, + }) + reach.unary({ + name: `reduce`, + initial: worlds[0]!, + batches, + inputPolicy: keyedPolicy, + outputPolicy: keyedPolicy, + build: (input) => + input.pipe( + reduce((values) => { + if (values.length === 0) return [] + return [ + [ + values.reduce( + (sum, [value, weight]) => sum + value * weight, + 0, + ), + 1, + ], + ] + }), + ), + evaluate: summed, + }) + reach.unary({ + name: `groupBy reduction`, + initial: worlds[0]!, + batches, + inputPolicy: keyedPolicy, + outputPolicy: groupedPolicy, + build: (input) => + input.pipe( + groupBy(([key]) => ({ bucket: key % 2 }), { + total: groupByOperators.sum(([, value]) => value), + minimum: groupByOperators.min(([, value]) => value), + maximum: groupByOperators.max(([, value]) => value), + mean: groupByOperators.avg(([, value]) => value), + }), + ), + evaluate: groupedAggregates, + }) + reach.unary({ + name: `groupBy mean alone`, + initial: worlds[0]!, + batches, + inputPolicy: keyedPolicy, + outputPolicy: groupedMeanPolicy, + build: (input) => + input.pipe( + groupBy(([key]) => ({ bucket: key % 2 }), { + mean: groupByOperators.avg(([, value]) => value), + }), + ), + evaluate: groupedMean, + }) + reach.unary({ + name: `groupBy string extrema`, + initial: worlds[0]!, + batches, + inputPolicy: keyedPolicy, + outputPolicy: groupedTextPolicy, + build: (input) => + input.pipe( + groupBy(([key]) => ({ bucket: key % 2 }), { + lowFirst: groupByOperators.min( + ([, value]) => textPair(value)[0], + ), + highFirst: groupByOperators.max( + ([, value]) => textPair(value)[0], + ), + lowSecond: groupByOperators.min( + ([, value]) => textPair(value)[1], + ), + highSecond: groupByOperators.max( + ([, value]) => textPair(value)[1], + ), + }), + ), + evaluate: groupedTextExtrema, + }) + }, + ), + campaign.parameters, + ) + reach.expectBothDeliveries() + }, + ) + + for (const campaign of generatedCampaigns) + campaign.test( + `checks simultaneous binary join deltas and split delivery (${campaign.name} campaign)`, + async () => { + const reach = new CampaignReach() + await fc.assert( + fc.asyncProperty( + fc.array(fc.tuple(weightedWorld, weightedWorld), { + minLength: 2, + maxLength: 7, + }), + async (generated) => { + await setImmediate() + const worlds = generated.map(([left, right]) => ({ left, right })) + const batches = worlds.slice(1).map((next, index) => ({ + left: weightedDifference( + worlds[index]!.left, + next.left, + keyedPolicy, + CANCELLATION_ROW, ), - evaluate: summed, - }) - reach.unary({ - name: `groupBy reduction`, - initial: worlds[0]!, - batches, - inputPolicy: keyedPolicy, - outputPolicy: groupedPolicy, - build: (input) => - input.pipe( - groupBy(([key]) => ({ bucket: key % 2 }), { - total: groupByOperators.sum(([, value]) => value), - minimum: groupByOperators.min(([, value]) => value), - maximum: groupByOperators.max(([, value]) => value), - mean: groupByOperators.avg(([, value]) => value), - }), + right: weightedDifference( + worlds[index]!.right, + next.right, + keyedPolicy, + CANCELLATION_ROW, ), - evaluate: groupedAggregates, - }) + })) + reach.binary({ + name: `inner join`, + initialLeft: worlds[0]!.left, + initialRight: worlds[0]!.right, + batches, + leftPolicy: keyedPolicy, + rightPolicy: keyedPolicy, + outputPolicy: joinPolicy, + build: (left, right) => left.pipe(innerJoined(right)), + evaluate: joined, + }) + reach.binary({ + name: `full outer join`, + initialLeft: worlds[0]!.left, + initialRight: worlds[0]!.right, + batches, + leftPolicy: keyedPolicy, + rightPolicy: keyedPolicy, + outputPolicy: outerJoinPolicy, + build: (left, right) => left.pipe(join(right, `full`)), + evaluate: fullJoined, + }) + reach.binary({ + name: `left outer join`, + initialLeft: worlds[0]!.left, + initialRight: worlds[0]!.right, + batches, + leftPolicy: keyedPolicy, + rightPolicy: keyedPolicy, + outputPolicy: outerJoinPolicy, + build: (left, right) => left.pipe(join(right, `left`)), + evaluate: leftJoined, + }) + reach.binary({ + name: `right outer join`, + initialLeft: worlds[0]!.left, + initialRight: worlds[0]!.right, + batches, + leftPolicy: keyedPolicy, + rightPolicy: keyedPolicy, + outputPolicy: outerJoinPolicy, + build: (left, right) => left.pipe(join(right, `right`)), + evaluate: rightJoined, + }) + reach.binary({ + name: `join then grouped reduction`, + initialLeft: worlds[0]!.left, + initialRight: worlds[0]!.right, + batches, + leftPolicy: keyedPolicy, + rightPolicy: keyedPolicy, + outputPolicy: keyedPolicy, + build: (left, right) => + left.pipe( + innerJoined(right), + map(([key, [leftValue, rightValue]]) => [ + key, + leftValue * rightValue, + ]), + reduce((values) => { + if (values.length === 0) return [] + return [ + [ + values.reduce( + (sum, [value, weight]) => sum + value * weight, + 0, + ), + 1, + ], + ] + }), + ), + evaluate: joinedSums, + }) + }, + ), + campaign.parameters, + ) + reach.expectBothDeliveries() + }, + ) + + for (const campaign of generatedCampaigns) + campaign.test( + `checks global ordering and window membership (${campaign.name} campaign)`, + async () => { + const reach = new CampaignReach() + await fc.assert( + fc.asyncProperty(orderedHistory, async (worlds) => { + await setImmediate() reach.unary({ - name: `groupBy mean alone`, + name: `ordered window`, initial: worlds[0]!, - batches, + batches: transitions(worlds), inputPolicy: keyedPolicy, - outputPolicy: groupedMeanPolicy, + outputPolicy: keyedPolicy, + splitDomain: uniqueRowSplitDomain, build: (input) => input.pipe( - groupBy(([key]) => ({ bucket: key % 2 }), { - mean: groupByOperators.avg(([, value]) => value), - }), + orderByWithFractionalIndex((rank) => rank, { limit: 3 }), + withoutIndex(), ), - evaluate: groupedMean, + evaluate: firstThree, }) reach.unary({ - name: `groupBy string extrema`, + name: `grouped orderBy`, initial: worlds[0]!, - batches, + batches: transitions(worlds), inputPolicy: keyedPolicy, - outputPolicy: groupedTextPolicy, + outputPolicy: keyedPolicy, + splitDomain: uniqueRowSplitDomain, build: (input) => input.pipe( - groupBy(([key]) => ({ bucket: key % 2 }), { - lowFirst: groupByOperators.min( - ([, value]) => textPair(value)[0], - ), - highFirst: groupByOperators.max( - ([, value]) => textPair(value)[0], - ), - lowSecond: groupByOperators.min( - ([, value]) => textPair(value)[1], - ), - highSecond: groupByOperators.max( - ([, value]) => textPair(value)[1], - ), + groupedOrderByWithFractionalIndex((rank) => rank, { + groupKeyFn: (key) => key % 2, + limit: 2, }), + map(([key, [rank]]) => [key, rank] as Keyed), ), - evaluate: groupedTextExtrema, + evaluate: firstTwoPerParity, }) - }, - ), - campaignParameters(campaign), - ) - reach.expectBothDeliveries() - }, - ) - - it.each(generatedCampaigns)( - `checks simultaneous binary join deltas and split delivery ($name campaign)`, - async (campaign) => { - const reach = new CampaignReach() - await fc.assert( - fc.asyncProperty( - fc.array(fc.tuple(weightedWorld, weightedWorld), { - minLength: 2, - maxLength: 7, }), - async (generated) => { - await setImmediate() - const worlds = generated.map(([left, right]) => ({ left, right })) - const batches = worlds.slice(1).map((next, index) => ({ - left: weightedDifference( - worlds[index]!.left, - next.left, - keyedPolicy, - CANCELLATION_ROW, - ), - right: weightedDifference( - worlds[index]!.right, - next.right, - keyedPolicy, - CANCELLATION_ROW, - ), - })) - reach.binary({ - name: `inner join`, - initialLeft: worlds[0]!.left, - initialRight: worlds[0]!.right, - batches, - leftPolicy: keyedPolicy, - rightPolicy: keyedPolicy, - outputPolicy: joinPolicy, - build: (left, right) => left.pipe(innerJoined(right)), - evaluate: joined, - }) - reach.binary({ - name: `full outer join`, - initialLeft: worlds[0]!.left, - initialRight: worlds[0]!.right, - batches, - leftPolicy: keyedPolicy, - rightPolicy: keyedPolicy, - outputPolicy: outerJoinPolicy, - build: (left, right) => left.pipe(join(right, `full`)), - evaluate: fullJoined, - }) - reach.binary({ - name: `left outer join`, - initialLeft: worlds[0]!.left, - initialRight: worlds[0]!.right, - batches, - leftPolicy: keyedPolicy, - rightPolicy: keyedPolicy, - outputPolicy: outerJoinPolicy, - build: (left, right) => left.pipe(join(right, `left`)), - evaluate: leftJoined, - }) - reach.binary({ - name: `right outer join`, - initialLeft: worlds[0]!.left, - initialRight: worlds[0]!.right, - batches, - leftPolicy: keyedPolicy, - rightPolicy: keyedPolicy, - outputPolicy: outerJoinPolicy, - build: (left, right) => left.pipe(join(right, `right`)), - evaluate: rightJoined, - }) - reach.binary({ - name: `join then grouped reduction`, - initialLeft: worlds[0]!.left, - initialRight: worlds[0]!.right, - batches, - leftPolicy: keyedPolicy, - rightPolicy: keyedPolicy, - outputPolicy: keyedPolicy, - build: (left, right) => - left.pipe( - innerJoined(right), - map(([key, [leftValue, rightValue]]) => [ - key, - leftValue * rightValue, - ]), - reduce((values) => { - if (values.length === 0) return [] - return [ - [ - values.reduce( - (sum, [value, weight]) => sum + value * weight, - 0, - ), - 1, - ], - ] - }), - ), - evaluate: joinedSums, - }) - }, - ), - campaignParameters(campaign), - ) - reach.expectBothDeliveries() - }, - ) - - it.each(generatedCampaigns)( - `checks global ordering and window membership ($name campaign)`, - async (campaign) => { - const reach = new CampaignReach() - await fc.assert( - fc.asyncProperty(orderedHistory, async (worlds) => { - await setImmediate() - reach.unary({ - name: `ordered window`, - initial: worlds[0]!, - batches: transitions(worlds), - inputPolicy: keyedPolicy, - outputPolicy: keyedPolicy, - splitDomain: uniqueRowSplitDomain, - build: (input) => - input.pipe( - orderByWithFractionalIndex((rank) => rank, { limit: 3 }), - withoutIndex(), - ), - evaluate: firstThree, - }) - reach.unary({ - name: `grouped orderBy`, - initial: worlds[0]!, - batches: transitions(worlds), - inputPolicy: keyedPolicy, - outputPolicy: keyedPolicy, - splitDomain: uniqueRowSplitDomain, - build: (input) => - input.pipe( - groupedOrderByWithFractionalIndex((rank) => rank, { - groupKeyFn: (key) => key % 2, - limit: 2, - }), - map(([key, [rank]]) => [key, rank] as Keyed), - ), - evaluate: firstTwoPerParity, - }) - }), - campaignParameters(campaign), - ) - reach.expectBothDeliveries() - }, - ) + campaign.parameters, + ) + reach.expectBothDeliveries() + }, + ) // Reconstruction: the fixed ordered campaign reaches promotion. Sampling // its arbitrary with its seed and run count yields exactly the histories it diff --git a/packages/db-ivm/tests/index-refinement-oracle.property.test.ts b/packages/db-ivm/tests/index-refinement-oracle.property.test.ts index a78621207a..0901900f6f 100644 --- a/packages/db-ivm/tests/index-refinement-oracle.property.test.ts +++ b/packages/db-ivm/tests/index-refinement-oracle.property.test.ts @@ -54,6 +54,11 @@ import { fc } from '@fast-check/vitest' import { describe, expect, it } from 'vitest' import { Index } from '../src/indexes.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' type Prefix = number | string | bigint type Value = [Prefix, number | string] | number | string @@ -228,58 +233,45 @@ function expectRefinement( // --------------------------------------------------------------------------- // Campaigns. The fixed and random campaigns run the same property, grammar, -// check, and budget. `TANSTACK_DB_IVM_INDEX_SEED` and -// `TANSTACK_DB_IVM_INDEX_PATH` select a direct replay. +// check, and budget. A seed alone reseeds the random campaign. A path replay +// of INDEX_REFINEMENT_PROPERTY runs its recorded history, and every other +// test in this file stands aside. -const replaySeed = process.env.TANSTACK_DB_IVM_INDEX_SEED -const replayPath = process.env.TANSTACK_DB_IVM_INDEX_PATH -const campaigns = - replaySeed === undefined && replayPath === undefined - ? [ - { name: `20260930`, seed: 20260930 as number | undefined }, - { name: `random`, seed: undefined }, - ] - : [ - { - name: `replay`, - seed: replaySeed === undefined ? undefined : Number(replaySeed), - }, - ] +const INDEX_REFINEMENT_PROPERTY = `ivm.index-refinement` describe(`Index refinement oracle`, () => { - if (replaySeed === undefined && replayPath === undefined) { - for (const { name, additions } of pinnedHistories) { - it(`matches the multiset model when ${name}`, () => { - expectRefinement(additions) - expectRefinement(additions, `prefixIdentity`) - }) - } + for (const { name, additions } of pinnedHistories) { + oracleReplayTest(it)(`matches the multiset model when ${name}`, () => { + expectRefinement(additions) + expectRefinement(additions, `prefixIdentity`) + }) } - for (const { name, seed } of campaigns) { - it(`matches the multiset model across generated additions (${name})`, () => { - if (replayPath !== undefined && replaySeed === undefined) - throw new Error(`TANSTACK_DB_IVM_INDEX_PATH requires a seed`) - if ( - replaySeed !== undefined && - (replaySeed.trim() === `` || - typeof seed !== `number` || - !Number.isSafeInteger(seed)) + oracleReplayTest(it)( + `matches the multiset model across generated additions (20260930)`, + () => { + fc.assert( + fc.property(historyArbitrary, (additions) => { + expectRefinement(additions) + expectRefinement(additions, `prefixIdentity`) + }), + { numRuns: oracleRuns(300), seed: 20260930 }, ) - throw new Error(`TANSTACK_DB_IVM_INDEX_SEED must be an integer`) + }, + ) + + oracleReplayTest(it, INDEX_REFINEMENT_PROPERTY)( + `matches the multiset model across generated additions (random or replayed)`, + () => { fc.assert( fc.property(historyArbitrary, (additions) => { expectRefinement(additions) expectRefinement(additions, `prefixIdentity`) }), - { - numRuns: 300, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - }, + oraclePropertyOptions(300, INDEX_REFINEMENT_PROPERTY), ) - }) - } + }, + ) // Positive execution witness: generated histories reach a NaN prefix that // cancels after another value joined its key. diff --git a/packages/db-ivm/tests/multiset-consolidate-oracle.property.test.ts b/packages/db-ivm/tests/multiset-consolidate-oracle.property.test.ts index 38ca1783e3..4fd317e87d 100644 --- a/packages/db-ivm/tests/multiset-consolidate-oracle.property.test.ts +++ b/packages/db-ivm/tests/multiset-consolidate-oracle.property.test.ts @@ -1,6 +1,11 @@ -import { describe, expect, it } from 'vitest' -import { fc } from '@fast-check/vitest' +import { describe, expect, it as vitestIt } from 'vitest' +import { fc, test as fcTest } from '@fast-check/vitest' import { MultiSet } from '../src/multiset.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config.js' import type { MultiSetArray } from '../src/multiset.js' /** @@ -463,23 +468,14 @@ function expectConsolidation(history: History): void { // --------------------------------------------------------------------------- // Campaigns. The fixed and random campaigns run the same property, grammar, -// check, and budget. `TANSTACK_DB_IVM_CONSOLIDATE_SEED` and -// `TANSTACK_DB_IVM_CONSOLIDATE_PATH` select a direct replay. - -const replaySeed = process.env.TANSTACK_DB_IVM_CONSOLIDATE_SEED -const replayPath = process.env.TANSTACK_DB_IVM_CONSOLIDATE_PATH -const campaigns = - replaySeed === undefined && replayPath === undefined - ? [ - { name: `2026929`, seed: 2026929 as number | undefined }, - { name: `random`, seed: undefined }, - ] - : [ - { - name: `replay`, - seed: replaySeed === undefined ? undefined : Number(replaySeed), - }, - ] +// check, and budget. A path replay of `ivm.multiset-consolidate` runs its +// recorded history once, and every other test stands aside. A seed alone +// reseeds the random campaign. + +const FIXED_SEED = 2026929 +const CONSOLIDATE = `ivm.multiset-consolidate` + +const it = oracleReplayTest(vitestIt) describe(`MultiSet consolidation oracle`, () => { it(`detects replacement of an input value with equal contents`, () => { @@ -542,33 +538,25 @@ describe(`MultiSet consolidation oracle`, () => { ], ])(`%s`, (_name, records) => expectRecordsConsolidated(records)) - for (const { name, seed } of campaigns) { - it(`matches the identity model across generated multisets (${name})`, () => { - if (replayPath !== undefined && replaySeed === undefined) - throw new Error(`TANSTACK_DB_IVM_CONSOLIDATE_PATH requires a seed`) - if ( - replaySeed !== undefined && - (replaySeed.trim() === `` || - typeof seed !== `number` || - !Number.isSafeInteger(seed)) - ) - throw new Error(`TANSTACK_DB_IVM_CONSOLIDATE_SEED must be an integer`) - fc.assert( - fc.property(historyArb, (history) => expectConsolidation(history)), - { - numRuns: 300, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - }, - ) - }) - } + it(`matches the identity model across generated multisets (${FIXED_SEED})`, () => { + fc.assert( + fc.property(historyArb, (history) => expectConsolidation(history)), + { numRuns: oracleRuns(300), seed: FIXED_SEED }, + ) + }) + oracleReplayTest(fcTest, CONSOLIDATE).prop( + [historyArb], + oraclePropertyOptions(300, CONSOLIDATE), + )( + `matches the identity model across generated multisets (random)`, + expectConsolidation, + ) // Positive execution witness: the fixed campaign reaches every mode, and many // histories have fewer result identities than distinct input records. That // happens only when records merge or cancel. it(`reaches every mode and merges records in the fixed campaign`, () => { - const sample = fc.sample(historyArb, { seed: 2026929, numRuns: 300 }) + const sample = fc.sample(historyArb, { seed: FIXED_SEED, numRuns: 300 }) const modes = new Set(sample.map((history) => history.mode)) expect([...modes].sort()).toEqual( [ diff --git a/packages/db-ivm/tests/murmur-stream-oracle.property.test.ts b/packages/db-ivm/tests/murmur-stream-oracle.property.test.ts index e867686d46..7e736c28b5 100644 --- a/packages/db-ivm/tests/murmur-stream-oracle.property.test.ts +++ b/packages/db-ivm/tests/murmur-stream-oracle.property.test.ts @@ -1,5 +1,6 @@ import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest' import { fc } from '@fast-check/vitest' +import { oracleRuns } from '../../db/tests/oracle-config.js' /** * # Does the murmur stream hash the bytes it is given? @@ -86,7 +87,7 @@ describe(`MurmurHashStream`, () => { fc.property(fc.array(chunk, { minLength: 1, maxLength: 8 }), (chunks) => { expect(digests(Current, chunks)).toEqual(digests(Reference, chunks)) }), - { numRuns: 2000 }, + { numRuns: oracleRuns(2000) }, ) }) diff --git a/packages/db-ivm/tests/operators/reduce-fast-path-oracle.property.test.ts b/packages/db-ivm/tests/operators/reduce-fast-path-oracle.property.test.ts index caa78e055d..b75d7346fd 100644 --- a/packages/db-ivm/tests/operators/reduce-fast-path-oracle.property.test.ts +++ b/packages/db-ivm/tests/operators/reduce-fast-path-oracle.property.test.ts @@ -4,6 +4,7 @@ import { D2 } from '../../src/d2.js' import { MultiSet } from '../../src/multiset.js' import { reduce } from '../../src/operators/reduce.js' import { output } from '../../src/operators/output.js' +import { oracleRuns } from '../../../db/tests/oracle-config.js' /** * # Does the reduce fast path emit what the general path emits? @@ -137,7 +138,7 @@ describe(`reduce fast path`, () => { expect(emitted(runs, false)).toEqual(emitted(runs, true)) }, ), - { numRuns: 1000 }, + { numRuns: oracleRuns(1000) }, ) }) }) diff --git a/packages/db-ivm/tests/operators/topKWithFractionalIndex-oracle.test.ts b/packages/db-ivm/tests/operators/topKWithFractionalIndex-oracle.test.ts index 82f2e236e3..38f34828cb 100644 --- a/packages/db-ivm/tests/operators/topKWithFractionalIndex-oracle.test.ts +++ b/packages/db-ivm/tests/operators/topKWithFractionalIndex-oracle.test.ts @@ -11,6 +11,7 @@ import { assertOnlyKeysAffected, compareFractionalIndex, } from '../test-utils.js' +import { oracleRuns } from '../../../db/tests/oracle-config.js' import { TopKMessageTracker, TopKRelation } from './topk-relation-oracle.js' // Helper function to check if indices are in lexicographic order @@ -163,7 +164,7 @@ describe.each([ ), { seed, - numRuns: 30, + numRuns: oracleRuns(30), examples: [ [ false, @@ -243,7 +244,7 @@ describe.each([ ), { seed, - numRuns: 100, + numRuns: oracleRuns(100), examples: [ [ 1, @@ -361,7 +362,7 @@ describe.each([ ), { seed, - numRuns: 100, + numRuns: oracleRuns(100), examples: [ [ { diff --git a/packages/db-ivm/tests/operators/topk-support-window-oracle.test.ts b/packages/db-ivm/tests/operators/topk-support-window-oracle.test.ts index fcd801355f..9e2726bf79 100644 --- a/packages/db-ivm/tests/operators/topk-support-window-oracle.test.ts +++ b/packages/db-ivm/tests/operators/topk-support-window-oracle.test.ts @@ -5,6 +5,11 @@ import { MultiSet } from '../../src/multiset.js' import { output } from '../../src/operators/index.js' import { topKWithFractionalIndex } from '../../src/operators/topKWithFractionalIndex.js' import { groupedTopKWithFractionalIndex } from '../../src/operators/groupedTopKWithFractionalIndex.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../../db/tests/oracle-config.js' import { TopKRelation } from './topk-relation-oracle.js' /** @@ -33,37 +38,23 @@ type Key = (typeof keys)[number] type Row = { id: number; value: string } type Step = { slot: number; support: number; offset: number; limit: number } -// Normal campaigns share the same property, recorder, checks, and run budget. -// Replay with TANSTACK_DB_IVM_TOPK_SUPPORT_SEED and -// TANSTACK_DB_IVM_TOPK_SUPPORT_PATH on the focused Vitest command. -const replaySeedText = process.env.TANSTACK_DB_IVM_TOPK_SUPPORT_SEED -const replaySeed = - replaySeedText === undefined ? undefined : Number(replaySeedText) -const replayPath = process.env.TANSTACK_DB_IVM_TOPK_SUPPORT_PATH -const runs = Number(process.env.TANSTACK_DB_IVM_TOPK_SUPPORT_RUNS ?? 100) -if ( - replaySeedText !== undefined && - (replaySeedText.trim() === `` || !Number.isSafeInteger(replaySeed)) -) - throw new Error(`TANSTACK_DB_IVM_TOPK_SUPPORT_SEED must be an integer`) -if (!Number.isSafeInteger(runs) || runs <= 0) - throw new Error(`TANSTACK_DB_IVM_TOPK_SUPPORT_RUNS must be positive`) -if (replayPath !== undefined && replaySeed === undefined) - throw new Error( - `TANSTACK_DB_IVM_TOPK_SUPPORT_PATH requires TANSTACK_DB_IVM_TOPK_SUPPORT_SEED`, - ) -const campaigns: Array<{ - name: string - seed: number | undefined - path: string | undefined -}> = - replaySeed === undefined - ? [ - { name: `fixed`, seed: 409033, path: undefined }, - { name: `random`, seed: undefined, path: undefined }, - ] - : [{ name: `replay`, seed: replaySeed, path: replayPath }] - +// The fixed and random campaigns share the same property, recorder, checks, +// and run budget. A path replay of this property selects only its random +// campaign; a seed without a path reseeds it. +const supportWindowProperty = `ivm.topk-support-window` +const runs = oracleRuns(100) +const campaigns = [ + { + name: `fixed`, + test: oracleReplayTest(it), + parameters: { numRuns: runs, seed: 409033 }, + }, + { + name: `random`, + test: oracleReplayTest(it, supportWindowProperty), + parameters: oraclePropertyOptions(100, supportWindowProperty), + }, +] // Slot chooses typed identity and rank; support chooses the presence crossing; // offset chooses the first selected rank; limit chooses window width. Removing // any of these axes loses its corresponding distinction. The grammar rejects @@ -95,134 +86,133 @@ describe.each([ { name: `array`, operator: topKWithFractionalIndex }, { name: `grouped`, operator: grouped }, ])(`Signed support and window oracle: $name`, ({ operator }) => { - it.each(campaigns)( - `matches counted typed keys at every cut ($name campaign)`, - ({ seed, path }) => { - let firstFailure: - | { - observation: `rows` | `size` - checkpoint: `input` | `move` - step: number - steps: Array - message: string - } - | undefined - fc.assert( - fc.property( - fc.array( - fc.record({ - slot: fc.integer({ min: 0, max: keys.length - 1 }), - support: fc.integer({ min: 0, max: 3 }), - offset: fc.integer({ min: 0, max: 6 }), - limit: fc.integer({ min: 0, max: 6 }), - }), - { minLength: 1, maxLength: 35 }, - ), - (steps) => { - const graph = new D2() - const input = graph.newInput<[Key, Row]>() - const relation = new TopKRelation() - const counts = new Map() - let window = { offset: 0, limit: 3 } - let size!: () => number - let move!: (options: typeof window) => void - const rowFor = (key: Key): Row => ({ - id: 10 - keys.indexOf(key), - value: `tie`, - }) - input.pipe( - operator(() => 0, { - ...window, - setSizeCallback: (getSize) => { - size = getSize - }, - setWindowFn: (setWindow) => { - move = setWindow - }, + for (const campaign of campaigns) + campaign.test( + `matches counted typed keys at every cut (${campaign.name} campaign)`, + () => { + let firstFailure: + | { + observation: `rows` | `size` + checkpoint: `input` | `move` + step: number + steps: Array + message: string + } + | undefined + fc.assert( + fc.property( + fc.array( + fc.record({ + slot: fc.integer({ min: 0, max: keys.length - 1 }), + support: fc.integer({ min: 0, max: 3 }), + offset: fc.integer({ min: 0, max: 6 }), + limit: fc.integer({ min: 0, max: 6 }), }), - output((message) => relation.add(message.getInner())), - ) - graph.finalize() - expect(size).toBeTypeOf(`function`) - expect(move).toBeTypeOf(`function`) - const check = ( - checkpoint: `input` | `move`, - step: number, - ): boolean => { - const selected = orderedKeys - .filter((key) => (counts.get(key) ?? 0) > 0) - .slice(window.offset, window.offset + window.limit) - const assertAt = ( - observation: `rows` | `size`, - assertion: () => void, + { minLength: 1, maxLength: 35 }, + ), + (steps) => { + const graph = new D2() + const input = graph.newInput<[Key, Row]>() + const relation = new TopKRelation() + const counts = new Map() + let window = { offset: 0, limit: 3 } + let size!: () => number + let move!: (options: typeof window) => void + const rowFor = (key: Key): Row => ({ + id: 10 - keys.indexOf(key), + value: `tie`, + }) + input.pipe( + operator(() => 0, { + ...window, + setSizeCallback: (getSize) => { + size = getSize + }, + setWindowFn: (setWindow) => { + move = setWindow + }, + }), + output((message) => relation.add(message.getInner())), + ) + graph.finalize() + expect(size).toBeTypeOf(`function`) + expect(move).toBeTypeOf(`function`) + const check = ( + checkpoint: `input` | `move`, + step: number, ): boolean => { - try { - assertion() - return true - } catch (error) { - firstFailure ??= { - observation, - checkpoint, - step, - steps: [...steps], - message: - error instanceof Error ? error.message : String(error), - } - // A smaller history is a replay only if it violates the - // same observation at the same graph-run checkpoint. - if ( - firstFailure.observation === observation && - firstFailure.checkpoint === checkpoint - ) { - if (error instanceof Error) - error.message += `\nOriginal ${firstFailure.observation} mismatch after ${firstFailure.checkpoint} run at step ${firstFailure.step}: ${firstFailure.message}; generated history: ${JSON.stringify(firstFailure.steps)}` - throw error + const selected = orderedKeys + .filter((key) => (counts.get(key) ?? 0) > 0) + .slice(window.offset, window.offset + window.limit) + const assertAt = ( + observation: `rows` | `size`, + assertion: () => void, + ): boolean => { + try { + assertion() + return true + } catch (error) { + firstFailure ??= { + observation, + checkpoint, + step, + steps: [...steps], + message: + error instanceof Error ? error.message : String(error), + } + // A smaller history is a replay only if it violates the + // same observation at the same graph-run checkpoint. + if ( + firstFailure.observation === observation && + firstFailure.checkpoint === checkpoint + ) { + if (error instanceof Error) + error.message += `\nOriginal ${firstFailure.observation} mismatch after ${firstFailure.checkpoint} run at step ${firstFailure.step}: ${firstFailure.message}; generated history: ${JSON.stringify(firstFailure.steps)}` + throw error + } + return false } - return false } - } - if ( - !assertAt(`rows`, () => - relation.expectRows( - selected.map((key) => [key, rowFor(key).id, `tie`]), - ), + if ( + !assertAt(`rows`, () => + relation.expectRows( + selected.map((key) => [key, rowFor(key).id, `tie`]), + ), + ) ) - ) - return false - return assertAt(`size`, () => - expect(size()).toBe(selected.length), - ) - } - // Pin positivity crossings before every generated history. - const history = [1, 2, 1, 0, 2, 1, 0].map((support) => ({ - slot: 0, - support, - offset: 0, - limit: 3, - })) - for (const [index, action] of [...history, ...steps].entries()) { - const key = keys[action.slot]! - const weight = action.support - (counts.get(key) ?? 0) - counts.set(key, action.support) - input.sendData(new MultiSet([[[key, rowFor(key)], weight]])) - graph.run() - if (!check(`input`, index)) return - window = { offset: action.offset, limit: action.limit } - move(window) - graph.run() - if (!check(`move`, index)) return - } + return false + return assertAt(`size`, () => + expect(size()).toBe(selected.length), + ) + } + // Pin positivity crossings before every generated history. + const history = [1, 2, 1, 0, 2, 1, 0].map((support) => ({ + slot: 0, + support, + offset: 0, + limit: 3, + })) + for (const [index, action] of [...history, ...steps].entries()) { + const key = keys[action.slot]! + const weight = action.support - (counts.get(key) ?? 0) + counts.set(key, action.support) + input.sendData(new MultiSet([[[key, rowFor(key)], weight]])) + graph.run() + if (!check(`input`, index)) return + window = { offset: action.offset, limit: action.limit } + move(window) + graph.run() + if (!check(`move`, index)) return + } + }, + ), + { + ...campaign.parameters, + examples: [[boundaryHistory]], }, - ), - { - numRuns: runs, - ...(seed === undefined ? {} : { seed }), - ...(path === undefined ? {} : { path }), - examples: [[boundaryHistory]], - }, - ) - }, - ) + ) + }, + ) }) it.each([-2, 2])( diff --git a/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts index 647a9520e4..225e6cba9a 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts @@ -17,8 +17,9 @@ import { } from '@tanstack/db' import { oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../../db/tests/oracle-config.js' import { IndeterminateCommitError, @@ -210,19 +211,17 @@ function expectExactSnapshot( for (const [key, row] of expected) expect(actual.get(key)).toStrictEqual(row) } -const requestedOracleReplayProperty = readOracleRunConfig().replayProperty -const describeUnlessOracleReplay = - requestedOracleReplayProperty === undefined || - requestedOracleReplayProperty === `persistence.retained-demand` || - requestedOracleReplayProperty === `persistence.generation-notice` || - requestedOracleReplayProperty === `persistence.follower-remote-demand` - ? describe - : describe.skip +// A path replay of one of these properties runs this suite. A replay of any +// other property stands it aside. +const describeUnlessOracleReplay = oracleReplayTest( + describe, + `persistence.retained-demand`, + `persistence.generation-notice`, + `persistence.follower-remote-demand`, +) -const itUnlessAnyOracleReplay = - requestedOracleReplayProperty === undefined ? it : it.skip -const describeUnlessAnyOracleReplay = - requestedOracleReplayProperty === undefined ? describe : describe.skip +const itUnlessAnyOracleReplay = oracleReplayTest(it) +const describeUnlessAnyOracleReplay = oracleReplayTest(describe) const hydrationScopeModes = [ `none`, @@ -3975,7 +3974,7 @@ function generatedPersistenceGrammarSamples(): ReadonlyArray<{ if (match === undefined) throw new Error(`missing grammar: ${property}`) return match } - const sampleOptions = (seed: number) => ({ seed, numRuns: 4 }) + const sampleOptions = (seed: number) => ({ seed, numRuns: oracleRuns(4) }) return [ { grammar: grammar(`sqlite-persistence.source-fifo-order`), @@ -4102,31 +4101,27 @@ function registerGeneratedPersistenceProperty(options: { run: (value: Value) => Promise }): void { const { property, title, arbitrary, fixedSeed, run } = options - if (requestedOracleReplayProperty === undefined) { - fcTest.prop([arbitrary], { seed: fixedSeed, numRuns: oracleRuns(4) })( - `${title} (fixed)`, - run, - ) - fcTest.prop([arbitrary], oraclePropertyOptions(4, property))( - `${title} (random)`, - run, - ) - } else if (requestedOracleReplayProperty === property) { - fcTest.prop([arbitrary], oraclePropertyOptions(4, property))( - `${title} (replay)`, - run, - ) - } + oracleReplayTest(fcTest).prop([arbitrary], { + seed: fixedSeed, + numRuns: oracleRuns(4), + })(`${title} (fixed)`, run) + oracleReplayTest(fcTest, property).prop( + [arbitrary], + oraclePropertyOptions(4, property), + )(`${title} (random)`, run) } -const describeDurabilityOracles = - requestedOracleReplayProperty === undefined || - requestedOracleReplayProperty.startsWith(`sqlite-persistence.`) - ? describe - : describe.skip +const describeDurabilityOracles = oracleReplayTest( + describe, + `sqlite-persistence.source-fifo-order`, + `sqlite-persistence.abort-graph`, + `sqlite-persistence.owner-isolation`, + `sqlite-persistence.open-transaction-boundary`, + `sqlite-persistence.request-local-failure-buffered-source`, +) describeDurabilityOracles(`generated persistence durability oracles`, () => { - if (requestedOracleReplayProperty === undefined) { + if (!oracleReplayRequested()) { it(`reconstructs the full grammar and rejects ablated, out-of-range, and foreign histories`, () => { for (const { grammar, diff --git a/packages/db-sqlite-persistence-core/tests/sqlite-boolean-arity-oracle.test.ts b/packages/db-sqlite-persistence-core/tests/sqlite-boolean-arity-oracle.test.ts index ce469d7897..d6f4f480f8 100644 --- a/packages/db-sqlite-persistence-core/tests/sqlite-boolean-arity-oracle.test.ts +++ b/packages/db-sqlite-persistence-core/tests/sqlite-boolean-arity-oracle.test.ts @@ -3,6 +3,13 @@ import { DatabaseSync } from 'node:sqlite' import { fc } from '@fast-check/vitest' import { expect, it, vi } from 'vitest' import { IR } from '@tanstack/db' +import { + oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, + oracleRuns, + recordOracleReplay, +} from '../../db/tests/oracle-config' import { SQLiteCorePersistenceAdapter, createPersistedTableName } from '../src' import type { SQLiteDriver } from '../src' @@ -450,359 +457,364 @@ function brief(failure: Failure | undefined) { } } -it('refines boolean arity and SQL row work across generated SQLite loads', async () => { - const driver = new CountingDriver() - const adapter = new SQLiteCorePersistenceAdapter({ driver }) +// A path replay of this property runs its random campaign only. A seed without +// a path reseeds that campaign. +const booleanArityProperty = `sqlite-persistence.boolean-arity` - async function check(model: Model): Promise> { - driver.reads = [] - const result = await adapter.loadSubset( - { kind: `eager`, collectionId: 'boolean-arity' }, - { - where: toIR(model), - }, - ) - const actual = result.map((row) => String(row.key)).sort() - const expected = expectedKeys(model) - const read = driver.reads[0] - if (driver.reads.length !== 1 || !read) { - throw new Error( - `ordinary load must make one Collection SELECT; got ${driver.reads.length}`, +oracleReplayTest(it, booleanArityProperty)( + 'refines boolean arity and SQL row work across generated SQLite loads', + async () => { + const driver = new CountingDriver() + const adapter = new SQLiteCorePersistenceAdapter({ driver }) + + async function check(model: Model): Promise> { + driver.reads = [] + const result = await adapter.loadSubset( + { kind: `eager`, collectionId: 'boolean-arity' }, + { + where: toIR(model), + }, ) + const actual = result.map((row) => String(row.key)).sort() + const expected = expectedKeys(model) + const read = driver.reads[0] + if (driver.reads.length !== 1 || !read) { + throw new Error( + `ordinary load must make one Collection SELECT; got ${driver.reads.length}`, + ) + } + const fail = (law: Failure['law']): Failure => ({ + law, + checkpoint: 'loadSubset-return', + model, + expected, + actual, + read, + }) + const failures: Array = [] + if (JSON.stringify(actual) !== JSON.stringify(expected)) + failures.push(fail('public')) + if ( + canPush(model) && + expected.length < fixture.length && + (!read.sql.includes(' WHERE ') || read.rawRows !== expected.length) + ) { + failures.push(fail('work')) + } + if ( + model.kind === 'and' && + model.children.length === 0 && + (!read.sql.includes(' WHERE ') || read.parameters !== 0) + ) { + failures.push(fail('compile')) + } + if (read.rawRows < expected.length || read.rawRows > fixture.length) + failures.push(fail('candidate')) + return failures } - const fail = (law: Failure['law']): Failure => ({ - law, - checkpoint: 'loadSubset-return', - model, - expected, - actual, - read, - }) - const failures: Array = [] - if (JSON.stringify(actual) !== JSON.stringify(expected)) - failures.push(fail('public')) - if ( - canPush(model) && - expected.length < fixture.length && - (!read.sql.includes(' WHERE ') || read.rawRows !== expected.length) - ) { - failures.push(fail('work')) - } - if ( - model.kind === 'and' && - model.children.length === 0 && - (!read.sql.includes(' WHERE ') || read.parameters !== 0) - ) { - failures.push(fail('compile')) - } - if (read.rawRows < expected.length || read.rawRows > fixture.length) - failures.push(fail('candidate')) - return failures - } - const runs = Number(process.env.TANSTACK_DB_SQLITE_BOOLEAN_ORACLE_RUNS ?? 150) - const replaySeed = process.env.TANSTACK_DB_SQLITE_BOOLEAN_ORACLE_SEED - const replayPath = process.env.TANSTACK_DB_SQLITE_BOOLEAN_ORACLE_PATH - if (!Number.isSafeInteger(runs) || runs < 1) - throw new Error('oracle runs must be positive') - if (replayPath !== undefined && !/^\d+(?::\d+)*$/.test(replayPath)) { - throw new Error('oracle replay path must be numeric') - } - if (replaySeed !== undefined && !Number.isSafeInteger(Number(replaySeed))) { - throw new Error('oracle replay seed must be an integer') - } - - async function campaign(seed?: number, path?: string) { - let original: Failure | undefined - const property = fc.asyncProperty(modelArb, async (model) => { - const failures = await check(model) - if (failures.length === 0) return - const originalLaw = original?.law - const sameLaw = originalLaw - ? failures.find((failure) => failure.law === originalLaw) - : failures[0] - if (!sameLaw) return - original ??= sameLaw - throw new Error(JSON.stringify(brief(sameLaw))) - }) - const result = await fc.check(property, { seed, path, numRuns: runs }) - const reducedFailures = result.counterexample - ? await check(result.counterexample[0]) - : [] - const reduced = reducedFailures.find( - (failure) => failure.law === original?.law, - ) - if (result.failed && original && !reduced) { - throw new Error(`shrinking lost the ${original.law} law`) - } - return { - failed: result.failed, - seed: result.seed, - path: result.counterexamplePath, - counterexample: result.counterexample, - original: brief(original), - reduced: brief(reduced), + async function campaign(parameters: { + numRuns: number + seed?: number + path?: string + }) { + let original: Failure | undefined + const property = fc.asyncProperty(modelArb, async (model) => { + const failures = await check(model) + if (failures.length === 0) return + const originalLaw = original?.law + const sameLaw = originalLaw + ? failures.find((failure) => failure.law === originalLaw) + : failures[0] + if (!sameLaw) return + original ??= sameLaw + throw new Error(JSON.stringify(brief(sameLaw))) + }) + const result = await fc.check(property, parameters) + recordOracleReplay(booleanArityProperty, result) + const reducedFailures = result.counterexample + ? await check(result.counterexample[0]) + : [] + const reduced = reducedFailures.find( + (failure) => failure.law === original?.law, + ) + if (result.failed && original && !reduced) { + throw new Error(`shrinking lost the ${original.law} law`) + } + return { + failed: result.failed, + seed: result.seed, + path: result.counterexamplePath, + counterexample: result.counterexample, + original: brief(original), + reduced: brief(reduced), + } } - } - async function runOracle(): Promise { - await adapter.applyCommittedTx( - { kind: `eager`, collectionId: 'boolean-arity' }, - { - txId: 'seed', - term: 1, - seq: 1, - rowVersion: 1, - mutations: fixture.map((row) => ({ - type: 'insert' as const, - key: row.id, - value: structuredClone(row), - })), - }, - ) - if (replaySeed !== undefined) { - const replay = await campaign(Number(replaySeed), replayPath) - if (replay.failed) - throw new Error( - `direct boolean-arity replay RED: ${JSON.stringify(replay)}`, + async function runOracle(): Promise { + await adapter.applyCommittedTx( + { kind: `eager`, collectionId: 'boolean-arity' }, + { + txId: 'seed', + term: 1, + seq: 1, + rowVersion: 1, + mutations: fixture.map((row) => ({ + type: 'insert' as const, + key: row.id, + value: structuredClone(row), + })), + }, + ) + // A path replay of this property runs only its random campaign. + if (oracleReplayRequested()) { + const replay = await campaign( + oraclePropertyOptions(150, booleanArityProperty), ) - return - } - - driver.reads = [] - const baseline = await adapter.loadSubset( - { kind: `eager`, collectionId: 'boolean-arity' }, - {}, - ) - expect(baseline).toHaveLength(fixture.length) - expect(driver.reads).toHaveLength(1) - expect(driver.reads[0]?.rawRows).toBe(fixture.length) - expect(driver.reads[0]?.sql).not.toContain(' WHERE ') + if (replay.failed) + throw new Error( + `direct boolean-arity replay RED: ${JSON.stringify(replay)}`, + ) + return + } - const failures: Array = [] - for (const model of matrix()) failures.push(...(await check(model))) - const fixed = await campaign(1994) - const random = await campaign() + driver.reads = [] + const baseline = await adapter.loadSubset( + { kind: `eager`, collectionId: 'boolean-arity' }, + {}, + ) + expect(baseline).toHaveLength(fixture.length) + expect(driver.reads).toHaveLength(1) + expect(driver.reads[0]?.rawRows).toBe(fixture.length) + expect(driver.reads[0]?.sql).not.toContain(' WHERE ') + + const failures: Array = [] + for (const model of matrix()) failures.push(...(await check(model))) + const fixed = await campaign({ numRuns: oracleRuns(150), seed: 1994 }) + const random = await campaign( + oraclePropertyOptions(150, booleanArityProperty), + ) - driver.reads = [] - const emptyIn = await adapter.loadSubset( - { kind: `eager`, collectionId: 'boolean-arity' }, - { - where: toIR(atom('in-empty')), - }, - ) - expect(emptyIn).toHaveLength(0) - expect(driver.reads).toHaveLength(1) - expect(driver.reads[0]?.rawRows).toBe(0) - expect(driver.reads[0]?.sql).toContain(' WHERE ') + driver.reads = [] + const emptyIn = await adapter.loadSubset( + { kind: `eager`, collectionId: 'boolean-arity' }, + { + where: toIR(atom('in-empty')), + }, + ) + expect(emptyIn).toHaveLength(0) + expect(driver.reads).toHaveLength(1) + expect(driver.reads[0]?.rawRows).toBe(0) + expect(driver.reads[0]?.sql).toContain(' WHERE ') - // The cursor receiver forms two new AND nodes around the base predicate. - // A root-only arity repair would leave this nested empty-OR work unbounded. - driver.reads = [] - const cursorRows = await adapter.loadSubset( - { kind: `eager`, collectionId: 'boolean-arity' }, - { - where: toIR(op('or')), - orderBy: [ - { - expression: new IR.PropRef(['id']), - compareOptions: { direction: 'asc', nulls: 'last' }, + // The cursor receiver forms two new AND nodes around the base predicate. + // A root-only arity repair would leave this nested empty-OR work unbounded. + driver.reads = [] + const cursorRows = await adapter.loadSubset( + { kind: `eager`, collectionId: 'boolean-arity' }, + { + where: toIR(op('or')), + orderBy: [ + { + expression: new IR.PropRef(['id']), + compareOptions: { direction: 'asc', nulls: 'last' }, + }, + ], + limit: 2, + cursor: { + whereCurrent: toIR(atom('first')), + whereFrom: new IR.Func('gt', [ + new IR.PropRef(['id']), + new IR.Value('row-1'), + ]), }, - ], - limit: 2, - cursor: { - whereCurrent: toIR(atom('first')), - whereFrom: new IR.Func('gt', [ - new IR.PropRef(['id']), - new IR.Value('row-1'), - ]), }, - }, - ) - const cursorReads = driver.reads.slice() - const cursorWorkFailed = - cursorRows.length !== 0 || - cursorReads.length !== 2 || - cursorReads.some( - (read) => !read.sql.includes(' WHERE ') || read.rawRows !== 0, ) + const cursorReads = driver.reads.slice() + const cursorWorkFailed = + cursorRows.length !== 0 || + cursorReads.length !== 2 || + cursorReads.some( + (read) => !read.sql.includes(' WHERE ') || read.rawRows !== 0, + ) - const directFailures: Array = [] - for (const name of [ - 'ilike-unicode-exact', - 'ilike-unicode-prefix', - 'ilike-ascii-prefix', - 'gt-positive', - 'eq-lone-surrogate', - 'not-is-undefined', - 'not-is-null', - 'like-emoji-two', - ] as const) { - directFailures.push(...(await check(atom(name)))) - } + const directFailures: Array = [] + for (const name of [ + 'ilike-unicode-exact', + 'ilike-unicode-prefix', + 'ilike-ascii-prefix', + 'gt-positive', + 'eq-lone-surrogate', + 'not-is-undefined', + 'not-is-null', + 'like-emoji-two', + ] as const) { + directFailures.push(...(await check(atom(name)))) + } - expect(await check(op('and', atom('unsupported-function')))).toEqual([]) - expect(await check(op('or', atom('unsupported-path')))).toEqual([]) + expect(await check(op('and', atom('unsupported-function')))).toEqual([]) + expect(await check(op('or', atom('unsupported-path')))).toEqual([]) - const mixedAnd = op('and', atom('first'), atom('gt-positive')) - async function observeMixedAnd(model: Model = mixedAnd) { - driver.reads = [] - const rows = await adapter.loadSubset( + const mixedAnd = op('and', atom('first'), atom('gt-positive')) + async function observeMixedAnd(model: Model = mixedAnd) { + driver.reads = [] + const rows = await adapter.loadSubset( + { kind: `eager`, collectionId: 'boolean-arity' }, + { + where: toIR(model), + }, + ) + return { + keys: rows.map((row) => String(row.key)).sort(), + read: driver.reads[0], + } + } + const mixed = await observeMixedAnd() + expect(mixed.keys).toEqual(['row-1']) // NaN compares above zero in core. + expect(mixed.read?.sql).toContain(' WHERE ') + expect(mixed.read?.rawRows).toBe(1) // Certified id equality is selective. + + for (const model of [ + op('and', atom('gt-positive'), atom('first')), + op('and', op('and', atom('first'), atom('gt-positive'))), + op( + 'and', + atom('first'), + op('or', atom('ilike-unicode-exact'), atom('false')), + ), + op('and', atom('first'), atom('not-is-null')), + ]) { + const observation = await observeMixedAnd(model) + expect(observation.keys).toEqual(expectedKeys(model)) + expect(observation.read?.sql).toContain(' WHERE ') + expect(observation.read?.rawRows).toBe(1) + } + + driver.mutantDropWhere = true + const droppedConjunct = await observeMixedAnd() + driver.mutantDropWhere = false + expect(droppedConjunct.keys).toEqual(mixed.keys) + expect(droppedConjunct.read?.rawRows).toBe(fixture.length) + expect(droppedConjunct.read?.rawRows).not.toBe(mixed.read?.rawRows) + + // A legal nested predicate should require work proportional to its depth. + // Reading each node's `args` is a deterministic proxy for compiler visits. + let argumentReads = 0 + let nested: IR.BasicExpression = new IR.Value(true) + const depth = 80 + for (let index = 0; index < depth; index++) { + const child: IR.BasicExpression = nested + const node: IR.Func = new IR.Func('and', [child]) + Object.defineProperty(node, 'args', { + get() { + argumentReads++ + return [child] + }, + }) + nested = node + } + const nestedRows = await adapter.loadSubset( { kind: `eager`, collectionId: 'boolean-arity' }, { - where: toIR(model), + where: nested, }, ) - return { - keys: rows.map((row) => String(row.key)).sort(), - read: driver.reads[0], - } - } - const mixed = await observeMixedAnd() - expect(mixed.keys).toEqual(['row-1']) // NaN compares above zero in core. - expect(mixed.read?.sql).toContain(' WHERE ') - expect(mixed.read?.rawRows).toBe(1) // Certified id equality is selective. - - for (const model of [ - op('and', atom('gt-positive'), atom('first')), - op('and', op('and', atom('first'), atom('gt-positive'))), - op( - 'and', - atom('first'), + expect(nestedRows).toHaveLength(fixture.length) + expect(argumentReads).toBeLessThanOrEqual(depth * 10) + + driver.mutantPushUnsafeOr = true + const unsafeOr = await check( op('or', atom('ilike-unicode-exact'), atom('false')), - ), - op('and', atom('first'), atom('not-is-null')), - ]) { - const observation = await observeMixedAnd(model) - expect(observation.keys).toEqual(expectedKeys(model)) - expect(observation.read?.sql).toContain(' WHERE ') - expect(observation.read?.rawRows).toBe(1) - } + ) + driver.mutantPushUnsafeOr = false + expect(unsafeOr.map((failure) => failure.law)).toContain('public') + expect( + unsafeOr.find((failure) => failure.law === 'public')?.read.sql, + ).toContain(' WHERE ') - driver.mutantDropWhere = true - const droppedConjunct = await observeMixedAnd() - driver.mutantDropWhere = false - expect(droppedConjunct.keys).toEqual(mixed.keys) - expect(droppedConjunct.read?.rawRows).toBe(fixture.length) - expect(droppedConjunct.read?.rawRows).not.toBe(mixed.read?.rawRows) - - // A legal nested predicate should require work proportional to its depth. - // Reading each node's `args` is a deterministic proxy for compiler visits. - let argumentReads = 0 - let nested: IR.BasicExpression = new IR.Value(true) - const depth = 80 - for (let index = 0; index < depth; index++) { - const child: IR.BasicExpression = nested - const node: IR.Func = new IR.Func('and', [child]) - Object.defineProperty(node, 'args', { - get() { - argumentReads++ - return [child] - }, - }) - nested = node + driver.mutantUnaryValue = 'x' + expect( + (await check(op('and', atom('word')))).map((failure) => failure.law), + ).toContain('public') + driver.mutantUnaryValue = 1 + const blindOr = await check(op('or', atom('one'))) + driver.mutantUnaryValue = undefined + driver.mutantDropWhere = true + expect( + (await check(op('and', atom('first'), atom('true')))).map( + (failure) => failure.law, + ), + ).toContain('work') + driver.mutantDropWhere = false + + if ( + failures.length || + directFailures.length || + fixed.failed || + random.failed || + cursorWorkFailed + ) { + const count = (law: Failure['law']) => + failures.filter((failure) => failure.law === law).length + throw new Error( + `Boolean-arity oracle RED on current SQLite adapter: ` + + JSON.stringify({ + cases: matrix().length, + failures: { + public: count('public'), + work: count('work'), + compile: count('compile'), + candidate: count('candidate'), + }, + examples: failures + .filter((failure) => + ['public', 'work', 'compile'].includes(failure.law), + ) + .slice(0, 5) + .map(brief), + direct: directFailures.map(brief), + fixed, + random, + cursorWorkFailed, + cursorReads, + blindOrOutcome: + blindOr.length === 0 + ? 'survived: JS post-filter removed SQL over-selection' + : 'assertion failure', + }), + ) + } } - const nestedRows = await adapter.loadSubset( - { kind: `eager`, collectionId: 'boolean-arity' }, - { - where: nested, - }, - ) - expect(nestedRows).toHaveLength(fixture.length) - expect(argumentReads).toBeLessThanOrEqual(depth * 10) - driver.mutantPushUnsafeOr = true - const unsafeOr = await check( - op('or', atom('ilike-unicode-exact'), atom('false')), - ) - driver.mutantPushUnsafeOr = false - expect(unsafeOr.map((failure) => failure.law)).toContain('public') - expect( - unsafeOr.find((failure) => failure.law === 'public')?.read.sql, - ).toContain(' WHERE ') + let primary: unknown + let failed = false + try { + await runOracle() + } catch (error) { + failed = true + primary = error + } - driver.mutantUnaryValue = 'x' - expect( - (await check(op('and', atom('word')))).map((failure) => failure.law), - ).toContain('public') - driver.mutantUnaryValue = 1 - const blindOr = await check(op('or', atom('one'))) - driver.mutantUnaryValue = undefined - driver.mutantDropWhere = true - expect( - (await check(op('and', atom('first'), atom('true')))).map( - (failure) => failure.law, - ), - ).toContain('work') - driver.mutantDropWhere = false + let cleanupError: unknown + let cleanupFailed = false + try { + driver.db.close() + } catch (error) { + cleanupFailed = true + cleanupError = error + } - if ( - failures.length || - directFailures.length || - fixed.failed || - random.failed || - cursorWorkFailed - ) { - const count = (law: Failure['law']) => - failures.filter((failure) => failure.law === law).length - throw new Error( - `Boolean-arity oracle RED on current SQLite adapter: ` + - JSON.stringify({ - cases: matrix().length, - failures: { - public: count('public'), - work: count('work'), - compile: count('compile'), - candidate: count('candidate'), - }, - examples: failures - .filter((failure) => - ['public', 'work', 'compile'].includes(failure.law), - ) - .slice(0, 5) - .map(brief), - direct: directFailures.map(brief), - fixed, - random, - cursorWorkFailed, - cursorReads, - blindOrOutcome: - blindOr.length === 0 - ? 'survived: JS post-filter removed SQL over-selection' - : 'assertion failure', - }), + if (failed && cleanupFailed) { + throw new AggregateError( + [primary, cleanupError], + 'SQLite oracle and cleanup failed', + { cause: primary }, ) } - } - - let primary: unknown - let failed = false - try { - await runOracle() - } catch (error) { - failed = true - primary = error - } - - let cleanupError: unknown - let cleanupFailed = false - try { - driver.db.close() - } catch (error) { - cleanupFailed = true - cleanupError = error - } - - if (failed && cleanupFailed) { - throw new AggregateError( - [primary, cleanupError], - 'SQLite oracle and cleanup failed', - { cause: primary }, - ) - } - if (failed) throw primary - if (cleanupFailed) throw cleanupError -}) + if (failed) throw primary + if (cleanupFailed) throw cleanupError + }, +) it('keeps large string-ID OR scopes selective at the SQLite boundary', async () => { const driver = new CountingDriver() diff --git a/packages/db-sqlite-persistence-core/tests/sqlite-core-adapter-oracle.test.ts b/packages/db-sqlite-persistence-core/tests/sqlite-core-adapter-oracle.test.ts index 16633978b4..6595df4a91 100644 --- a/packages/db-sqlite-persistence-core/tests/sqlite-core-adapter-oracle.test.ts +++ b/packages/db-sqlite-persistence-core/tests/sqlite-core-adapter-oracle.test.ts @@ -9,6 +9,12 @@ import { inspect, promisify } from 'node:util' import { fc } from '@fast-check/vitest' import { afterEach, describe, expect, it } from 'vitest' import { IR, createCollection } from '@tanstack/db' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, + recordOracleReplay, +} from '../../db/tests/oracle-config.js' import { SQLiteCorePersistenceAdapter, createPersistedTableName, @@ -2527,138 +2533,136 @@ export function runSQLiteCoreAdapterContractSuite( * is removed after evidence capture. The failure reports first and reduced * histories plus a verified fast-check seed/path replay. */ - it(`resets collection metadata with its persisted baseline across generated adapter reopen histories`, async () => { - const historyArbitrary = fc - .constantFrom( - `compatible-reopen`, - `schema-reset`, - `partial-restore`, - `external-row-loss`, - ) - .chain((transition) => - fc.record({ - fromSchemaVersion: fc.integer({ min: 1, max: 4 }), - rows: fc - .uniqueArray(fc.integer({ min: 0, max: 20 }), { - minLength: - transition === `partial-restore` - ? 2 - : transition === `external-row-loss` - ? 1 - : 0, - maxLength: 4, - }) - .map((ids) => - ids - .sort((left, right) => left - right) - .map((id) => ({ - id: String(id), - title: `row-${id}`, - createdAt: `2026-01-01T00:00:00.000Z`, - score: id, - })), + oracleReplayTest(it, `sqlite-resume.baseline-lineage`)( + `resets collection metadata with its persisted baseline across generated adapter reopen histories`, + async () => { + const historyArbitrary = fc + .constantFrom( + `compatible-reopen`, + `schema-reset`, + `partial-restore`, + `external-row-loss`, + ) + .chain((transition) => + fc.record({ + fromSchemaVersion: fc.integer({ min: 1, max: 4 }), + rows: fc + .uniqueArray(fc.integer({ min: 0, max: 20 }), { + minLength: + transition === `partial-restore` + ? 2 + : transition === `external-row-loss` + ? 1 + : 0, + maxLength: 4, + }) + .map((ids) => + ids + .sort((left, right) => left - right) + .map((id) => ({ + id: String(id), + title: `row-${id}`, + createdAt: `2026-01-01T00:00:00.000Z`, + score: id, + })), + ), + resumeKind: fc.constantFrom(`none`, `reset`, `resume`), + transition: fc.constant(transition), + reopensBeforeTransition: fc.integer({ min: 0, max: 2 }), + reopensAfterTransition: fc.integer({ min: 0, max: 2 }), + unrelatedMetadataKeys: fc.uniqueArray( + fc.constantFrom(`gc`, `provider`, `custom`), + { maxLength: 3 }, ), - resumeKind: fc.constantFrom(`none`, `reset`, `resume`), - transition: fc.constant(transition), - reopensBeforeTransition: fc.integer({ min: 0, max: 2 }), - reopensAfterTransition: fc.integer({ min: 0, max: 2 }), - unrelatedMetadataKeys: fc.uniqueArray( - fc.constantFrom(`gc`, `provider`, `custom`), - { maxLength: 3 }, - ), - }), - ) + }), + ) - const seedText = - process.env.TANSTACK_DB_SQLITE_ORACLE_SEED ?? String(1659) - const seed = Number(seedText) - const path = process.env.TANSTACK_DB_SQLITE_ORACLE_PATH - const runsText = process.env.TANSTACK_DB_SQLITE_ORACLE_RUNS ?? String(24) - const numRuns = Number(runsText) - if (!Number.isSafeInteger(seed)) { - throw new Error(`TANSTACK_DB_SQLITE_ORACLE_SEED must be an integer`) - } - if (!Number.isSafeInteger(numRuns) || numRuns < 1) { - throw new Error(`TANSTACK_DB_SQLITE_ORACLE_RUNS must be positive`) - } - if (path !== undefined && !/^\d+(?::\d+)*$/.test(path)) { - throw new Error( - `TANSTACK_DB_SQLITE_ORACLE_PATH must be a numeric shrink path`, + // A seed-only shared run reseeds this campaign. Ordinary runs keep the + // fixed seed 1659, and a path replay runs once from its recorded path. + const parameters = oraclePropertyOptions( + 24, + `sqlite-resume.baseline-lineage`, ) - } - let originalFailure: ResetResumeOracleViolation | undefined - const property = fc.asyncProperty(historyArbitrary, async (history) => { - try { - await observeResetResumeHistory(history, harnessFactory) - } catch (error) { - if ( - originalFailure === undefined && - error instanceof ResetResumeOracleViolation - ) { - originalFailure = error + let originalFailure: ResetResumeOracleViolation | undefined + const property = fc.asyncProperty(historyArbitrary, async (history) => { + try { + await observeResetResumeHistory(history, harnessFactory) + } catch (error) { + if ( + originalFailure === undefined && + error instanceof ResetResumeOracleViolation + ) { + originalFailure = error + } + throw error } - throw error + }) + const failure = await fc.check(property, { + ...parameters, + seed: parameters.seed ?? 1659, + ...(parameters.path === undefined ? {} : { endOnFailure: true }), + }) + if (!failure.failed) return + if ( + !(failure.errorInstance instanceof ResetResumeOracleViolation) || + originalFailure === undefined || + failure.counterexamplePath === null + ) { + throw failure.errorInstance } - }) - const failure = await fc.check(property, { - seed, - numRuns, - ...(path === undefined ? {} : { path, endOnFailure: true }), - }) - if (!failure.failed) return - if ( - !(failure.errorInstance instanceof ResetResumeOracleViolation) || - originalFailure === undefined || - failure.counterexamplePath === null - ) { - throw failure.errorInstance - } - if (!hasSameResetResumeFailure(originalFailure, failure.errorInstance)) { - throw new Error( - `Shrinking changed the reset/resume law or observation checkpoint`, + if ( + !hasSameResetResumeFailure(originalFailure, failure.errorInstance) + ) { + throw new Error( + `Shrinking changed the reset/resume law or observation checkpoint`, + ) + } + + const replay = await fc.check( + fc.asyncProperty(historyArbitrary, async (history) => { + await observeResetResumeHistory(history, harnessFactory) + }), + { + seed: failure.seed, + path: failure.counterexamplePath, + endOnFailure: true, + }, ) - } + if ( + !replay.failed || + !(replay.errorInstance instanceof ResetResumeOracleViolation) || + JSON.stringify(replay.counterexample) !== + JSON.stringify(failure.counterexample) || + !hasSameResetResumeFailure( + failure.errorInstance, + replay.errorInstance, + ) + ) { + throw new Error( + `Reset/resume oracle replay did not reproduce the intended violation`, + ) + } - const replay = await fc.check( - fc.asyncProperty(historyArbitrary, async (history) => { - await observeResetResumeHistory(history, harnessFactory) - }), - { - seed: failure.seed, - path: failure.counterexamplePath, - endOnFailure: true, - }, - ) - if ( - !replay.failed || - !(replay.errorInstance instanceof ResetResumeOracleViolation) || - JSON.stringify(replay.counterexample) !== - JSON.stringify(failure.counterexample) || - !hasSameResetResumeFailure(failure.errorInstance, replay.errorInstance) - ) { + const reducedFailure = failure.errorInstance throw new Error( - `Reset/resume oracle replay did not reproduce the intended violation`, + `Reset/resume baseline-lineage violation. ` + + `seed=${failure.seed} path=${failure.counterexamplePath} ` + + `law=${reducedFailure.law} ` + + `discriminant=${reducedFailure.discriminant} ` + + `checkpoint=${reducedFailure.observation.checkpoint} ` + + `originalTrace=${JSON.stringify(originalFailure.history)} ` + + `reducedTrace=${JSON.stringify(reducedFailure.history)} ` + + `expected=${JSON.stringify(reducedFailure.expected)} ` + + `actual=${JSON.stringify(reducedFailure.actual)} ` + + `observation=${JSON.stringify(reducedFailure.observation)} ` + + `replay=verified ` + + `cleanup=${reducedFailure.cleanupEvidence}`, + { cause: reducedFailure }, ) - } - - const reducedFailure = failure.errorInstance - throw new Error( - `Reset/resume baseline-lineage violation. ` + - `seed=${failure.seed} path=${failure.counterexamplePath} ` + - `law=${reducedFailure.law} ` + - `discriminant=${reducedFailure.discriminant} ` + - `checkpoint=${reducedFailure.observation.checkpoint} ` + - `originalTrace=${JSON.stringify(originalFailure.history)} ` + - `reducedTrace=${JSON.stringify(reducedFailure.history)} ` + - `expected=${JSON.stringify(reducedFailure.expected)} ` + - `actual=${JSON.stringify(reducedFailure.actual)} ` + - `observation=${JSON.stringify(reducedFailure.observation)} ` + - `replay=verified ` + - `cleanup=${reducedFailure.cleanupEvidence}`, - { cause: reducedFailure }, - ) - }, 120_000) + }, + 120_000, + ) it(`leaves externally inconsistent metadata reachable for consumer validation`, async () => { const observation = await observeResetResumeHistory( @@ -3539,9 +3543,10 @@ export function runSQLiteCoreAdapterContractSuite( * fails the public result/binding comparison. The legacy CLI test interpolates * literals, so it cannot exercise this boundary. * - * Replay: pass `TANSTACK_DB_SQLITE_BINDING_SEED` and - * `TANSTACK_DB_SQLITE_BINDING_PATH` together and select this test by - * name. The normal package test runs fixed and random campaigns. + * Replay: `TANSTACK_DB_ORACLE_PATH` with `TANSTACK_DB_ORACLE_SEED` and + * `TANSTACK_DB_ORACLE_PROPERTY=sqlite-persistence.binding-capacity` replays the + * recorded random-campaign history. The normal package test runs fixed and + * random campaigns. */ const BINDING_CAP = 999 const BINDING_FIXED_SEED = 1_993_999 @@ -4261,27 +4266,8 @@ async function runCursorContext(): Promise { return problem } -const bindingReplaySeed = process.env.TANSTACK_DB_SQLITE_BINDING_SEED -const bindingReplayPath = process.env.TANSTACK_DB_SQLITE_BINDING_PATH -const bindingReplay = - bindingReplaySeed !== undefined && bindingReplayPath !== undefined -if ((bindingReplaySeed === undefined) !== (bindingReplayPath === undefined)) { - throw new Error(`Binding oracle replay requires both seed and path`) -} -if ( - bindingReplaySeed !== undefined && - !Number.isSafeInteger(Number(bindingReplaySeed)) -) { - throw new Error(`Binding oracle replay seed must be an integer`) -} -if ( - bindingReplayPath !== undefined && - !/^\d+(?::\d+)*$/.test(bindingReplayPath) -) { - throw new Error(`Binding oracle replay path must be numeric`) -} - -const bindingFixedIt = bindingReplay ? it.skip : it +const BINDING_PROPERTY = `sqlite-persistence.binding-capacity` +const bindingFixedIt = oracleReplayTest(it) export function runSQLiteBindingCapacityOracleSuite(): void { describe(`SQLite subset binding-capacity oracle`, () => { @@ -4399,7 +4385,7 @@ export function runSQLiteBindingCapacityOracleSuite(): void { async function campaign( label: string, - parameters: { seed?: number; path?: string }, + parameters: { numRuns: number; seed?: number; path?: string }, ) { let firstFailure: Violation | undefined const property = fc.asyncProperty(generated, async (spec) => { @@ -4413,10 +4399,8 @@ export function runSQLiteBindingCapacityOracleSuite(): void { ) } }) - const details = await fc.check(property, { - numRuns: BINDING_RUNS, - ...parameters, - }) + const details = await fc.check(property, parameters) + recordOracleReplay(BINDING_PROPERTY, details) const reduced = details.counterexample?.[0] const reducedProblem = reduced ? violation(reduced, await observe(reduced)) @@ -4435,20 +4419,24 @@ export function runSQLiteBindingCapacityOracleSuite(): void { ).toBe(false) } - if (bindingReplay) { - it(`replays the requested binding history directly`, async () => { - await campaign(`direct replay`, { - seed: Number(bindingReplaySeed), - path: bindingReplayPath, + bindingFixedIt( + `preserves exact rows across fixed-seed binding histories`, + async () => { + await campaign(`fixed seed`, { + numRuns: oracleRuns(BINDING_RUNS), + seed: BINDING_FIXED_SEED, }) - }) - } else { - it(`preserves exact rows across fixed-seed binding histories`, async () => { - await campaign(`fixed seed`, { seed: BINDING_FIXED_SEED }) - }) - it(`preserves exact rows across random-seed binding histories`, async () => { - await campaign(`random seed`, {}) - }) - } + }, + ) + // Runs in ordinary and seed-only runs, and as the replay of its property. + oracleReplayTest(it, BINDING_PROPERTY)( + `preserves exact rows across random-seed binding histories`, + async () => { + await campaign( + `random seed`, + oraclePropertyOptions(BINDING_RUNS, BINDING_PROPERTY), + ) + }, + ) }) } diff --git a/packages/db/e2e/indexed-db/cross-tab-oracle.spec.ts b/packages/db/e2e/indexed-db/cross-tab-oracle.spec.ts index 3993d3ba38..624ae33f8d 100644 --- a/packages/db/e2e/indexed-db/cross-tab-oracle.spec.ts +++ b/packages/db/e2e/indexed-db/cross-tab-oracle.spec.ts @@ -22,6 +22,11 @@ import { assertStatuses, } from '../../tests/indexed-db/cross-tab-oracle' import { runCampaign } from '../../tests/indexed-db/campaign' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../tests/oracle-config' import { retainedDocument } from './browser-evidence-oracle' import type { BrowserContext, Page } from '@playwright/test' import type { BrowserEvidence } from './browser' @@ -399,46 +404,62 @@ async function nativeHistoryRun( } } -const replaySeed = process.env.TANSTACK_INDEXEDDB_BROWSER_SEED -const replayPath = process.env.TANSTACK_INDEXEDDB_BROWSER_PATH -for (const seed of replaySeed ? [Number(replaySeed)] : [1179, undefined]) - test(`native authored histories preserve every publication (${seed === undefined ? 'fresh' : 'fixed'})`, async ({ - context, - }) => { - await withPages(context, async (events, database) => { - let count = 0 - await runCampaign( - `native-${test.info().project.name}`, - nativeHistory, - async (steps, receipt) => { - let primary: unknown - try { - await nativeHistoryRun( - context, - events, - `${database}-${count++}`, - steps, - receipt.reach, - ) - } catch (error) { - primary = error - } - const cleanup = await closePages(context) - if (cleanup.length) - throw new AggregateError(cleanup, 'Native history cleanup failed', { - cause: primary, - }) - if (primary !== undefined) throw primary - }, - { - runs: 10, - seed, - path: replayPath, - environment: `${test.info().project.name} ${context.browser()!.version()}`, - }, - ) - }) +// The fixed campaign stands aside during every path replay. A path replay of +// `indexeddb.browser-native-history` runs only the fresh campaign, from its +// recorded seed and path. +const nativeHistoryProperty = `indexeddb.browser-native-history` +async function nativeHistoryCampaign( + context: BrowserContext, + options: { numRuns: number; seed?: number; path?: string }, +) { + await withPages(context, async (events, database) => { + let count = 0 + await runCampaign( + `native-${test.info().project.name}`, + nativeHistory, + async (steps, receipt) => { + let primary: unknown + try { + await nativeHistoryRun( + context, + events, + `${database}-${count++}`, + steps, + receipt.reach, + ) + } catch (error) { + primary = error + } + const cleanup = await closePages(context) + if (cleanup.length) + throw new AggregateError(cleanup, 'Native history cleanup failed', { + cause: primary, + }) + if (primary !== undefined) throw primary + }, + { + runs: options.numRuns, + seed: options.seed, + path: options.path, + environment: `${test.info().project.name} ${context.browser()!.version()}`, + }, + ) }) +} + +oracleReplayTest(test)( + `native authored histories preserve every publication (fixed)`, + ({ context }) => + nativeHistoryCampaign(context, { seed: 1179, numRuns: oracleRuns(10) }), +) +oracleReplayTest(test, nativeHistoryProperty)( + `native authored histories preserve every publication (fresh)`, + ({ context }) => + nativeHistoryCampaign( + context, + oraclePropertyOptions(10, nativeHistoryProperty), + ), +) // Explicit manual acceptance creates ordered storage transactions before either // notification can run. Automatic write ordering has a separate native witness. diff --git a/packages/db/tests/btree-map-oracle.test.ts b/packages/db/tests/btree-map-oracle.test.ts index a3cd6b9e25..80a78cb6f5 100644 --- a/packages/db/tests/btree-map-oracle.test.ts +++ b/packages/db/tests/btree-map-oracle.test.ts @@ -1,6 +1,11 @@ import { describe, expect, it, vi } from 'vitest' import { fc } from '@fast-check/vitest' import { BTree } from '../src/utils/btree.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from './oracle-config.js' /** * # When does the B+ tree behave like a sorted Map? @@ -95,25 +100,12 @@ const arbitraryAction: fc.Arbitrary = fc.oneof( }), ) -// Replay a reported fast-check seed and shrink path directly with -// BTREE_MAP_ORACLE_SEED= BTREE_MAP_ORACLE_PATH= pnpm exec vitest run -// packages/db/tests/btree-map-oracle.test.ts. Ordinary runs execute both lanes. -const replaySeedText = process.env.BTREE_MAP_ORACLE_SEED -const replayPath = process.env.BTREE_MAP_ORACLE_PATH -if ((replaySeedText === undefined) !== (replayPath === undefined)) { - throw new Error(`BTree Map oracle replay requires both seed and path`) -} -const replaySeed = - replaySeedText === undefined ? undefined : Number(replaySeedText) -if ( - replaySeedText !== undefined && - (replaySeedText.trim() === `` || !Number.isSafeInteger(replaySeed)) -) { - throw new Error(`BTREE_MAP_ORACLE_SEED must be an integer`) -} -if (replayPath !== undefined && !/^\d+(?::\d+)*$/.test(replayPath)) { - throw new Error(`BTREE_MAP_ORACLE_PATH must be a fast-check shrink path`) -} +// A fixed campaign and a random campaign run the same history grammar. A path +// replay of this property runs only its random campaign from the recorded seed +// and shrink path: TANSTACK_DB_ORACLE_SEED, TANSTACK_DB_ORACLE_PATH, and +// TANSTACK_DB_ORACLE_PROPERTY=btree-map.history with +// `pnpm exec vitest run packages/db/tests/btree-map-oracle.test.ts`. +const btreeMapProperty = `btree-map.history` const mapHistoryProperty = fc.property( fc.integer({ min: 4, max: 8 }), @@ -129,14 +121,18 @@ const mapHistoryProperty = fc.property( }, ) -const mapHistoryCampaigns: Array<{ - name: string - seed?: number - path?: string -}> = - replayPath === undefined - ? [{ name: `fixed`, seed: 20260930 }, { name: `random` }] - : [{ name: `replay`, seed: replaySeed, path: replayPath }] +const mapHistoryCampaigns = [ + { + name: `fixed`, + test: oracleReplayTest(it), + parameters: { numRuns: oracleRuns(100), seed: 20260930 }, + }, + { + name: `random`, + test: oracleReplayTest(it, btreeMapProperty), + parameters: oraclePropertyOptions(100, btreeMapProperty), + }, +] // The model uses only Map operations and the declared numeric order. It does // not use the tree's nodes, search helpers, or traversal code. @@ -335,16 +331,13 @@ describe(`BTree Map oracle`, () => { }, ) - it.each(mapHistoryCampaigns)( - `matches complete Map cuts across shrinkable histories ($name)`, - ({ seed, path }) => { - fc.assert(mapHistoryProperty, { - numRuns: path === undefined ? 100 : 1, - seed, - path, - }) - }, - ) + for (const campaign of mapHistoryCampaigns) + campaign.test( + `matches complete Map cuts across shrinkable histories (${campaign.name})`, + () => { + fc.assert(mapHistoryProperty, campaign.parameters) + }, + ) it(`replays a shrunk Map mismatch with the reported seed and path`, () => { // A scanner that drops every row keeps the initial empty cut green. The diff --git a/packages/db/tests/collection-metadata-publication-oracle.property.test.ts b/packages/db/tests/collection-metadata-publication-oracle.property.test.ts index 518e1472aa..9d23d955d3 100644 --- a/packages/db/tests/collection-metadata-publication-oracle.property.test.ts +++ b/packages/db/tests/collection-metadata-publication-oracle.property.test.ts @@ -7,8 +7,9 @@ import { createLiveQueryCollection } from '../src/query/index.js' import { createTransaction } from '../src/transactions.js' import { oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from './oracle-config.js' import { withHistoryCleanup } from './optimistic-history-oracle.js' import type { Collection } from '../src/collection/index.js' @@ -164,7 +165,6 @@ const metadataCancellationArbitrary = fc.record({ const metadataOnlyProperty = `collection-publication.metadata-only` const metadataCancellationProperty = `collection-publication.metadata-cancellation` -const requestedReplayProperty = readOracleRunConfig().replayProperty async function createPublicationHarness(): Promise { let sync!: SyncActions @@ -818,7 +818,7 @@ const runCancellationHistory = ({ ) describe(`generated metadata publication histories`, () => { - if (requestedReplayProperty === undefined) { + if (!oracleReplayRequested()) { fcTest.prop([publicationHistoryArbitrary], { numRuns: oracleRuns(50), seed: 1_805_001, @@ -826,13 +826,6 @@ describe(`generated metadata publication histories`, () => { `keeps metadata-only optimistic settlement a valid keyed diff (fixed)`, runPublicationHistory, ) - fcTest.prop( - [publicationHistoryArbitrary], - oraclePropertyOptions(50, metadataOnlyProperty), - )( - `keeps metadata-only optimistic settlement a valid keyed diff (random)`, - runPublicationHistory, - ) fcTest.prop([metadataCancellationArbitrary], { numRuns: oracleRuns(50), @@ -841,22 +834,21 @@ describe(`generated metadata publication histories`, () => { `keeps metadata suppression owned by pending transactions (fixed)`, runCancellationHistory, ) - fcTest.prop( - [metadataCancellationArbitrary], - oraclePropertyOptions(50, metadataCancellationProperty), - )( - `keeps metadata suppression owned by pending transactions (random)`, - runCancellationHistory, - ) - } else if (requestedReplayProperty === metadataOnlyProperty) { - fcTest.prop( - [publicationHistoryArbitrary], - oraclePropertyOptions(50, metadataOnlyProperty), - )(`replays metadata-only optimistic settlement`, runPublicationHistory) - } else if (requestedReplayProperty === metadataCancellationProperty) { - fcTest.prop( - [metadataCancellationArbitrary], - oraclePropertyOptions(50, metadataCancellationProperty), - )(`replays metadata cancellation ownership`, runCancellationHistory) } + // A path replay of one property runs its one recorded history through that + // property's registration. The other registration stands aside. + oracleReplayTest(fcTest, metadataOnlyProperty).prop( + [publicationHistoryArbitrary], + oraclePropertyOptions(50, metadataOnlyProperty), + )( + `keeps metadata-only optimistic settlement a valid keyed diff (random)`, + runPublicationHistory, + ) + oracleReplayTest(fcTest, metadataCancellationProperty).prop( + [metadataCancellationArbitrary], + oraclePropertyOptions(50, metadataCancellationProperty), + )( + `keeps metadata suppression owned by pending transactions (random)`, + runCancellationHistory, + ) }) diff --git a/packages/db/tests/collection-subscription-lifecycle-history-oracle.property.test.ts b/packages/db/tests/collection-subscription-lifecycle-history-oracle.property.test.ts index df7a9775d5..2350cbab5a 100644 --- a/packages/db/tests/collection-subscription-lifecycle-history-oracle.property.test.ts +++ b/packages/db/tests/collection-subscription-lifecycle-history-oracle.property.test.ts @@ -23,6 +23,8 @@ import { flushPromises, withOracleCleanup } from './utils.js' import { oraclePropertyOptions, oracleRandomParameters, + oracleReplayRequested, + oracleReplayTest, readOracleRunConfig, } from './oracle-config.js' import type { LoadSubsetOptions, SyncConfig } from '../src/types.js' @@ -37,12 +39,9 @@ import type { } from './collection-subscription-lifecycle-grammar-oracle.js' const { multiplier, ...replay } = readOracleRunConfig() -const it = replay.replayPath === undefined ? vitestIt : vitestIt.skip -const fixedCampaign = replay.replayPath === undefined ? fcTest : fcTest.skip -const randomCampaign = (property: string) => - replay.replayPath === undefined || replay.replayProperty === property - ? fcTest - : fcTest.skip +const it = oracleReplayTest(vitestIt) +const fixedCampaign = oracleReplayTest(fcTest) +const randomCampaign = (property: string) => oracleReplayTest(fcTest, property) /** * # What settles a demand created while a Collection is cleaned up? @@ -853,7 +852,7 @@ async function runHistory( } if ( - replay.replayPath === undefined && + !oracleReplayRequested() && process.env.TANSTACK_DB_ORACLE_STATISTICS === `1` ) { fc.statistics( diff --git a/packages/db/tests/collection-subscription-lifecycle-oracle.test.ts b/packages/db/tests/collection-subscription-lifecycle-oracle.test.ts index 0a7ae23cca..6c7f9dbe20 100644 --- a/packages/db/tests/collection-subscription-lifecycle-oracle.test.ts +++ b/packages/db/tests/collection-subscription-lifecycle-oracle.test.ts @@ -12,6 +12,8 @@ import { import { oraclePropertyOptions, oracleRandomParameters, + oracleReplayRequested, + oracleReplayTest, readOracleRunConfig, } from './oracle-config.js' import type { CollectionSubscription } from '../src/collection/subscription.js' @@ -1049,7 +1051,7 @@ describe(`CollectionSubscription demand lifecycle oracle`, () => { afterAll(() => { // A direct property replay may select only that test by name. - if (replay.replayPath !== undefined) return + if (oracleReplayRequested()) return expect(observedAcquisitionCells).toEqual(legalAcquisitionCells) expect(observedAcquisitionInteractions).toEqual( requiredAcquisitionInteractions, @@ -4409,7 +4411,7 @@ describe(`CollectionSubscription demand lifecycle oracle`, () => { }, ) - const fixedCampaign = replay.replayPath === undefined ? fcTest : fcTest.skip + const fixedCampaign = oracleReplayTest(fcTest) fixedCampaign.prop([asyncRestartScenarioArbitrary], { numRuns: 30 * multiplier, diff --git a/packages/db/tests/comparison-oracle.property.test.ts b/packages/db/tests/comparison-oracle.property.test.ts index 1f8277456f..c497ba6cf4 100644 --- a/packages/db/tests/comparison-oracle.property.test.ts +++ b/packages/db/tests/comparison-oracle.property.test.ts @@ -10,6 +10,13 @@ import { normalizeValue, } from '../src/utils/comparison' import { DEFAULT_COMPARE_OPTIONS } from '../src/utils' +import { + oraclePropertyOptions, + oracleReplayRequested, + oracleReplaySkips, + oracleRuns, + recordOracleReplay, +} from './oracle-config.js' import type { CompareOptions } from '../src/query/builder/types' /** @@ -68,15 +75,14 @@ const nullsLastOpts: CompareOptions = { // These properties judge immediate comparator and normalization results; there // is no mutable history or cleanup. Each function return is its observation cut. // The fixed campaign preserves one sample set, while the random campaign explores -// new values. Set all three variables to replay one property directly, for -// example COMPARISON_ORACLE_PROPERTY=1 COMPARISON_ORACLE_SEED=123 -// COMPARISON_ORACLE_PATH='0:0' pnpm --filter @tanstack/db exec vitest run +// new values. A path replay runs one property directly, for example +// TANSTACK_DB_ORACLE_PROPERTY=comparison.1 TANSTACK_DB_ORACLE_SEED=123 +// TANSTACK_DB_ORACLE_PATH='0:0' pnpm --filter @tanstack/db exec vitest run // --configLoader runner tests/comparison-oracle.property.test.ts. const fixedSeed = 20260911 const campaignRuns = 100 -const requestedProperty = process.env.COMPARISON_ORACLE_PROPERTY -const replaySeed = process.env.COMPARISON_ORACLE_SEED -const replayPath = process.env.COMPARISON_ORACLE_PATH +// Each property registers as `comparison.`, in file order. A path +// replay selects one with TANSTACK_DB_ORACLE_PROPERTY=comparison.. let nextPropertyId = 0 type ComparisonFailure = { checkpoint: string; error: unknown } @@ -99,6 +105,7 @@ function checkWithOriginalFailure( arbitrary: fc.Arbitrary, check: (value: T) => void, options: Parameters[1], + replayProperty?: string, ) { let original: ComparisonFailure | undefined let reduced: ComparisonFailure | undefined @@ -116,6 +123,7 @@ function checkWithOriginalFailure( } }) const result = fc.check(property, options) + if (replayProperty !== undefined) recordOracleReplay(replayProperty, result) return { result, original, reduced } } @@ -124,11 +132,14 @@ function assertComparisonProperty( arbitrary: fc.Arbitrary, check: (value: T) => void, options: Parameters[1], + // The registered property, so a guarded replay can witness this run. + replayProperty?: string, ): void { const { result, original, reduced } = checkWithOriginalFailure( arbitrary, check, options, + replayProperty, ) if (!result.failed) return if (!original || !reduced || result.counterexamplePath === null) { @@ -161,45 +172,28 @@ function comparisonProperty< check: (...values: Ts) => void, ) => void { return (name: string, check: (...values: Ts) => void): void => { - const id = String(++nextPropertyId) + const id = ++nextPropertyId + const property = `comparison.${id}` const label = `${id}: ${name}` const arbitrary = fc.tuple(...arbitraries) const checkValues = (values: Ts) => { check(...values) } - if (requestedProperty !== undefined) { - if (requestedProperty !== id) { - it.skip(`${label} replay not selected`, () => {}) - return - } - if (replaySeed === undefined || replayPath === undefined) { - throw new Error(`comparison oracle replay needs a seed and shrink path`) - } - const seed = Number(replaySeed) - if (replaySeed.trim() === `` || !Number.isSafeInteger(seed)) { - throw new Error(`comparison oracle replay seed must be an integer`) - } - it(`${label} replay`, () => { - assertComparisonProperty(label, arbitrary, checkValues, { - seed, - path: replayPath, - numRuns: campaignRuns, - }) - }) - return - } - - it(`${label} fixed`, () => { + it.skipIf(oracleReplayRequested())(`${label} fixed`, () => { assertComparisonProperty(label, arbitrary, checkValues, { seed: fixedSeed, - numRuns: campaignRuns, + numRuns: oracleRuns(campaignRuns), }) }) - it(`${label} random`, () => { - assertComparisonProperty(label, arbitrary, checkValues, { - numRuns: campaignRuns, - }) + it.skipIf(oracleReplaySkips(property))(`${label} random`, () => { + assertComparisonProperty( + label, + arbitrary, + checkValues, + oraclePropertyOptions(campaignRuns, property), + property, + ) }) } } @@ -1302,11 +1296,3 @@ describe(`Temporal default-order refinement`, () => { expect(() => assertZonedDateTimeOrder(sample, truncated)).toThrow() }) }) - -if (requestedProperty !== undefined) { - it(`selects a registered comparison oracle property`, () => { - expect(requestedProperty).toMatch(/^\d+$/) - expect(Number(requestedProperty)).toBeGreaterThanOrEqual(1) - expect(Number(requestedProperty)).toBeLessThanOrEqual(nextPropertyId) - }) -} diff --git a/packages/db/tests/cursor-oracle.property.test.ts b/packages/db/tests/cursor-oracle.property.test.ts index 410107caaa..25c74e4742 100644 --- a/packages/db/tests/cursor-oracle.property.test.ts +++ b/packages/db/tests/cursor-oracle.property.test.ts @@ -5,8 +5,8 @@ import { PropRef, Value } from '../src/query/ir.js' import { buildCursor } from '../src/utils/cursor.js' import { oraclePropertyOptions, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from './oracle-config.js' import { withHistoryCleanup } from './optimistic-history-oracle.js' import { evaluateReferenceExpression } from './reference-expression-oracle.js' @@ -31,13 +31,9 @@ import type { OrderBy } from '../src/query/ir.js' * paginated query's acquisition and publication behavior. */ -const replay = readOracleRunConfig() -const it = replay.replayPath === undefined ? vitestIt : vitestIt.skip -const fixedCampaign = replay.replayPath === undefined ? fcTest : fcTest.skip -const randomCampaign = (property: string) => - replay.replayPath === undefined || replay.replayProperty === property - ? fcTest - : fcTest.skip +const it = oracleReplayTest(vitestIt) +const fixedCampaign = oracleReplayTest(fcTest) +const randomCampaign = (property: string) => oracleReplayTest(fcTest, property) const fixedSeed = 20260930 diff --git a/packages/db/tests/d2-source-reconciliation-oracle.property.test.ts b/packages/db/tests/d2-source-reconciliation-oracle.property.test.ts index a2f6480b0a..85ae9bd339 100644 --- a/packages/db/tests/d2-source-reconciliation-oracle.property.test.ts +++ b/packages/db/tests/d2-source-reconciliation-oracle.property.test.ts @@ -10,17 +10,15 @@ import { createEffect } from '../src/query/effect.js' import { reconcileChangesForD2 } from '../src/query/live/utils.js' import { oraclePropertyOptions, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from './oracle-config.js' import { flushPromises } from './utils.js' import type { ChangeMessage, SyncConfig } from '../src/types.js' -const { replayPath, replayProperty } = readOracleRunConfig() -const it = replayPath === undefined ? vitestIt : vitestIt.skip -const fixedCampaign = replayPath === undefined ? fcTest : fcTest.skip -const randomCampaign = (property: string) => - replayPath === undefined || replayProperty === property ? fcTest : fcTest.skip +const it = oracleReplayTest(vitestIt) +const fixedCampaign = oracleReplayTest(fcTest) +const randomCampaign = (property: string) => oracleReplayTest(fcTest, property) /** * D2 source reconciliation turns source-key snapshots into exact signed row diff --git a/packages/db/tests/flat-change-tracking-oracle.property.test.ts b/packages/db/tests/flat-change-tracking-oracle.property.test.ts index 634beafd2d..150f6c98c8 100644 --- a/packages/db/tests/flat-change-tracking-oracle.property.test.ts +++ b/packages/db/tests/flat-change-tracking-oracle.property.test.ts @@ -79,12 +79,12 @@ import { } from '../src/proxy.js' import { oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from './oracle-config.js' const property = `flat-change-tracking.equivalence` -const requestedReplayProperty = readOracleRunConfig().replayProperty const MISSING = Symbol(`missing`) const FRESH_OBJECT = Symbol(`fresh object`) @@ -660,7 +660,7 @@ function runHistory(history: History): void { } describe(`flat change tracking oracle`, () => { - if (requestedReplayProperty === undefined) { + if (!oracleReplayRequested()) { for (const { name, history } of pinnedHistories) { it(`matches the draft proxy when ${name}`, () => runHistory(history)) } @@ -713,16 +713,10 @@ describe(`flat change tracking oracle`, () => { seed: 44_502_101, numRuns: oracleRuns(200), })(`matches the draft proxy (fixed)`, runHistory) - fcTest.prop([historyArbitrary], oraclePropertyOptions(200, property))( - `matches the draft proxy (random)`, - runHistory, - ) - } else if (requestedReplayProperty === property) { - fcTest.prop([historyArbitrary], oraclePropertyOptions(200, property))( - `matches the draft proxy (replay)`, - runHistory, - ) - } else { - it.skip(`runs only when its replay property is selected`, () => {}) } + // A path replay of this property runs its one recorded history here. + oracleReplayTest(fcTest, property).prop( + [historyArbitrary], + oraclePropertyOptions(200, property), + )(`matches the draft proxy (random)`, runHistory) }) diff --git a/packages/db/tests/indexed-db/ORACLE.md b/packages/db/tests/indexed-db/ORACLE.md index 439c095e83..88de6cc50b 100644 --- a/packages/db/tests/indexed-db/ORACLE.md +++ b/packages/db/tests/indexed-db/ORACLE.md @@ -214,13 +214,15 @@ facet without replaying both. From this package directory: ```sh -INDEXEDDB_ORACLE_SEED=1179001 INDEXEDDB_ORACLE_PATH=0:1:0:0:2:2 \ +TANSTACK_DB_ORACLE_SEED=1179001 TANSTACK_DB_ORACLE_PATH=0:1:0:0:2:2 \ + TANSTACK_DB_ORACLE_PROPERTY=indexeddb.settled-history \ pnpm exec vitest run tests/indexed-db/persistence-oracle.test.ts \ - -t 'replay histories' --coverage.enabled=false --typecheck.enabled=false + -t 'random histories' --coverage.enabled=false --typecheck.enabled=false ``` -Replay selects only that campaign without fixed/random execution first. This -uses an array arbitrary, not fc.commands, so command replayPath does not apply. +A path replay of `indexeddb.settled-history` runs only its random campaign; the +fixed campaign stands aside. This uses an array arbitrary, not fc.commands, so +command replayPath does not apply. Those coordinates reproduced the original clear failure at the public snapshot checkpoint and pass with the repair. The update/clear counterexample is also a permanent authored case. Cleanup captures the primary error before releasing @@ -360,10 +362,11 @@ pnpm test:indexed-db:oracles TANSTACK_INDEXEDDB_ORACLE_PROFILE=stress pnpm test:indexed-db:oracles pnpm exec playwright install chromium firefox webkit pnpm test:indexed-db:browser -# Direct selected history; fixed/fresh campaigns do not run first: -TANSTACK_INDEXEDDB_ORACLE_SEED= TANSTACK_INDEXEDDB_ORACLE_PATH= pnpm test:indexed-db:oracles -TANSTACK_INDEXEDDB_PENDING_SEED= TANSTACK_INDEXEDDB_PENDING_PATH= pnpm test:indexed-db:oracles -TANSTACK_INDEXEDDB_BROWSER_SEED= TANSTACK_INDEXEDDB_BROWSER_PATH= pnpm test:indexed-db:browser --project=chromium +# Direct selected cross-tab history; the fixed campaign does not run first: +TANSTACK_DB_ORACLE_SEED= TANSTACK_DB_ORACLE_PATH= TANSTACK_DB_ORACLE_PROPERTY=indexeddb.cross-tab-history pnpm test:indexed-db:oracles +# Direct selected pending history; the fixed and other tests stand aside: +TANSTACK_DB_ORACLE_SEED= TANSTACK_DB_ORACLE_PATH= TANSTACK_DB_ORACLE_PROPERTY=indexeddb.pending-history pnpm test:indexed-db:oracles +TANSTACK_DB_ORACLE_SEED= TANSTACK_DB_ORACLE_PATH= TANSTACK_DB_ORACLE_PROPERTY=indexeddb.browser-native-history pnpm test:indexed-db:browser --project=chromium ``` ### Remaining boundaries diff --git a/packages/db/tests/indexed-db/cross-tab-history-oracle.test.ts b/packages/db/tests/indexed-db/cross-tab-history-oracle.test.ts index cfd33f97ba..d6903b9340 100644 --- a/packages/db/tests/indexed-db/cross-tab-history-oracle.test.ts +++ b/packages/db/tests/indexed-db/cross-tab-history-oracle.test.ts @@ -18,6 +18,12 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { fc } from '@fast-check/vitest' import { expect, it, vi } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayRequested, + oracleReplaySkips, + oracleRuns, +} from '../oracle-config.js' import { runCampaign } from './campaign' import { runTransportHistory } from './cross-tab-driver' import { @@ -59,7 +65,7 @@ const action = ( ): TransportStep => ({ kind, tab, key, value: 2, optional: true }) const stores = stress ? [0, 0, 1, 0, 1] : [0, 0, 1] -async function campaign(seed?: number, path?: string) { +async function campaign(numRuns: number, seed?: number, path?: string) { await runCampaign( 'transport', history, @@ -72,30 +78,30 @@ async function campaign(seed?: number, path?: string) { 'registered campaign calibration', ) }, - { runs, seed, path }, + { runs: numRuns, seed, path }, ) } -if (process.env.TANSTACK_INDEXEDDB_ORACLE_SEED) { - const seed = Number(process.env.TANSTACK_INDEXEDDB_ORACLE_SEED) - if (!Number.isInteger(seed)) throw new Error('Invalid oracle replay seed') - it( - 'replays the selected cross-tab history', - () => campaign(seed, process.env.TANSTACK_INDEXEDDB_ORACLE_PATH), - 120_000, - ) -} else { - it( - 'refines independent per-Collection snapshots with fixed histories', - () => campaign(11792026), - 120_000, - ) - it( - 'refines independent per-Collection snapshots with fresh histories', - () => campaign(), - 120_000, - ) -} +// A path replay of this property runs only the fresh campaign, from the +// recorded seed and path. A seed without a path reseeds the fresh campaign. +const crossTabProperty = `indexeddb.cross-tab-history` + +it.skipIf(oracleReplayRequested())( + 'refines independent per-Collection snapshots with fixed histories', + () => campaign(oracleRuns(runs), 11792026), + 120_000, +) +it.skipIf(oracleReplaySkips(crossTabProperty))( + 'refines independent per-Collection snapshots with fresh histories', + () => { + const { numRuns, seed, path } = oraclePropertyOptions( + runs, + crossTabProperty, + ) + return campaign(numRuns, seed, path) + }, + 120_000, +) it('reconstructs every action and overlapping receiver reads', async () => { const reached = await runTransportHistory([ diff --git a/packages/db/tests/indexed-db/pending-history-oracle.test.ts b/packages/db/tests/indexed-db/pending-history-oracle.test.ts index f921ac1776..0ed2a3e9b1 100644 --- a/packages/db/tests/indexed-db/pending-history-oracle.test.ts +++ b/packages/db/tests/indexed-db/pending-history-oracle.test.ts @@ -19,8 +19,13 @@ * every boundary. Fixed/fresh histories use the same generator and checker. */ import { fc } from '@fast-check/vitest' -import { expect, it } from 'vitest' +import { expect, it as vitestIt } from 'vitest' import { createLiveQueryCollection } from '../../src' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../oracle-config' import { Channel, deferred, @@ -41,6 +46,14 @@ import { runCampaign } from './campaign' import type { Change, OracleRow } from './cross-tab-oracle' import type { Operation } from './harness' +// Campaigns. The fixed campaign and the fresh campaign run the same generator +// and checker. A path replay of `indexeddb.pending-history` runs its recorded +// history once, and every other test in this file stands aside. A seed alone +// reseeds the fresh campaign. +const FIXED_SEED = 11792027 +const PENDING_HISTORY = `indexeddb.pending-history` +const it = oracleReplayTest(vitestIt) + type PeerStep = { kind: 'write' | 'delete' | 'clear' | 'import' overlap: boolean @@ -428,19 +441,26 @@ it('a peer no-op cannot spend another source acknowledgement', async () => { }) }) -const replaySeed = process.env.TANSTACK_INDEXEDDB_PENDING_SEED -const replayPath = process.env.TANSTACK_INDEXEDDB_PENDING_PATH -for (const seed of replaySeed ? [Number(replaySeed)] : [11792027, undefined]) - it(`refines held mutations through ${seed === undefined ? 'fresh' : 'fixed'} peer histories`, async () => { +const runs = + process.env.TANSTACK_INDEXEDDB_ORACLE_PROFILE === 'stress' ? 300 : 30 +it(`refines held mutations through fixed peer histories`, async () => { + await runCampaign( + 'pending', + histories, + (history, receipt) => run(history, receipt.reach), + { runs: oracleRuns(runs), seed: FIXED_SEED }, + ) +}, 120_000) +oracleReplayTest(vitestIt, PENDING_HISTORY)( + `refines held mutations through fresh peer histories`, + async () => { + const { numRuns, seed, path } = oraclePropertyOptions(runs, PENDING_HISTORY) await runCampaign( 'pending', histories, (history, receipt) => run(history, receipt.reach), - { - runs: - process.env.TANSTACK_INDEXEDDB_ORACLE_PROFILE === 'stress' ? 300 : 30, - seed, - path: replayPath, - }, + { runs: numRuns, seed, path }, ) - }, 120_000) + }, + 120_000, +) diff --git a/packages/db/tests/indexed-db/persistence-oracle.test.ts b/packages/db/tests/indexed-db/persistence-oracle.test.ts index 9b59fe8c47..4b30aaf855 100644 --- a/packages/db/tests/indexed-db/persistence-oracle.test.ts +++ b/packages/db/tests/indexed-db/persistence-oracle.test.ts @@ -23,6 +23,12 @@ import { fc } from '@fast-check/vitest' import { describe, expect, it } from 'vitest' import { createTransaction } from '../../src' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, + recordOracleReplay, +} from '../oracle-config' import { Channel, assertRows, @@ -302,50 +308,44 @@ describe('settled histories', () => { expect(sorted(expected)).toEqual(expected) }) - const seedText = process.env.INDEXEDDB_ORACLE_SEED - const path = process.env.INDEXEDDB_ORACLE_PATH - if ((seedText === undefined) !== (path === undefined)) - throw new Error( - 'Replay requires INDEXEDDB_ORACLE_SEED and INDEXEDDB_ORACLE_PATH together', - ) - if ( - seedText !== undefined && - (!/^-?\d+$/.test(seedText) || !/^\d+(?::\d+)*$/.test(path!)) - ) - throw new Error('Invalid oracle replay coordinates') - const campaigns = - seedText === undefined - ? [{ mode: 'fixed', seed: 1179001 }, { mode: 'random' }] - : [{ mode: 'replay', seed: Number(seedText), path }] - for (const { mode, ...parameters } of campaigns) { - it( - 'matches authored data through ' + mode + ' histories', - async () => { - console.info('indexeddb.persistence campaign', mode, { - numRuns: 30, - ...parameters, - }) - await fc.assert(fc.asyncProperty(history, runHistory), { - numRuns: 30, - verbose: true, - ...parameters, - asyncReporter: async (details) => { - console.info('indexeddb.persistence completed', { - mode, - seed: details.seed, - path: details.counterexamplePath, - runs: details.numRuns, - failed: details.failed, - }) - const report = await fc.asyncDefaultReportMessage(details) - if (report !== undefined) - throw new Error(report, { cause: details.errorInstance }) - }, - }) - }, - 30_000, - ) + // A fixed campaign and a random campaign share one grammar and budget. A path + // replay of this property selects only its random campaign. + const settledProperty = `indexeddb.settled-history` + const settledCampaign = ( + mode: `fixed` | `random`, + parameters: { numRuns: number; seed?: number; path?: string }, + ) => { + return async () => { + console.info('indexeddb.persistence campaign', mode, parameters) + await fc.assert(fc.asyncProperty(history, runHistory), { + verbose: true, + ...parameters, + asyncReporter: async (details) => { + recordOracleReplay(settledProperty, details) + console.info('indexeddb.persistence completed', { + mode, + seed: details.seed, + path: details.counterexamplePath, + runs: details.numRuns, + failed: details.failed, + }) + const report = await fc.asyncDefaultReportMessage(details) + if (report !== undefined) + throw new Error(report, { cause: details.errorInstance }) + }, + }) + } } + oracleReplayTest(it)( + 'matches authored data through fixed histories', + settledCampaign(`fixed`, { numRuns: oracleRuns(30), seed: 1179001 }), + 30_000, + ) + oracleReplayTest(it, settledProperty)( + 'matches authored data through random histories', + settledCampaign(`random`, oraclePropertyOptions(30, settledProperty)), + 30_000, + ) }) /** Three distinct Collection identities receive one mixed manual payload. diff --git a/packages/db/tests/local-storage-value-fidelity-oracle.property.test.ts b/packages/db/tests/local-storage-value-fidelity-oracle.property.test.ts index 90c78df154..466fc51910 100644 --- a/packages/db/tests/local-storage-value-fidelity-oracle.property.test.ts +++ b/packages/db/tests/local-storage-value-fidelity-oracle.property.test.ts @@ -34,8 +34,9 @@ import { localStorageCollectionOptions } from '../src/local-storage' import { createTransaction } from '../src/transactions' import { oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from './oracle-config.js' import { observeHistoryPromise, @@ -43,7 +44,6 @@ import { } from './optimistic-history-oracle' const property = `local-storage.value-fidelity` -const requestedReplayProperty = readOracleRunConfig().replayProperty // --------------------------------------------------------------------------- // Model. Each generated value carries the JSON form a restore must return, or @@ -373,7 +373,7 @@ async function runHistory({ entry, handler, payload }: History) { } describe(`LocalStorage value fidelity oracle`, () => { - if (requestedReplayProperty === undefined) { + if (!oracleReplayRequested()) { // Pinned witnesses. The first is the reported case: a function property in // an inserted row. The others place each lost kind under an array or an // object, and check each documented normalization on every entry path. @@ -441,16 +441,10 @@ describe(`LocalStorage value fidelity oracle`, () => { seed: 20_261_021, numRuns: oracleRuns(300), })(`matches the JSON value model (fixed)`, runHistory) - fcTest.prop([historyArbitrary], oraclePropertyOptions(300, property))( - `matches the JSON value model (random)`, - runHistory, - ) - } else if (requestedReplayProperty === property) { - fcTest.prop([historyArbitrary], oraclePropertyOptions(300, property))( - `matches the JSON value model (replay)`, - runHistory, - ) - } else { - it.skip(`runs only when its replay property is selected`, () => {}) } + // A path replay of this property runs its one recorded history here. + oracleReplayTest(fcTest, property).prop( + [historyArbitrary], + oraclePropertyOptions(300, property), + )(`matches the JSON value model (random)`, runHistory) }) diff --git a/packages/db/tests/notification-laws-oracle.test.ts b/packages/db/tests/notification-laws-oracle.test.ts index 606fc8240f..f09ec97a4d 100644 --- a/packages/db/tests/notification-laws-oracle.test.ts +++ b/packages/db/tests/notification-laws-oracle.test.ts @@ -23,6 +23,7 @@ import { expectNotificationsStopped, } from '../../db-collection-e2e/src/utils/notification-laws-oracle' import { ScenarioLifetime } from './conformance/scenario-lifetime' +import { oracleRuns } from './oracle-config' import type { Notification } from '../../db-collection-e2e/src/utils/notification-laws-oracle' type Row = { id: string; count: number } @@ -110,7 +111,7 @@ it.each([20260913, undefined])( ), { seed, - numRuns: 30, + numRuns: oracleRuns(30), examples: [ [1, [2, null]], [1, [null]], @@ -176,7 +177,7 @@ it.each([20260913, undefined])( ).toThrow() }, ), - { seed, numRuns: 100 }, + { seed, numRuns: oracleRuns(100) }, ) }, ) @@ -250,7 +251,7 @@ it.each([20260913, undefined])( expectNotificationHistory([], [...callbacks, callbacks[1]!], [next]), ).toThrow() }), - { seed, numRuns: 50 }, + { seed, numRuns: oracleRuns(50) }, ) }, ) diff --git a/packages/db/tests/oracle-config.ts b/packages/db/tests/oracle-config.ts index 46e6efb6b2..e7b7adf5ad 100644 --- a/packages/db/tests/oracle-config.ts +++ b/packages/db/tests/oracle-config.ts @@ -1,5 +1,7 @@ import { oracleReplayReporter } from './oracle-replay-witness.js' +export { recordOracleReplay } from './oracle-replay-witness.js' + type OracleEnvironment = Record const staticOracleProperties = [ @@ -25,6 +27,80 @@ const staticOracleProperties = [ `structural-identity.equal-representations`, `structural-identity.independent-values`, `opfs-worker.diagnostics`, + `indexeddb.cross-tab-history`, + `indexeddb.browser-native-history`, + `proxy.callback-history`, + `indexeddb.pending-history`, + `proxy.revert-history`, + `proxy.revert-nested-round-trips`, + `proxy.revert-class-writes`, + `proxy.revert-partial-reverts`, + `proxy.revert-native-histories`, + `proxy.native-array-methods`, + `proxy.native-typed-methods`, + `ivm.hash-identity`, + `ivm.hash-identity-cycles`, + `ivm.multiset-consolidate`, + `electric.adapter-failure-policy`, + `electric.design-convergence`, + `electric.durable-prefix`, + `electric.initial-tagged-move-outs`, + `electric.invalid-resume-partition`, + `electric.metadata-merge-complete-state`, + `electric.process-grammar`, + `electric.publication-before-persistence`, + `electric.queued-presence-order`, + `electric.resume-metadata-merge`, + `electric.retry-decision`, + `electric.subset-request-row-validity`, + `electric.synthetic-batch-partition`, + `electric.valid-batch-partition`, + `ivm.index-refinement`, + `offline.key-scheduler-fifo-ledger`, + `offline.interruption-settlement`, + `offline.fifo-retry`, + `offline.leadership.serial-work`, + `offline.leadership.report`, + `offline.leadership.delayed-read`, + `offline.leadership.startup`, + `offline.leadership.stale-read`, + `offline.leadership.storage-failure`, + `offline.serializer.mutation-wire`, + `offline.serializer.date-marker`, + `offline.serializer.version-two`, + `offline.settlement.transaction`, + `offline.settlement.retry-record`, + `offline.settlement.admission`, + `offline.settlement.retry-decision`, + `op-sqlite.columnar-sql-aliases`, + `op-sqlite.result-carrier-equivalence`, + `op-sqlite.write-marker-aliases`, + `pg-serializer.array-determinism`, + `pg-serializer.bigint-round-trip`, + `pg-serializer.bigint-text`, + `pg-serializer.boolean-array-contents`, + `pg-serializer.boolean-round-trip`, + `pg-serializer.boolean-tokens`, + `pg-serializer.date-iso`, + `pg-serializer.date-round-trip`, + `pg-serializer.double-round-trip`, + `pg-serializer.integer-array-contents`, + `pg-serializer.integer-array-format`, + `pg-serializer.integer-round-trip`, + `pg-serializer.integer-text`, + `pg-serializer.mixed-array-contents`, + `pg-serializer.nullish-array-contents`, + `pg-serializer.nullish-empty`, + `pg-serializer.scalar-determinism`, + `pg-serializer.string-array-contents`, + `pg-serializer.string-escaping`, + `pg-serializer.string-idempotent`, + `pg-serializer.string-passthrough`, + `sqlite-persistence.bigint-rejection`, + `sqlite-persistence.bigint-roundtrip`, + `sqlite-persistence.binding-capacity`, + `sqlite-persistence.expression-index`, + `sqlite-resume.baseline-lineage`, `sqlite-persistence.source-fifo-order`, `sqlite-persistence.abort-graph`, `sqlite-persistence.owner-isolation`, @@ -55,7 +131,6 @@ const staticOracleProperties = [ `collection-state.accepted-snapshot.before-delete`, `collection-state.accepted-snapshot.during-delete`, `collection-state.accepted-snapshot.after-rollback`, - `collection-state.queued-update-dependency`, `collection-state.change-event-history`, `collection-state.eager-index-history`, `collection-state.optimistic-history`, @@ -85,9 +160,6 @@ const staticOracleProperties = [ `derived-publication.membership-work`, `collection-publication.metadata-cancellation`, `collection-publication.metadata-only`, - `collection-publication.metadata-rollback`, - `coverage-registry.claim-churn`, - `coverage-registry.state-machine`, `d2-source.exact-retractions`, `d2-source.changed-restart`, `d2-source.disjoint-commutation`, @@ -135,32 +207,9 @@ const staticOracleProperties = [ `includes.nested-scalar-materialization`, `includes.optimistic-convergence`, `includes.scenario-statistics`, - `load-subset-full-flow.atomic-replacement`, - `load-subset-full-flow.automatic-progress`, - `load-subset-full-flow.boundary-provenance`, - `load-subset-full-flow.consumer-parity`, - `load-subset-full-flow.continuation-evidence`, - `load-subset-full-flow.continuation-statistics`, - `load-subset-full-flow.multi-source-ordered`, - `load-subset-full-flow.multi-source-statistics`, - `load-subset-full-flow.truncate-evidence`, - `load-subset-lifecycle.state-machine`, - `load-subset.async-settlement`, - `load-subset.changing-predicate`, - `load-subset.concurrent-dedupe`, - `load-subset.coverage`, - `load-subset.distinct-window-predicate`, `load-subset.exact-completion`, `load-subset.exact-inflight`, - `load-subset.ordered-window`, `load-subset.rejected-waiter`, - `ordered-work.forward-exhaustion`, - `ordered-work.forward-prefix`, - `ordered-work.custom-comparator-fallback`, - `ordered-work.public-key-suffix`, - `ordered-work.reverse-exhaustion`, - `ordered-work.reverse-prefix`, - `ordered-work.snapshot-reuse`, `ordered-work.consumer-parity`, `ordered-work.eager-indexed-window`, `ordered-work.eager-joined-window`, @@ -178,12 +227,8 @@ const staticOracleProperties = [ `pagination.pending-mutation`, `pagination.window-transition`, `persistence.retained-demand`, - `predicate-subtraction.duplicate-terms`, - `predicate-subtraction.finite-world`, - `predicate-subtraction.unbounded`, `subscription-replay.completion`, `subscription-replay.optimistic`, - `subscription-replay.ownership`, `subscription-replay.restart`, `subscription-replay.sequential`, `subscription-replay.shared`, @@ -198,6 +243,22 @@ const staticOracleProperties = [ `cleanup-queue.history`, `sorted-map.ascending`, `sorted-map.descending`, + `btree-map.history`, + `evaluators.like`, + `indexeddb.settled-history`, + `ivm.hash-failure-retry`, + `ivm.hash-graph`, + `ivm.hash-mixed-graph`, + `ivm.hash-value`, + `ivm.incrementalization-law`, + `ivm.topk-support-window`, + `opfs.close`, + `opfs.failure-cleanup`, + `opfs.mixed-execute`, + `opfs.page-lifecycle`, + `opfs.request-order`, + `opfs.terminal-event`, + `sqlite-persistence.boolean-arity`, ] as const const publicationProperties = [ @@ -212,9 +273,14 @@ const publicationProperties = [ ), ) -const refinementProperties = Array.from( - { length: 11 }, - (_, index) => `load-subset-refinement.${1_779_001 + index}`, +// utils-oracle and comparison-oracle number their properties in file order. +const utilsProperties = Array.from( + { length: 40 }, + (_, index) => `utils.${index + 1}`, +) +const comparisonProperties = Array.from( + { length: 39 }, + (_, index) => `comparison.${index + 1}`, ) const paginationMatrixProperties = [`window`, `state`].flatMap((family) => Array.from( @@ -301,7 +367,8 @@ export function validateOraclePropertyRegistry( export const registeredOracleProperties = validateOraclePropertyRegistry([ ...staticOracleProperties, ...publicationProperties, - ...refinementProperties, + ...utilsProperties, + ...comparisonProperties, ...paginationMatrixProperties, ...includesMatrixProperties, ]) @@ -318,9 +385,121 @@ export type OracleReplayConfig = { replayProperty: string | undefined } +/** + * Replay variables that oracles read before they moved to the shared + * TANSTACK_DB_ORACLE_* variables. Nothing reads them now, so a command that + * still sets one would run the full campaigns and appear to pass. Reading the + * config rejects them instead. + */ +const retiredReplayVariables = [ + `UTILS_ORACLE_PROPERTY`, + `UTILS_ORACLE_SEED`, + `UTILS_ORACLE_PATH`, + `COMPARISON_ORACLE_PROPERTY`, + `COMPARISON_ORACLE_SEED`, + `COMPARISON_ORACLE_PATH`, + `PG_SERIALIZER_ORACLE_PROPERTY`, + `PG_SERIALIZER_ORACLE_SEED`, + `PG_SERIALIZER_ORACLE_PATH`, + `PG_SERIALIZER_ORACLE_RUNS`, + `TANSTACK_DB_ELECTRIC_ORACLE_PROPERTY`, + `TANSTACK_DB_ELECTRIC_HISTORY_PROPERTY`, + `TANSTACK_DB_ELECTRIC_HISTORY_SEED`, + `TANSTACK_DB_ELECTRIC_HISTORY_PATH`, + `TANSTACK_DB_ELECTRIC_DURABILITY_SEED`, + `TANSTACK_DB_ELECTRIC_DURABILITY_PATH`, + `TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_SEED`, + `TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_PATH`, + `TANSTACK_DB_IVM_INDEX_SEED`, + `TANSTACK_DB_IVM_INDEX_PATH`, + `TANSTACK_DB_SQLITE_BINDING_SEED`, + `TANSTACK_DB_SQLITE_BINDING_PATH`, + `TANSTACK_DB_SQLITE_ORACLE_SEED`, + `TANSTACK_DB_SQLITE_ORACLE_PATH`, + `TANSTACK_DB_SQLITE_ORACLE_RUNS`, + `TANSTACK_DB_WS5A_PROPERTY`, + `TANSTACK_DB_WS5A_SEED`, + `TANSTACK_DB_WS5A_PATH`, + `TANSTACK_INDEXEDDB_ORACLE_SEED`, + `TANSTACK_INDEXEDDB_ORACLE_PATH`, + `TANSTACK_INDEXEDDB_BROWSER_SEED`, + `TANSTACK_INDEXEDDB_BROWSER_PATH`, + `TANSTACK_DB_PROXY_CALLBACK_SEED`, + `TANSTACK_DB_PROXY_CALLBACK_PATH`, + `TANSTACK_DB_PROXY_REVERT_SEED`, + `TANSTACK_DB_PROXY_REVERT_PATH`, + `TANSTACK_DB_PROXY_NATIVE_SEED`, + `TANSTACK_DB_PROXY_NATIVE_PATH`, + `TANSTACK_DB_IVM_HASH_IDENTITY_SEED`, + `TANSTACK_DB_IVM_HASH_IDENTITY_PATH`, + `TANSTACK_DB_IVM_CONSOLIDATE_SEED`, + `TANSTACK_DB_IVM_CONSOLIDATE_PATH`, + `TANSTACK_INDEXEDDB_PENDING_SEED`, + `TANSTACK_INDEXEDDB_PENDING_PATH`, + `TANSTACK_DB_OFFLINE_ORACLE_SEED`, + `TANSTACK_DB_OFFLINE_ORACLE_PATH`, + `TANSTACK_DB_OFFLINE_ORACLE_RUNS`, + `OFFLINE_ORACLE_SEED`, + `OFFLINE_ORACLE_PATH`, + `OFFLINE_ORACLE_RUNS`, + `TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_SEED`, + `TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_RUNS`, + `TANSTACK_DB_OPFS_LIFECYCLE_ORACLE_PATH`, + `TANSTACK_DB_OPFS_MIXED_ORACLE_SEED`, + `TANSTACK_DB_OPFS_MIXED_ORACLE_RUNS`, + `TANSTACK_DB_OPFS_MIXED_ORACLE_PATH`, + `TANSTACK_DB_OPFS_FAILURE_ORACLE_SEED`, + `TANSTACK_DB_OPFS_FAILURE_ORACLE_RUNS`, + `TANSTACK_DB_OPFS_FAILURE_ORACLE_PATH`, + `TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_SEED`, + `TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_RUNS`, + `TANSTACK_DB_OPFS_TERMINAL_EVENT_ORACLE_PATH`, + `TANSTACK_DB_OPFS_CLOSE_ORACLE_SEED`, + `TANSTACK_DB_OPFS_CLOSE_ORACLE_RUNS`, + `TANSTACK_DB_OPFS_CLOSE_ORACLE_PATH`, + `TANSTACK_DB_OPFS_REQUEST_ORDER_ORACLE_SEED`, + `TANSTACK_DB_OPFS_REQUEST_ORDER_ORACLE_RUNS`, + `TANSTACK_DB_OPFS_REQUEST_ORDER_ORACLE_PATH`, + `TANSTACK_DB_IVM_HASH_RETRY_SEED`, + `TANSTACK_DB_IVM_HASH_RETRY_PATH`, + `TANSTACK_DB_IVM_HASH_GRAPH_SEED`, + `TANSTACK_DB_IVM_HASH_GRAPH_PATH`, + `TANSTACK_DB_IVM_MIXED_GRAPH_SEED`, + `TANSTACK_DB_IVM_MIXED_GRAPH_PATH`, + `TANSTACK_DB_IVM_MIXED_GRAPH_FAULT`, + `TANSTACK_DB_IVM_HASH_SEED`, + `TANSTACK_DB_IVM_HASH_PATH`, + `TANSTACK_DB_IVM_ORACLE_SEED`, + `TANSTACK_DB_IVM_ORACLE_RUNS`, + `TANSTACK_DB_IVM_ORACLE_PATH`, + `TANSTACK_DB_IVM_TOPK_SUPPORT_SEED`, + `TANSTACK_DB_IVM_TOPK_SUPPORT_PATH`, + `TANSTACK_DB_IVM_TOPK_SUPPORT_RUNS`, + `TANSTACK_DB_SQLITE_BOOLEAN_ORACLE_RUNS`, + `TANSTACK_DB_SQLITE_BOOLEAN_ORACLE_SEED`, + `TANSTACK_DB_SQLITE_BOOLEAN_ORACLE_PATH`, + `BTREE_MAP_ORACLE_SEED`, + `BTREE_MAP_ORACLE_PATH`, + `INDEXEDDB_ORACLE_SEED`, + `INDEXEDDB_ORACLE_PATH`, + `TANSTACK_DB_LIKE_ORACLE_SEED`, + `TANSTACK_DB_LIKE_ORACLE_RUNS`, +] as const + export function readOracleRunConfig( environment: OracleEnvironment = process.env, ): OracleReplayConfig & { multiplier: number } { + const retired = retiredReplayVariables.filter( + (variable) => environment[variable] !== undefined, + ) + if (retired.length > 0) { + throw new Error( + `Retired oracle replay variables are set: ${retired.join(`, `)}. ` + + `Use TANSTACK_DB_ORACLE_SEED, TANSTACK_DB_ORACLE_PATH, and ` + + `TANSTACK_DB_ORACLE_PROPERTY to select a replay, and ` + + `TANSTACK_DB_ORACLE_RUNS_MULTIPLIER to scale run counts.`, + ) + } const multiplierValue = environment.TANSTACK_DB_ORACLE_RUNS_MULTIPLIER ?? `1` const multiplier = Number(multiplierValue) if ( @@ -427,17 +606,75 @@ export function oracleRandomParameters( const { multiplier, ...replay } = readOracleRunConfig() /** - * Whether a path replay is selected. It replays one property's recorded - * history, so a test that is not that property stands aside with - * `it.skipIf(oracleReplayRequested())`. A seed without a path only reseeds - * the random campaigns, so every test still runs. The `oracle/no-disabled` - * lint rule accepts only this condition, so a replay selection cannot switch - * an oracle test off in ordinary runs. + * Replay selection. + * + * A path replay runs one property's recorded history. Every other test + * stands aside, so the replay reports only that history. A seed without a + * path only reseeds the random campaigns, so every test still runs. These + * helpers are the only replay conditions that the `oracle/no-disabled` lint + * rule accepts. Each is false in an ordinary run, so no replay selection can + * switch an oracle test off there. */ + +/** Whether a path replay is selected. */ export function oracleReplayRequested(): boolean { return replay.replayPath !== undefined } +/** + * Whether a path replay of a property outside `properties` is selected. A + * test that serves several properties' replays passes all of them. + */ +export function oracleReplaySkips( + ...properties: ReadonlyArray +): boolean { + for (const property of properties) assertRegisteredOracleProperty(property) + return ( + replay.replayPath !== undefined && + (replay.replayProperty === undefined || + !properties.includes(replay.replayProperty)) + ) +} + +/** + * The registration to use for a test that stands aside during another + * test's replay: its `skip` form while that replay runs, and itself + * otherwise. Without `properties`, the test stands aside during every path + * replay. With them, it runs only when no path replay is selected or the + * replay selects one of them. + */ +export function oracleReplayTest( + registration: TRegistration, + ...properties: ReadonlyArray +): TRegistration { + const standsAside = + properties.length === 0 + ? oracleReplayRequested() + : oracleReplaySkips(...properties) + // `skip` registers through the same API, so it has the registration's type. + return standsAside ? (registration.skip as TRegistration) : registration +} + +/** + * The registration for a calibration witness. During a path replay of + * `property`, TANSTACK_DB_ORACLE_CALIBRATION selects a hostile mutant that the + * main property runs against, and the witness for that `fault` checks the + * failure. The witness runs in ordinary runs and in that calibration replay. + * It stands aside during every other path replay, where it has nothing to + * check. + */ +export function oracleCalibrationTest( + registration: TRegistration, + property: string, + fault: string, +): TRegistration { + const standsAside = + oracleReplaySkips(property) || + (oracleReplayRequested() && + process.env.TANSTACK_DB_ORACLE_CALIBRATION !== fault) + return standsAside ? (registration.skip as TRegistration) : registration +} + /** Keeps ordinary CI bounded while allowing long randomized oracle campaigns. */ export function oracleRuns(baseRuns: number): number { return baseRuns * multiplier diff --git a/packages/db/tests/oracle-replay-manifest.ts b/packages/db/tests/oracle-replay-manifest.ts index 7bcf5f9337..75893eda53 100644 --- a/packages/db/tests/oracle-replay-manifest.ts +++ b/packages/db/tests/oracle-replay-manifest.ts @@ -7,7 +7,11 @@ import { registeredOracleProperties } from './oracle-config.js' * separate campaign gates. An absent or statistics-only entry must not be * promoted to assertion evidence merely because a process exited cleanly. */ -const ownerGroups: ReadonlyArray = [ +// Each group is [owner file, prefix, suffixes] and, when the owner runs the +// properties from another module, the module that names them. +const ownerGroups: ReadonlyArray< + readonly [string, string, string] | readonly [string, string, string, string] +> = [ [ `db/tests/cleanup-queue-oracle.property.test.ts`, `cleanup-queue`, @@ -18,7 +22,11 @@ const ownerGroups: ReadonlyArray = [ `sorted-map`, `key ascending descending`, ], - [`db/tests/oracle-replay.fixture.test.ts`, `oracle-replay`, `calibration`], + [ + `db/tests/replay-guard-calibration.fixture.test.ts`, + `oracle-replay`, + `calibration`, + ], [ `db-ivm/tests/structural-identity-oracle.property.test.ts`, `structural-identity`, @@ -209,15 +217,314 @@ const ownerGroups: ReadonlyArray = [ `persistence`, `retained-demand`, ], + // Owners of properties that a module names and a host file runs carry + // that module as the fourth element. + [ + `query-db-collection/tests/cursor-pagination.no-peek.test.ts`, + `cursor-pagination`, + `no-peek`, + ], + [ + `query-db-collection/tests/cursor-pagination.oracle.test.ts`, + `cursor-pagination`, + `history backend-ownership partition reset failure`, + ], + [ + `query-db-collection/tests/cursor-pagination.cache-oracle.test.ts`, + `cursor-pagination`, + `cache defaults cancellation`, + ], + [ + `query-db-collection/tests/cursor-pagination.boundary-oracle.test.ts`, + `cursor-pagination`, + `nested-cancellation reader-abort`, + ], + [ + `query-db-collection/tests/cursor-pagination-oracle.integration.test.ts`, + `cursor-pagination`, + `manual-write`, + ], + [ + `query-db-collection/tests/cursor-pagination.publication-oracle.test.ts`, + `cursor-pagination`, + `refresh-publication protocol-publication slice-work`, + ], + [ + `query-db-collection/tests/refetch-ordered-prefix-oracle.test.ts`, + `query-db`, + `refetch-ordered-prefix`, + ], + [ + `db/tests/query/ordered-source-loader-state-oracle.test.ts`, + `ordered-loader`, + `request-authority`, + ], + [ + `db/tests/query/load-subset-replay-refinement-oracle.test.ts`, + `replay`, + `ordered-window-faults`, + ], + [ + `db-sqlite-persistence-core/tests/sqlite-core-adapter-cli-runtime.test.ts`, + `sqlite-persistence`, + `binding-capacity`, + `db-sqlite-persistence-core/tests/sqlite-core-adapter-oracle.test.ts`, + ], + [ + `db-sqlite-persistence-core/tests/sqlite-core-adapter-cli-runtime.test.ts`, + `sqlite-resume`, + `baseline-lineage`, + `db-sqlite-persistence-core/tests/sqlite-core-adapter-oracle.test.ts`, + ], + [ + `db-sqlite-persistence-core/tests/sqlite-resume-snapshot-oracle.test.ts`, + `sqlite-ordinary`, + `write-history independent-work`, + `db-sqlite-persistence-core/tests/ordinary-transaction-work-oracle.ts`, + ], + [ + `db-sqlite-persistence-core/tests/persisted-oracle.test.ts`, + `sqlite-persistence`, + `request-local-failure-buffered-source`, + ], + [ + `db-sqlite-persistence-core/tests/persisted-oracle.test.ts`, + `persistence`, + `generation-notice follower-remote-demand`, + ], + [ + `db-sqlite-persistence-core/tests/persisted-optimistic-history.test.ts`, + `sqlite-persistence`, + `optimistic-history`, + ], + [ + `db-sqlite-persistence-core/tests/sqlite-resume-snapshot-oracle.test.ts`, + `sqlite-resume`, + `managed-claim-read retired-storage`, + ], + [ + `trailbase-db-collection/tests/lifecycle-oracle.property.test.ts`, + `trailbase`, + `page-event`, + ], + [ + `browser-db-sqlite-persistence/tests/reload-class-oracle.test.ts`, + `sqlite-reload`, + `class`, + ], + [ + `electron-db-sqlite-persistence/tests/reload-class-oracle.test.ts`, + `sqlite-reload`, + `class.electron`, + ], + [ + `db/tests/query/where-predicate-publication-oracle.property.test.ts`, + `where-predicate`, + `publication`, + ], + [ + `db/tests/query/join-result-key-oracle.property.test.ts`, + `join-result-key`, + `pairs`, + ], + [ + `db/tests/query/pooled-live-query-oracle.property.test.ts`, + `pooled-live-query`, + `publication`, + ], + [ + `db/tests/flat-change-tracking-oracle.property.test.ts`, + `flat-change-tracking`, + `equivalence`, + ], + [ + `db/tests/local-storage-value-fidelity-oracle.property.test.ts`, + `local-storage`, + `value-fidelity`, + ], + [ + `db/tests/live-query-observer-snapshot-oracle.property.test.ts`, + `live-query-observer`, + `retained-snapshot`, + ], + [ + `db/tests/query/bucket-facade-rollback-oracle.property.test.ts`, + `bucket-facade`, + `rollback-history`, + ], + [ + `db/tests/query/aggregate-select-oracle.property.test.ts`, + `aggregate`, + `select-shapes`, + ], + [ + `db/tests/query/compiler/in-evaluator-oracle.property.test.ts`, + `evaluators`, + `in`, + ], + [ + `db/tests/transaction-ownership-oracle.property.test.ts`, + `transaction-ownership`, + `settlement-release`, + ], + [ + `db/tests/query/includes-work-counter-oracle.test.ts`, + `includes-work`, + `ordered-window`, + ], + [`db/tests/reverse-index-oracle.property.test.ts`, `reverse-index`, `reads`], + // Owners of the replays that moved to the shared variables. + [ + `browser-db-sqlite-persistence/tests/opfs-worker-diagnostics-oracle.test.ts`, + `opfs-worker`, + `diagnostics`, + ], + [ + `db/tests/indexed-db/cross-tab-history-oracle.test.ts`, + `indexeddb`, + `cross-tab-history`, + ], + [`db/tests/proxy-oracle.test.ts`, `proxy`, `callback-history`], + [ + `db/e2e/indexed-db/cross-tab-oracle.spec.ts`, + `indexeddb`, + `browser-native-history`, + ], + [ + `db/tests/indexed-db/pending-history-oracle.test.ts`, + `indexeddb`, + `pending-history`, + ], + [ + `db/tests/proxy-revert-oracle.property.test.ts`, + `proxy`, + `revert-history revert-nested-round-trips revert-class-writes revert-partial-reverts revert-native-histories`, + ], + [ + `db/tests/proxy-native-methods-oracle.property.test.ts`, + `proxy`, + `native-array-methods native-typed-methods`, + ], + [ + `db-ivm/tests/hash-identity-oracle.property.test.ts`, + `ivm`, + `hash-identity hash-identity-cycles`, + ], + [ + `db-ivm/tests/multiset-consolidate-oracle.property.test.ts`, + `ivm`, + `multiset-consolidate`, + ], + [ + `electric-db-collection/tests/electric-oracle.property.test.ts`, + `electric`, + `adapter-failure-policy design-convergence durable-prefix initial-tagged-move-outs invalid-resume-partition metadata-merge-complete-state process-grammar publication-before-persistence queued-presence-order resume-metadata-merge retry-decision subset-request-row-validity synthetic-batch-partition valid-batch-partition`, + ], + [ + `electric-db-collection/tests/pg-serializer-oracle.property.test.ts`, + `pg-serializer`, + `array-determinism bigint-round-trip bigint-text boolean-array-contents boolean-round-trip boolean-tokens date-iso date-round-trip double-round-trip integer-array-contents integer-array-format integer-round-trip integer-text mixed-array-contents nullish-array-contents nullish-empty scalar-determinism string-array-contents string-escaping string-idempotent string-passthrough`, + ], + [ + `db-ivm/tests/index-refinement-oracle.property.test.ts`, + `ivm`, + `index-refinement`, + ], + [ + `offline-transactions/tests/KeyScheduler-oracle.property.test.ts`, + `offline`, + `key-scheduler-fifo-ledger`, + ], + [ + `offline-transactions/tests/interruption-settlement-oracle.property.test.ts`, + `offline`, + `interruption-settlement`, + ], + [ + `offline-transactions/tests/fifo-retry-oracle.property.test.ts`, + `offline`, + `fifo-retry`, + ], + [ + `offline-transactions/tests/leadership-replay-oracle.property.test.ts`, + `offline`, + `leadership.serial-work leadership.report leadership.delayed-read leadership.startup leadership.stale-read leadership.storage-failure`, + ], + [ + `offline-transactions/tests/transaction-serializer-oracle.property.test.ts`, + `offline`, + `serializer.mutation-wire serializer.date-marker serializer.version-two`, + ], + [ + `offline-transactions/tests/transaction-settlement-oracle.property.test.ts`, + `offline`, + `settlement.transaction settlement.retry-record settlement.admission settlement.retry-decision`, + ], + [ + `react-native-db-sqlite-persistence/tests/op-sqlite-driver-oracle.test.ts`, + `op-sqlite`, + `columnar-sql-aliases result-carrier-equivalence write-marker-aliases`, + ], + [ + `node-db-sqlite-persistence/tests/expression-index-oracle.test.ts`, + `sqlite-persistence`, + `expression-index bigint-roundtrip bigint-rejection`, + ], + [ + `browser-db-sqlite-persistence/tests/opfs-page-lifecycle-oracle.test.ts`, + `opfs`, + `page-lifecycle mixed-execute terminal-event close failure-cleanup`, + ], + [ + `browser-db-sqlite-persistence/tests/opfs-request-order-oracle.test.ts`, + `opfs`, + `request-order`, + ], + [ + `db-ivm/tests/hash-failure-retry-oracle.property.test.ts`, + `ivm`, + `hash-failure-retry`, + ], + [`db-ivm/tests/hash-graph-oracle.property.test.ts`, `ivm`, `hash-graph`], + [ + `db-ivm/tests/hash-mixed-graph-oracle.property.test.ts`, + `ivm`, + `hash-mixed-graph`, + ], + [`db-ivm/tests/hash-oracle.property.test.ts`, `ivm`, `hash-value`], + [ + `db-ivm/tests/incrementalization-law-oracle.property.test.ts`, + `ivm`, + `incrementalization-law`, + ], + [ + `db-ivm/tests/operators/topk-support-window-oracle.test.ts`, + `ivm`, + `topk-support-window`, + ], + [ + `db-sqlite-persistence-core/tests/sqlite-boolean-arity-oracle.test.ts`, + `sqlite-persistence`, + `boolean-arity`, + ], + [`db/tests/btree-map-oracle.test.ts`, `btree-map`, `history`], + [ + `db/tests/indexed-db/persistence-oracle.test.ts`, + `indexeddb`, + `settled-history`, + ], + [`db/tests/query/compiler/evaluators-oracle.test.ts`, `evaluators`, `like`], ] const owners = new Map() -for (const [file, prefix, suffixes] of ownerGroups) { +const sources = new Map() +for (const [file, prefix, suffixes, source] of ownerGroups) { for (const suffix of suffixes.split(` `)) { const property = `${prefix}.${suffix}` if (owners.has(property)) throw new Error(`duplicate replay owner: ${property}`) owners.set(property, `packages/${file}`) + if (source !== undefined) sources.set(property, `packages/${source}`) } } @@ -245,6 +552,12 @@ for (const property of registeredOracleProperties) { `packages/db/tests/query/includes-oracle.property.test.ts`, ) } + if (property.startsWith(`utils.`)) { + owners.set(property, `packages/db/tests/utils-oracle.property.test.ts`) + } + if (property.startsWith(`comparison.`)) { + owners.set(property, `packages/db/tests/comparison-oracle.property.test.ts`) + } if (property.startsWith(`pagination.matrix.`)) { owners.set( property, @@ -255,8 +568,23 @@ for (const property of registeredOracleProperties) { export type OracleReplayManifestEntry = { property: string - status: `assertion` | `statistics-only` | `no-named-owner` + /** + * `browser-replay` names a Playwright owner. The guarded Vitest runner + * cannot witness it, so it rejects the target. Replay it with `command`. + * `seed-replay` names a property whose generator has no shrink path. A + * seed alone reproduces its history: `command` runs the owner directly + * with TANSTACK_DB_ORACLE_SEED, which reseeds its random campaign. The + * guarded runner, which witnesses a seed and path, rejects it. + */ + status: + | `assertion` + | `statistics-only` + | `browser-replay` + | `seed-replay` + | `no-named-owner` file?: string + /** The module that names the property, when the owner file imports it. */ + source?: string cwd?: string command?: string testNamePattern?: string @@ -293,6 +621,10 @@ const replayTestNames = new Map([ ], ]) +// Properties whose own seeded generator replaces fast-check, so they have no +// shrink path. +const seedReplayProperties = new Set([`evaluators.like`]) + function directReplayTestName(property: string): string | undefined { if (property === `cold-join.compound`) return `recomputes generated compound histories, seed=undefined` @@ -310,31 +642,42 @@ export const oracleReplayManifest: ReadonlyArray = [ const status = file === undefined ? `no-named-owner` - : property.endsWith(`-statistics`) || - property.endsWith(`.scenario-statistics`) - ? `statistics-only` - : `assertion` + : file.endsWith(`.spec.ts`) + ? `browser-replay` + : seedReplayProperties.has(property) + ? `seed-replay` + : property.endsWith(`-statistics`) || + property.endsWith(`.scenario-statistics`) + ? `statistics-only` + : `assertion` const cwd = file?.split(`/`).slice(0, 2).join(`/`) return { property, status, file, + ...(sources.has(property) ? { source: sources.get(property) } : {}), cwd, testNamePattern: directReplayTestName(property), command: file === undefined ? undefined - : `node --import tsx ${cwd === `packages/db` ? `tests` : `../db/tests`}/oracle-replay.ts ${file.split(`/`).slice(2).join(`/`)}`, + : status === `browser-replay` + ? `pnpm exec playwright test --config playwright.indexed-db.config.ts ${file.split(`/`).slice(2).join(`/`)}` + : status === `seed-replay` + ? `pnpm exec vitest run ${file.split(`/`).slice(2).join(`/`)}` + : `node --import tsx ${cwd === `packages/db` ? `tests` : `../db/tests`}/oracle-replay.ts ${file.split(`/`).slice(2).join(`/`)}`, environment: status === `statistics-only` ? [ `TANSTACK_DB_ORACLE_STATISTICS=1 (sampling only; no exact assertion replay)`, ] - : [ - `TANSTACK_DB_ORACLE_SEED`, - `TANSTACK_DB_ORACLE_PATH`, - `TANSTACK_DB_ORACLE_PROPERTY`, - ], + : status === `seed-replay` + ? [`TANSTACK_DB_ORACLE_SEED`] + : [ + `TANSTACK_DB_ORACLE_SEED`, + `TANSTACK_DB_ORACLE_PATH`, + `TANSTACK_DB_ORACLE_PROPERTY`, + ], } }) diff --git a/packages/db/tests/oracle-replay-witness.ts b/packages/db/tests/oracle-replay-witness.ts index 26c264e6a9..25a8a00a76 100644 --- a/packages/db/tests/oracle-replay-witness.ts +++ b/packages/db/tests/oracle-replay-witness.ts @@ -1,5 +1,5 @@ import { appendFileSync } from 'node:fs' -import { asyncDefaultReportMessage } from 'fast-check' +import { asyncDefaultReportMessage, defaultReportMessage } from 'fast-check' import type { RunDetails } from 'fast-check' export type OracleReplayWitness = { @@ -10,6 +10,53 @@ export type OracleReplayWitness = { failed: boolean } +function witnessChannel( + property: string, + seed: number, + path: string | undefined, +): string | undefined { + const channel = process.env.TANSTACK_DB_ORACLE_REPLAY_WITNESS + return channel !== undefined && + property === process.env.TANSTACK_DB_ORACLE_PROPERTY && + seed === Number(process.env.TANSTACK_DB_ORACLE_SEED) && + path === process.env.TANSTACK_DB_ORACLE_PATH + ? channel + : undefined +} + +function appendWitness( + channel: string, + property: string, + details: RunDetails, +): void { + const witness: OracleReplayWitness = { + property, + seed: details.seed, + path: details.runConfiguration.path, + numRuns: details.numRuns, + failed: details.failed, + } + appendFileSync(channel, `${JSON.stringify(witness)}\n`) +} + +/** + * Record a guarded replay witness for an owner that runs its property with + * `fc.check`. fast-check calls reporters only from `fc.assert`, so such an + * owner records its run details itself. It never throws: the owner keeps its + * own failure analysis. + */ +export function recordOracleReplay( + property: string, + details: RunDetails, +): void { + const channel = witnessChannel( + property, + details.seed, + details.runConfiguration.path, + ) + if (channel !== undefined) appendWitness(channel, property, details) +} + /** * Attach a witness only when the guarded runner's property, seed, and path all * match. Constructing fast-check options is not evidence that a property ran. @@ -23,42 +70,42 @@ export function oracleReplayReporter( path: string, ): { asyncReporter?: (details: RunDetails) => Promise + reporter?: (details: RunDetails) => void } { - const channel = process.env.TANSTACK_DB_ORACLE_REPLAY_WITNESS - if ( - channel === undefined || - property !== process.env.TANSTACK_DB_ORACLE_PROPERTY || - seed !== Number(process.env.TANSTACK_DB_ORACLE_SEED) || - path !== process.env.TANSTACK_DB_ORACLE_PATH - ) - return {} + const channel = witnessChannel(property, seed, path) + if (channel === undefined) return {} + const record = (details: RunDetails) => + appendWitness(channel, property, details) + const rethrow = (details: RunDetails, message: string | undefined) => { + // A custom reporter replaces fast-check's throw. Preserve its public + // formatter and errorWithCause policy, including precondition failures. + if (message === undefined) return + const error = new Error(message) + if (details.runConfiguration.errorWithCause) { + Object.defineProperty(error, `cause`, { + value: details.errorInstance, + configurable: true, + writable: true, + }) + } + throw error + } + // fast-check accepts an async reporter only for an async property. When a + // synchronous property rejects it, the guarded runner retries in the + // synchronous mode, which uses fast-check's synchronous formatter. + if (process.env.TANSTACK_DB_ORACLE_REPLAY_WITNESS_MODE === `sync`) { + return { + reporter: (details) => { + record(details) + rethrow(details, defaultReportMessage(details)) + }, + } + } return { asyncReporter: async (details) => { - // All named assertion owners use async fast-check properties. // Statistics sampling has no assertion reporter and cannot earn reach. - const witness: OracleReplayWitness = { - property, - seed: details.seed, - path: details.runConfiguration.path, - numRuns: details.numRuns, - failed: details.failed, - } - appendFileSync(channel, `${JSON.stringify(witness)}\n`) - - // A custom reporter replaces fast-check's throw. Preserve its public async - // formatter and errorWithCause policy, including precondition failures. - const message = await asyncDefaultReportMessage(details) - if (message !== undefined) { - const error = new Error(message) - if (details.runConfiguration.errorWithCause) { - Object.defineProperty(error, `cause`, { - value: details.errorInstance, - configurable: true, - writable: true, - }) - } - throw error - } + record(details) + rethrow(details, await asyncDefaultReportMessage(details)) }, } } diff --git a/packages/db/tests/oracle-replay.test.ts b/packages/db/tests/oracle-replay.test.ts index 05da1ba869..dd26c2d1a9 100644 --- a/packages/db/tests/oracle-replay.test.ts +++ b/packages/db/tests/oracle-replay.test.ts @@ -33,7 +33,7 @@ import { const packageDirectory = resolve(dirname(fileURLToPath(import.meta.url)), `..`) const repositoryDirectory = resolve(packageDirectory, `../..`) -const fixture = `tests/oracle-replay.fixture.test.ts` +const fixture = `tests/replay-guard-calibration.fixture.test.ts` function runReplay( args: Array = [fixture], @@ -207,6 +207,13 @@ describe(`guarded oracle replay`, () => { 40_000, ) + it(`witnesses a synchronous target through the synchronous witness`, () => { + const result = runReplay([fixture], `sync`) + expect(result.status, result.output).toBe(0) + expect(result.output).toContain(`"witnessMode":"sync"`) + expect(result.output).toContain(`"executions":1`) + }, 60_000) + it.each([ { name: `wrong file`, @@ -269,7 +276,6 @@ describe(`guarded oracle replay`, () => { }, 40_000) it.each([ - [`coverage-registry.claim-churn`, `no-named-owner`], [`includes.scenario-statistics`, `statistics-only`], [`absent.property`, `unknown oracle property`], ])( @@ -364,28 +370,19 @@ describe(`oracle replay reporting`, () => { }) describe(`named oracle replay manifest`, () => { - it(`accounts for every registry name without deleting unavailable names`, () => { + it(`accounts for every registry name`, () => { expect( new Set(oracleReplayManifest.map(({ property }) => property)), ).toEqual(registeredOracleProperties) - expect( - oracleReplayManifest.filter(({ property }) => - /^(coverage-registry|predicate-subtraction|load-subset-full-flow)\./.test( - property, - ), - ), - ).toHaveLength(14) - expect( - oracleReplayManifest.filter(({ property }) => - property.startsWith(`load-subset-refinement.`), - ), - ).toHaveLength(11) }) it(`maps literal owners and all computed publication cells to source files`, () => { for (const entry of oracleReplayManifest) { if (entry.file === undefined) continue - const source = readFileSync(join(repositoryDirectory, entry.file), `utf8`) + const source = readFileSync( + join(repositoryDirectory, entry.source ?? entry.file), + `utf8`, + ) if ( entry.property.startsWith(`includes-publication.`) && entry.property.split(`.`).length === 4 @@ -433,6 +430,10 @@ describe(`named oracle replay manifest`, () => { expect(source).toContain( family?.[1] ?? `includes.matrix.flat-materialization`, ) + } else if (/^(utils|comparison)\.\d+$/.test(entry.property)) { + // These oracles number their properties in file order. + const [family] = entry.property.split(`.`) + expect(source).toContain(`\`${family}.\${id}\``) } else if (entry.property.startsWith(`pagination.matrix.`)) { const [, , familyCell] = entry.property.split(`.`) const [family, cell] = familyCell!.split(`-`) @@ -441,11 +442,47 @@ describe(`named oracle replay manifest`, () => { expect(Number(cell)).toBeLessThan(8) expect(source).toContain(`\`pagination.matrix.${family}-\${index}\``) } else { - expect(source).toContain(`\`${entry.property}\``) + // A module may spell the name in any string quotes. + expect( + [`\``, `'`, `"`].some((quote) => + source.includes(`${quote}${entry.property}${quote}`), + ), + `${entry.source ?? entry.file} names ${entry.property}`, + ).toBe(true) } } }) + it(`names an owner for every registered property`, () => { + // A registered name that no file owns would pass replay validation and + // run nothing. Every new registered property must name its owner. + expect( + oracleReplayManifest + .filter(({ status }) => status === `no-named-owner`) + .map(({ property }) => property), + ).toEqual([]) + }) + + it(`replays a property without a shrink path from its seed alone`, () => { + // The LIKE oracle draws its cases from its own seeded generator, so the + // guarded runner, which witnesses a seed and a path, cannot witness it. + expect( + oracleReplayManifest + .filter(({ status }) => status === `seed-replay`) + .map(({ property, command, environment }) => ({ + property, + command, + environment, + })), + ).toEqual([ + { + property: `evaluators.like`, + command: `pnpm exec vitest run tests/query/compiler/evaluators-oracle.test.ts`, + environment: [`TANSTACK_DB_ORACLE_SEED`], + }, + ]) + }) + it(`gives every index and pagination matrix property a direct test selector`, () => { const direct = oracleReplayManifest.filter(({ property }) => /^(index-update|pagination\.matrix)\./.test(property), diff --git a/packages/db/tests/oracle-replay.ts b/packages/db/tests/oracle-replay.ts index 60ae4784de..3d0e11b37b 100644 --- a/packages/db/tests/oracle-replay.ts +++ b/packages/db/tests/oracle-replay.ts @@ -26,7 +26,10 @@ const entry = oracleReplayManifest.find( ) if (entry?.status !== `assertion`) { throw new Error( - `oracle replay target ${property}: ${entry?.status ?? `missing manifest entry`}`, + `oracle replay target ${property}: ${entry?.status ?? `missing manifest entry`}` + + (entry?.command === undefined + ? `` + : `. Replay it with: ${entry.environment.join(` `)} ${entry.command}`), ) } @@ -73,21 +76,38 @@ try { entry.testNamePattern === undefined ? requestedArgs : selectReplayTest(requestedArgs, entry.testNamePattern) - const result = spawnSync(process.execPath, [vitest, `run`, ...args], { - cwd: process.cwd(), - env: { ...process.env, TANSTACK_DB_ORACLE_REPLAY_WITNESS: channel }, - stdio: `inherit`, - }) - if (result.error) throw result.error - const records = existsSync(channel) - ? readFileSync(channel, `utf8`).trim() - : `` - const witnesses: Array = - records === `` - ? [] - : records - .split(`\n`) - .map((line: string) => JSON.parse(line) as OracleReplayWitness) + // A synchronous property rejects the async witness before it runs, so the + // first attempt records nothing and fails. Retry once with the synchronous + // witness. An async property records its witness on the first attempt. + const attempt = (mode: `async` | `sync`) => { + rmSync(channel, { force: true }) + const run = spawnSync(process.execPath, [vitest, `run`, ...args], { + cwd: process.cwd(), + env: { + ...process.env, + TANSTACK_DB_ORACLE_REPLAY_WITNESS: channel, + TANSTACK_DB_ORACLE_REPLAY_WITNESS_MODE: mode, + }, + stdio: `inherit`, + }) + if (run.error) throw run.error + const lines = existsSync(channel) + ? readFileSync(channel, `utf8`).trim() + : `` + const recorded: Array = + lines === `` + ? [] + : lines + .split(`\n`) + .map((line: string) => JSON.parse(line) as OracleReplayWitness) + return { run, recorded } + } + let { run: result, recorded: witnesses } = attempt(`async`) + let witnessMode: `async` | `sync` = `async` + if (witnesses.length === 0 && result.status !== 0) { + ;({ run: result, recorded: witnesses } = attempt(`sync`)) + witnessMode = `sync` + } const reached = witnesses.filter( (witness) => witness.property === property && @@ -102,6 +122,7 @@ try { path: config.replayPath, owner: entry.file, command: [process.execPath, vitest, `run`, ...args], + witnessMode, witnesses, executions: reached.length, })}`, diff --git a/packages/db/tests/proxy-detachment-contract-oracle.test.ts b/packages/db/tests/proxy-detachment-contract-oracle.test.ts index 9f4ea51bfe..31a5251e50 100644 --- a/packages/db/tests/proxy-detachment-contract-oracle.test.ts +++ b/packages/db/tests/proxy-detachment-contract-oracle.test.ts @@ -3,6 +3,7 @@ import { describe, expect, it, vi } from 'vitest' import fc from 'fast-check' import { createCollection } from '../src/collection/index.js' import { createChangeProxy, withChangeTracking } from '../src/proxy.js' +import { oracleRuns } from './oracle-config.js' import type { StandardSchemaV1 } from '@standard-schema/spec' import type { CollectionConfig } from '../src/types.js' @@ -505,7 +506,11 @@ describe(`Mutation result detachment`, () => { expect(original).toEqual({ payload: { before: true } }) }, ), - { seed, numRuns: 100, examples: [[`__proto__`, 1, false]] }, + { + seed, + numRuns: oracleRuns(100), + examples: [[`__proto__`, 1, false]], + }, ) }, ) diff --git a/packages/db/tests/proxy-native-methods-oracle.property.test.ts b/packages/db/tests/proxy-native-methods-oracle.property.test.ts index 4e00ba04d1..2d79ef610f 100644 --- a/packages/db/tests/proxy-native-methods-oracle.property.test.ts +++ b/packages/db/tests/proxy-native-methods-oracle.property.test.ts @@ -1,6 +1,11 @@ -import { describe, expect, it } from 'vitest' -import { fc } from '@fast-check/vitest' +import { describe, expect, it as vitestIt } from 'vitest' +import { fc, test as fcTest } from '@fast-check/vitest' import { withChangeTracking } from '../src/proxy' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from './oracle-config.js' /** * # Do built-in array and typed-array methods behave on a draft as natively? @@ -41,36 +46,17 @@ import { withChangeTracking } from '../src/proxy' */ // --------------------------------------------------------------------------- -// Campaigns. `TANSTACK_DB_PROXY_NATIVE_SEED` and -// `TANSTACK_DB_PROXY_NATIVE_PATH` select a direct replay. +// Campaigns. Each generated property has a fixed campaign and a random +// campaign over the same grammar and check. A path replay of one property +// runs its recorded history once and every other test stands aside. A seed +// alone reseeds the random campaigns. -const replaySeed = process.env.TANSTACK_DB_PROXY_NATIVE_SEED -const replayPath = process.env.TANSTACK_DB_PROXY_NATIVE_PATH const FIXED_SEED = 2026102 const RUNS = 400 -const campaigns = - replaySeed === undefined && replayPath === undefined - ? [ - { name: String(FIXED_SEED), seed: FIXED_SEED as number | undefined }, - { name: `random`, seed: undefined }, - ] - : [ - { - name: `replay`, - seed: replaySeed === undefined ? undefined : Number(replaySeed), - }, - ] -function campaignOptions(seed: number | undefined): fc.Parameters { - if (replayPath !== undefined && replaySeed === undefined) - throw new Error(`TANSTACK_DB_PROXY_NATIVE_PATH requires a seed`) - if (seed !== undefined && !Number.isSafeInteger(seed)) - throw new Error(`TANSTACK_DB_PROXY_NATIVE_SEED must be an integer`) - return { - numRuns: RUNS, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - } -} +const ARRAY_METHODS = `proxy.native-array-methods` +const TYPED_METHODS = `proxy.native-typed-methods` + +const it = oracleReplayTest(vitestIt) const prototypeMethods = (prototype: object, skip: Set) => Object.getOwnPropertyNames(prototype).filter( @@ -253,19 +239,21 @@ describe(`array methods behave like native arrays`, () => { expectArrayCall([{ x: 1 }, 2, [{ x: 3 }], { x: 0 }], m, k) }) - for (const { name, seed } of campaigns) { - it(`matches native arrays across generated rows (${name})`, () => { - fc.assert( - fc.property( - arraySpecArb, - fc.constantFrom(...arrayMethods), - fc.nat(4), - expectArrayCall, - ), - campaignOptions(seed), - ) - }) - } + it(`matches native arrays across generated rows (${FIXED_SEED})`, () => { + fc.assert( + fc.property( + arraySpecArb, + fc.constantFrom(...arrayMethods), + fc.nat(4), + expectArrayCall, + ), + { numRuns: oracleRuns(RUNS), seed: FIXED_SEED }, + ) + }) + oracleReplayTest(fcTest, ARRAY_METHODS).prop( + [arraySpecArb, fc.constantFrom(...arrayMethods), fc.nat(4)], + oraclePropertyOptions(RUNS, ARRAY_METHODS), + )(`matches native arrays across generated rows (random)`, expectArrayCall) it(`reaches every method, holes, empty rows, and nested objects in the fixed campaign`, () => { const sample = fc.sample( @@ -410,20 +398,25 @@ describe(`typed-array methods behave like native typed arrays`, () => { expectTypedCall([-0, 0], `reverse`, 0, false) }) - for (const { name, seed } of campaigns) { - it(`matches native typed arrays across generated rows (${name})`, () => { - fc.assert( - fc.property( - typedSpecArb, - fc.constantFrom(...typedMethods), - fc.nat(4), - fc.boolean(), - expectTypedCall, - ), - campaignOptions(seed), - ) - }) - } + it(`matches native typed arrays across generated rows (${FIXED_SEED})`, () => { + fc.assert( + fc.property( + typedSpecArb, + fc.constantFrom(...typedMethods), + fc.nat(4), + fc.boolean(), + expectTypedCall, + ), + { numRuns: oracleRuns(RUNS), seed: FIXED_SEED }, + ) + }) + oracleReplayTest(fcTest, TYPED_METHODS).prop( + [typedSpecArb, fc.constantFrom(...typedMethods), fc.nat(4), fc.boolean()], + oraclePropertyOptions(RUNS, TYPED_METHODS), + )( + `matches native typed arrays across generated rows (random)`, + expectTypedCall, + ) it(`reaches every method, empty rows, NaN, and -0 in the fixed campaign`, () => { const sample = fc.sample( diff --git a/packages/db/tests/proxy-oracle.test.ts b/packages/db/tests/proxy-oracle.test.ts index a6135a77dc..d8111605e7 100644 --- a/packages/db/tests/proxy-oracle.test.ts +++ b/packages/db/tests/proxy-oracle.test.ts @@ -8,6 +8,11 @@ import { withArrayChangeTracking, withChangeTracking, } from '../src/proxy' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from './oracle-config' /** * # Does a draft behave like the native value it represents? @@ -248,8 +253,8 @@ describe(`native array callback oracle`, () => { target: fc.integer({ min: 0, max: 5 }), delta: fc.integer({ min: -5, max: 5 }), }) - // A fixed and a random campaign. TANSTACK_DB_PROXY_CALLBACK_SEED and - // TANSTACK_DB_PROXY_CALLBACK_PATH select a direct replay instead. + // A fixed and a random campaign. A path replay of + // `proxy.callback-history` runs only the random campaign, from its path. const callbackHistory = fc.record({ values: fc.array(fc.integer({ min: -10, max: 10 }), { minLength: 1, @@ -257,38 +262,24 @@ describe(`native array callback oracle`, () => { }), steps: fc.tuple(step, step), }) - const replaySeed = process.env.TANSTACK_DB_PROXY_CALLBACK_SEED - const replayPath = process.env.TANSTACK_DB_PROXY_CALLBACK_PATH - const callbackCampaigns = - replaySeed === undefined && replayPath === undefined - ? [ - { name: `2026103`, seed: 2026103 as number | undefined }, - { name: `random`, seed: undefined }, - ] - : [ - { - name: `replay`, - seed: replaySeed === undefined ? undefined : Number(replaySeed), - }, - ] - for (const { name, seed } of callbackCampaigns) { - it(`matches native two-step callback histories (${name})`, () => { - if (replayPath !== undefined && replaySeed === undefined) - throw new Error(`TANSTACK_DB_PROXY_CALLBACK_PATH requires a seed`) - if (seed !== undefined && !Number.isSafeInteger(seed)) - throw new Error(`TANSTACK_DB_PROXY_CALLBACK_SEED must be an integer`) + const callbackProperty = fc.property(callbackHistory, ({ values, steps }) => { + assertArrayCallbacks(observeArrayCallbacks(values, steps)) + }) + oracleReplayTest(it)( + `matches native two-step callback histories (2026103)`, + () => { + fc.assert(callbackProperty, { seed: 2026103, numRuns: oracleRuns(200) }) + }, + ) + oracleReplayTest(it, `proxy.callback-history`)( + `matches native two-step callback histories (random)`, + () => { fc.assert( - fc.property(callbackHistory, ({ values, steps }) => { - assertArrayCallbacks(observeArrayCallbacks(values, steps)) - }), - { - numRuns: 200, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - }, + callbackProperty, + oraclePropertyOptions(200, `proxy.callback-history`), ) - }) - } + }, + ) it.each([`lost-write`, `extra-visit`, `wrong-peer`] as const)( `rejects a captured %s independently of the native authority`, diff --git a/packages/db/tests/proxy-revert-oracle.property.test.ts b/packages/db/tests/proxy-revert-oracle.property.test.ts index 7edad41010..112964b051 100644 --- a/packages/db/tests/proxy-revert-oracle.property.test.ts +++ b/packages/db/tests/proxy-revert-oracle.property.test.ts @@ -1,6 +1,11 @@ -import { describe, expect, it } from 'vitest' -import { fc } from '@fast-check/vitest' +import { describe, expect, it as vitestIt } from 'vitest' +import { fc, test as fcTest } from '@fast-check/vitest' import { createChangeProxy, withChangeTracking } from '../src/proxy' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from './oracle-config' /** * # Which draft writes count as changes, and which count as reverts? @@ -780,65 +785,76 @@ function expectHistory({ original, ops }: History): void { } // --------------------------------------------------------------------------- -// Campaigns. `TANSTACK_DB_PROXY_REVERT_SEED` and -// `TANSTACK_DB_PROXY_REVERT_PATH` select a direct replay. +// Campaigns. Each generated property has a fixed campaign and a random +// campaign over the same grammar and check. A path replay of one property +// runs its recorded history once and every other test stands aside. A seed +// alone reseeds the random campaigns. -const replaySeed = process.env.TANSTACK_DB_PROXY_REVERT_SEED -const replayPath = process.env.TANSTACK_DB_PROXY_REVERT_PATH const FIXED_SEED = 2026101 -const campaigns = - replaySeed === undefined && replayPath === undefined - ? [ - { name: String(FIXED_SEED), seed: FIXED_SEED as number | undefined }, - { name: `random`, seed: undefined }, - ] - : [ - { - name: `replay`, - seed: replaySeed === undefined ? undefined : Number(replaySeed), - }, - ] +const WRITE_HISTORY = `proxy.revert-history` +const NESTED_ROUND_TRIPS = `proxy.revert-nested-round-trips` +const CLASS_WRITES = `proxy.revert-class-writes` +const PARTIAL_REVERTS = `proxy.revert-partial-reverts` +const NATIVE_HISTORIES = `proxy.revert-native-histories` + +const it = oracleReplayTest(vitestIt) describe(`draft revert oracle`, () => { - for (const { name, seed } of campaigns) { - it(`matches the model across generated write histories (${name})`, () => { - if (replayPath !== undefined && replaySeed === undefined) - throw new Error(`TANSTACK_DB_PROXY_REVERT_PATH requires a seed`) - if ( - replaySeed !== undefined && - (replaySeed.trim() === `` || - typeof seed !== `number` || - !Number.isSafeInteger(seed)) - ) - throw new Error(`TANSTACK_DB_PROXY_REVERT_SEED must be an integer`) - fc.assert(fc.property(historyArb, expectHistory), { - numRuns: 400, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - }) + it(`matches the model across generated write histories (${FIXED_SEED})`, () => { + fc.assert(fc.property(historyArb, expectHistory), { + numRuns: oracleRuns(400), + seed: FIXED_SEED, }) - it(`matches the model across generated nested round trips (${name})`, () => { - fc.assert(fc.property(nestedRoundTripArb, expectHistory), { - numRuns: 200, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - }) + }) + oracleReplayTest(fcTest, WRITE_HISTORY).prop( + [historyArb], + oraclePropertyOptions(400, WRITE_HISTORY), + )( + `matches the model across generated write histories (random)`, + expectHistory, + ) + + it(`matches the model across generated nested round trips (${FIXED_SEED})`, () => { + fc.assert(fc.property(nestedRoundTripArb, expectHistory), { + numRuns: oracleRuns(200), + seed: FIXED_SEED, }) - it(`matches the model across generated URL and keyless writes (${name})`, () => { - fc.assert(fc.property(classWriteArb, expectHistory), { - numRuns: 200, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - }) + }) + oracleReplayTest(fcTest, NESTED_ROUND_TRIPS).prop( + [nestedRoundTripArb], + oraclePropertyOptions(200, NESTED_ROUND_TRIPS), + )( + `matches the model across generated nested round trips (random)`, + expectHistory, + ) + + it(`matches the model across generated URL and keyless writes (${FIXED_SEED})`, () => { + fc.assert(fc.property(classWriteArb, expectHistory), { + numRuns: oracleRuns(200), + seed: FIXED_SEED, }) - it(`matches the model across generated partial reverts (${name})`, () => { - fc.assert(fc.property(partialRevertArb, expectHistory), { - numRuns: 200, - ...(seed === undefined ? {} : { seed }), - ...(replayPath === undefined ? {} : { path: replayPath }), - }) + }) + oracleReplayTest(fcTest, CLASS_WRITES).prop( + [classWriteArb], + oraclePropertyOptions(200, CLASS_WRITES), + )( + `matches the model across generated URL and keyless writes (random)`, + expectHistory, + ) + + it(`matches the model across generated partial reverts (${FIXED_SEED})`, () => { + fc.assert(fc.property(partialRevertArb, expectHistory), { + numRuns: oracleRuns(200), + seed: FIXED_SEED, }) - } + }) + oracleReplayTest(fcTest, PARTIAL_REVERTS).prop( + [partialRevertArb], + oraclePropertyOptions(200, PARTIAL_REVERTS), + )( + `matches the model across generated partial reverts (random)`, + expectHistory, + ) // Positive execution witness: the fixed campaign reaches each operation, a // partial revert (some change survives one revert), a full revert, and a @@ -1426,14 +1442,19 @@ describe(`draft revert oracle: native mutators mixed with assignments`, () => { } }) - for (const { name, seed } of campaigns) { - it(`matches the model across generated native histories (${name})`, () => { - fc.assert(fc.property(nativeHistoryArb, expectNativeHistory), { - numRuns: 400, - ...(seed === undefined ? {} : { seed }), - }) + it(`matches the model across generated native histories (${FIXED_SEED})`, () => { + fc.assert(fc.property(nativeHistoryArb, expectNativeHistory), { + numRuns: oracleRuns(400), + seed: FIXED_SEED, }) - } + }) + oracleReplayTest(fcTest, NATIVE_HISTORIES).prop( + [nativeHistoryArb], + oraclePropertyOptions(400, NATIVE_HISTORIES), + )( + `matches the model across generated native histories (random)`, + expectNativeHistory, + ) // Positive execution witness: the fixed campaign reaches a native write // that an assignment then replaces, and a mutator through a detached handle. diff --git a/packages/db/tests/query/compiler/evaluators-oracle.test.ts b/packages/db/tests/query/compiler/evaluators-oracle.test.ts index d0af418d60..03af6bd8c8 100644 --- a/packages/db/tests/query/compiler/evaluators-oracle.test.ts +++ b/packages/db/tests/query/compiler/evaluators-oracle.test.ts @@ -5,6 +5,11 @@ import { compileSingleRowExpression, } from '../../../src/query/compiler/evaluators.js' import { Func, PropRef, Value } from '../../../src/query/ir.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../oracle-config.js' import type { NamespacedRow } from '../../../src/types.js' /** @@ -44,12 +49,8 @@ type LikeMismatch = LikeCase & { } const FIXED_LIKE_ORACLE_SEED = 0x1745 -const likeReplaySeedText = process.env.TANSTACK_DB_LIKE_ORACLE_SEED -const likeReplaySeed = - likeReplaySeedText === undefined ? undefined : Number(likeReplaySeedText) -const LIKE_ORACLE_RUNS = Number( - process.env.TANSTACK_DB_LIKE_ORACLE_RUNS ?? 10_000, -) +const likeProperty = `evaluators.like` +const LIKE_ORACLE_RUNS = oracleRuns(10_000) const likeCalibrationCases: Array = [ { value: ``, pattern: ``, caseInsensitive: false }, { value: `A`, pattern: `a`, caseInsensitive: true }, @@ -58,31 +59,23 @@ const likeCalibrationCases: Array = [ { value: `100% done`, pattern: `100%done`, caseInsensitive: false }, ] -if ( - likeReplaySeedText !== undefined && - (likeReplaySeedText.trim() === `` || - !Number.isSafeInteger(likeReplaySeed) || - likeReplaySeed! < 0 || - likeReplaySeed! > 0xffff_ffff) -) { - throw new Error(`TANSTACK_DB_LIKE_ORACLE_SEED must be a uint32 integer`) -} -if ( - !Number.isSafeInteger(LIKE_ORACLE_RUNS) || - LIKE_ORACLE_RUNS < likeCalibrationCases.length -) { - throw new Error( - `TANSTACK_DB_LIKE_ORACLE_RUNS must be an integer of at least ${likeCalibrationCases.length}`, - ) -} - -const likeOracleCampaigns = - likeReplaySeed === undefined - ? [ - { name: `fixed`, seed: FIXED_LIKE_ORACLE_SEED }, - { name: `random`, seed: Math.floor(Math.random() * 0x1_0000_0000) }, - ] - : [{ name: `replay`, seed: likeReplaySeed }] +// The fixed campaign stands aside during a path replay. The random campaign uses +// the seed of a replay of this property, when TANSTACK_DB_ORACLE_SEED supplies +// one. This generator has no shrink path, so a path replay only reseeds it. +const likeOracleCampaigns = [ + { + name: `fixed`, + test: oracleReplayTest(it), + seed: FIXED_LIKE_ORACLE_SEED, + }, + { + name: `random`, + test: oracleReplayTest(it, likeProperty), + seed: + oraclePropertyOptions(10_000, likeProperty).seed ?? + Math.floor(Math.random() * 0x1_0000_0000), + }, +] function referenceLike( value: string, @@ -548,49 +541,51 @@ describe(`evaluators`, () => { }) }) - it.each(likeOracleCampaigns)( - `matches the LIKE law in the $name campaign`, - ({ seed }) => { - let mismatchCount = 0 - const mismatchSamples: Array = [] - const reach = { - caseSensitive: 0, - caseInsensitive: 0, - percentPattern: 0, - underscorePattern: 0, - literalPattern: 0, - } + for (const campaign of likeOracleCampaigns) + campaign.test( + `matches the LIKE law in the ${campaign.name} campaign`, + () => { + const { seed } = campaign + let mismatchCount = 0 + const mismatchSamples: Array = [] + const reach = { + caseSensitive: 0, + caseInsensitive: 0, + percentPattern: 0, + underscorePattern: 0, + literalPattern: 0, + } - for (const testCase of seededLikeCases(LIKE_ORACLE_RUNS, seed)) { - reach[ - testCase.caseInsensitive ? `caseInsensitive` : `caseSensitive` - ]++ - if (testCase.pattern.includes(`%`)) reach.percentPattern++ - if (testCase.pattern.includes(`_`)) reach.underscorePattern++ - if (/[^%_]/u.test(testCase.pattern)) reach.literalPattern++ - - const mismatch = findLikeMismatch( - testCase, - evaluateLikeWithProduction(testCase), - ) - if (mismatch !== undefined) { - mismatchCount++ - if (mismatchSamples.length < 5) { - mismatchSamples.push(mismatch) + for (const testCase of seededLikeCases(LIKE_ORACLE_RUNS, seed)) { + reach[ + testCase.caseInsensitive ? `caseInsensitive` : `caseSensitive` + ]++ + if (testCase.pattern.includes(`%`)) reach.percentPattern++ + if (testCase.pattern.includes(`_`)) reach.underscorePattern++ + if (/[^%_]/u.test(testCase.pattern)) reach.literalPattern++ + + const mismatch = findLikeMismatch( + testCase, + evaluateLikeWithProduction(testCase), + ) + if (mismatch !== undefined) { + mismatchCount++ + if (mismatchSamples.length < 5) { + mismatchSamples.push(mismatch) + } } } - } - expect({ seed, mismatchCount, mismatchSamples }).toEqual({ - seed, - mismatchCount: 0, - mismatchSamples: [], - }) - expect( - Object.entries(reach).filter(([, count]) => count === 0), - ).toEqual([]) - }, - ) + expect({ seed, mismatchCount, mismatchSamples }).toEqual({ + seed, + mismatchCount: 0, + mismatchSamples: [], + }) + expect( + Object.entries(reach).filter(([, count]) => count === 0), + ).toEqual([]) + }, + ) it(`handles like where _ must match exactly one character`, () => { const func = new Func(`like`, [new Value(`hell`), new Value(`hell_`)]) diff --git a/packages/db/tests/query/includes-collection-oracle.property.test.ts b/packages/db/tests/query/includes-collection-oracle.property.test.ts index b162ba7e5e..fd3b643982 100644 --- a/packages/db/tests/query/includes-collection-oracle.property.test.ts +++ b/packages/db/tests/query/includes-collection-oracle.property.test.ts @@ -20,8 +20,8 @@ import { runTrace } from '../trace-runner.js' import { withHistoryCleanup } from '../optimistic-history-oracle.js' import { oraclePropertyOptions, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../oracle-config.js' import { flushPromises, withExpectedRejection } from '../utils.js' import { createControlledCollection as createOracleControlledCollection } from './includes-oracle-helpers.js' @@ -675,7 +675,6 @@ const exhaustiveActions: ReadonlyArray = [ { type: `deleteChild`, id: 10 }, ] -const requestedReplayProperty = readOracleRunConfig().replayProperty const relationshipProperty = `includes-collection.relationship-history` const publicKeyOrderProperty = `includes-collection.public-key-order` const layoutSwapProperty = `includes-collection.layout-swap` @@ -822,27 +821,20 @@ describe(`Collection-valued includes oracle`, () => { projection: collectionProjection, }) - if (requestedReplayProperty === undefined) { - fcTest.prop([collectionScenarioArbitrary], { - numRuns: oracleRuns(30), - seed: 1_658_101, - })( - `keeps Collection, toArray, and materialize equivalent across fixed relationship histories`, - runRelationshipHistory, - ) - } - if ( - requestedReplayProperty === undefined || - requestedReplayProperty === relationshipProperty - ) { - fcTest.prop( - [collectionScenarioArbitrary], - oraclePropertyOptions(30, relationshipProperty), - )( - `keeps Collection, toArray, and materialize equivalent across random or replayed relationship histories`, - runRelationshipHistory, - ) - } + oracleReplayTest(fcTest).prop([collectionScenarioArbitrary], { + numRuns: oracleRuns(30), + seed: 1_658_101, + })( + `keeps Collection, toArray, and materialize equivalent across fixed relationship histories`, + runRelationshipHistory, + ) + oracleReplayTest(fcTest, relationshipProperty).prop( + [collectionScenarioArbitrary], + oraclePropertyOptions(30, relationshipProperty), + )( + `keeps Collection, toArray, and materialize equivalent across random or replayed relationship histories`, + runRelationshipHistory, + ) fcTest( `exhaustively matches every two-step history in the smallest relationship domain`, @@ -2078,27 +2070,20 @@ describe(`Collection-valued includes oracle`, () => { ) } - if (requestedReplayProperty === undefined) { - fcTest.prop([publicKeyOrderArbitrary], { - numRuns: oracleRuns(20), - seed: 1_658_102, - })( - `uses one raw public-key order across Collection and inline materializations with a fixed seed`, - runPublicKeyOrder, - ) - } - if ( - requestedReplayProperty === undefined || - requestedReplayProperty === publicKeyOrderProperty - ) { - fcTest.prop( - [publicKeyOrderArbitrary], - oraclePropertyOptions(20, publicKeyOrderProperty), - )( - `uses one raw public-key order across Collection and inline materializations with a random or replayed seed`, - runPublicKeyOrder, - ) - } + oracleReplayTest(fcTest).prop([publicKeyOrderArbitrary], { + numRuns: oracleRuns(20), + seed: 1_658_102, + })( + `uses one raw public-key order across Collection and inline materializations with a fixed seed`, + runPublicKeyOrder, + ) + oracleReplayTest(fcTest, publicKeyOrderProperty).prop( + [publicKeyOrderArbitrary], + oraclePropertyOptions(20, publicKeyOrderProperty), + )( + `uses one raw public-key order across Collection and inline materializations with a random or replayed seed`, + runPublicKeyOrder, + ) const runLayoutSwap = async ({ length, @@ -2213,27 +2198,20 @@ describe(`Collection-valued includes oracle`, () => { ) } - if (requestedReplayProperty === undefined) { - fcTest.prop([orderSwapArbitrary], { - numRuns: oracleRuns(20), - seed: 1_658_103, - })( - `propagates order-only child swaps through every materialization with a fixed seed`, - runLayoutSwap, - ) - } - if ( - requestedReplayProperty === undefined || - requestedReplayProperty === layoutSwapProperty - ) { - fcTest.prop( - [orderSwapArbitrary], - oraclePropertyOptions(20, layoutSwapProperty), - )( - `propagates order-only child swaps through every materialization with a random or replayed seed`, - runLayoutSwap, - ) - } + oracleReplayTest(fcTest).prop([orderSwapArbitrary], { + numRuns: oracleRuns(20), + seed: 1_658_103, + })( + `propagates order-only child swaps through every materialization with a fixed seed`, + runLayoutSwap, + ) + oracleReplayTest(fcTest, layoutSwapProperty).prop( + [orderSwapArbitrary], + oraclePropertyOptions(20, layoutSwapProperty), + )( + `propagates order-only child swaps through every materialization with a random or replayed seed`, + runLayoutSwap, + ) for (const { intermediate, final } of [ { intermediate: `A`, final: `A` }, @@ -3218,25 +3196,18 @@ describe(`Collection-valued includes oracle`, () => { }, ) - if (requestedReplayProperty === undefined) { - fcTest.prop([optimisticChildArbitrary], { - numRuns: oracleRuns(20), - seed: 1_658_104, - })( - `matches optimistic child settlement histories with a fixed seed`, - runOptimisticChildHistory, - ) - } - if ( - requestedReplayProperty === undefined || - requestedReplayProperty === optimisticChildProperty - ) { - fcTest.prop( - [optimisticChildArbitrary], - oraclePropertyOptions(20, optimisticChildProperty), - )( - `matches optimistic child settlement histories with a random or replayed seed`, - runOptimisticChildHistory, - ) - } + oracleReplayTest(fcTest).prop([optimisticChildArbitrary], { + numRuns: oracleRuns(20), + seed: 1_658_104, + })( + `matches optimistic child settlement histories with a fixed seed`, + runOptimisticChildHistory, + ) + oracleReplayTest(fcTest, optimisticChildProperty).prop( + [optimisticChildArbitrary], + oraclePropertyOptions(20, optimisticChildProperty), + )( + `matches optimistic child settlement histories with a random or replayed seed`, + runOptimisticChildHistory, + ) }) diff --git a/packages/db/tests/query/includes-cross-formulation-oracle.property.test.ts b/packages/db/tests/query/includes-cross-formulation-oracle.property.test.ts index 46523e423b..4a136ec386 100644 --- a/packages/db/tests/query/includes-cross-formulation-oracle.property.test.ts +++ b/packages/db/tests/query/includes-cross-formulation-oracle.property.test.ts @@ -16,7 +16,10 @@ import { toArray, } from '../../src/query/index.js' import { + oracleCalibrationTest, oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, readOracleRunConfig, } from '../oracle-config.js' @@ -983,17 +986,6 @@ const windowedExamples: Array< describe(`includes cross-formulation oracle`, () => { const replay = readOracleRunConfig() - const fixedCampaign = replay.replayPath === undefined ? fcTest : fcTest.skip - const randomCampaign = (property: string) => - replay.replayPath === undefined || replay.replayProperty === property - ? fcTest - : fcTest.skip - const calibrationTest = (property: string, fault: string) => - replay.replayPath === undefined || - (replay.replayProperty === property && - process.env.TANSTACK_DB_ORACLE_CALIBRATION === fault) - ? test - : test.skip test(`bounded grammar axes distinguish routes, order, partitions, and windows`, () => { const parents = new Map( @@ -1523,7 +1515,7 @@ describe(`includes cross-formulation oracle`, () => { // The literal wrong-result and captured replay controls above distinguish // merged reference routes and symbol groups at their public checkpoints. - fixedCampaign.prop([fc.integer()], { + oracleReplayTest(fcTest).prop([fc.integer()], { numRuns: oracleRuns(4), seed: 1_658_001, })( @@ -1533,7 +1525,7 @@ describe(`includes cross-formulation oracle`, () => { }, ) - randomCampaign(`includes-cross-formulation.reference-context`).prop( + oracleReplayTest(fcTest, `includes-cross-formulation.reference-context`).prop( [fc.integer()], oraclePropertyOptions(4, `includes-cross-formulation.reference-context`), )( @@ -1828,7 +1820,7 @@ describe(`includes cross-formulation oracle`, () => { }, ) - fixedCampaign.prop([fc.integer()], { + oracleReplayTest(fcTest).prop([fc.integer()], { numRuns: oracleRuns(4), seed: 1_658_002, })( @@ -1836,7 +1828,7 @@ describe(`includes cross-formulation oracle`, () => { checkReferenceKeyRouting, ) - randomCampaign(`includes-cross-formulation.reference-key`).prop( + oracleReplayTest(fcTest, `includes-cross-formulation.reference-key`).prop( [fc.integer()], oraclePropertyOptions(4, `includes-cross-formulation.reference-key`), )( @@ -1989,7 +1981,7 @@ describe(`includes cross-formulation oracle`, () => { ]) } - fixedCampaign.prop([fc.integer()], { + oracleReplayTest(fcTest).prop([fc.integer()], { numRuns: oracleRuns(4), seed: 1_658_003, })( @@ -1997,7 +1989,10 @@ describe(`includes cross-formulation oracle`, () => { checkSymbolGroupRouting, ) - randomCampaign(`includes-cross-formulation.symbol-group-route`).prop( + oracleReplayTest( + fcTest, + `includes-cross-formulation.symbol-group-route`, + ).prop( [fc.integer()], oraclePropertyOptions(4, `includes-cross-formulation.symbol-group-route`), )( @@ -2171,7 +2166,11 @@ describe(`includes cross-formulation oracle`, () => { // tests/query/includes-cross-formulation-oracle.property.test.ts // -t 'agrees across nested includes' // Expected: one main-property failure at cross-formulation checkpoint 0. - calibrationTest(`includes-cross-formulation.equivalence`, `missing-parent`)( + oracleCalibrationTest( + test, + `includes-cross-formulation.equivalence`, + `missing-parent`, + )( `generated equivalence rejects a missing production parent and replays its checkpoint`, async () => { // The hostile observation is injected after the real nested query is @@ -2181,7 +2180,7 @@ describe(`includes cross-formulation oracle`, () => { await expectFormulationsEquivalent(scenario, (rows) => rows.slice(1)) }) const propertyId = `includes-cross-formulation.equivalence` - const directReplay = replay.replayPath !== undefined + const directReplay = oracleReplayRequested() const failure = await fc.check( property, directReplay @@ -2222,7 +2221,11 @@ describe(`includes cross-formulation oracle`, () => { // tests/query/includes-cross-formulation-oracle.property.test.ts // -t 'matches recomputation for ordered offset and limit child windows for a random' // Expected: one main-property failure at ordered-window checkpoint 0. - calibrationTest(`includes-cross-formulation.ordered-window`, `extra-child`)( + oracleCalibrationTest( + test, + `includes-cross-formulation.ordered-window`, + `extra-child`, + )( `generated window rejects an extra production child and replays its checkpoint`, async () => { const property = fc.asyncProperty( @@ -2242,7 +2245,7 @@ describe(`includes cross-formulation oracle`, () => { }, ) const propertyId = `includes-cross-formulation.ordered-window` - const directReplay = replay.replayPath !== undefined + const directReplay = oracleReplayRequested() const failure = await fc.check( property, directReplay @@ -2286,7 +2289,7 @@ describe(`includes cross-formulation oracle`, () => { : undefined, ) - fixedCampaign.prop([scenarioArbitrary], { + oracleReplayTest(fcTest).prop([scenarioArbitrary], { numRuns: oracleRuns(8), seed: 1_658_004, examples: scenarioExamples, @@ -2295,11 +2298,11 @@ describe(`includes cross-formulation oracle`, () => { checkEquivalenceScenario, ) - randomCampaign(`includes-cross-formulation.equivalence`).prop( + oracleReplayTest(fcTest, `includes-cross-formulation.equivalence`).prop( [scenarioArbitrary], { ...oraclePropertyOptions(8, `includes-cross-formulation.equivalence`), - examples: replay.replayPath === undefined ? scenarioExamples : [], + examples: oracleReplayRequested() ? [] : scenarioExamples, }, )( `agrees across nested includes, flat joins, per-parent queries, and TLP partitions for a random or replayed seed`, @@ -2334,7 +2337,7 @@ describe(`includes cross-formulation oracle`, () => { : undefined, ) - fixedCampaign.prop([windowedScenarioArbitrary], { + oracleReplayTest(fcTest).prop([windowedScenarioArbitrary], { numRuns: oracleRuns(12), seed: 1_658_005, examples: windowedExamples, @@ -2343,11 +2346,11 @@ describe(`includes cross-formulation oracle`, () => { checkWindowedScenario, ) - randomCampaign(`includes-cross-formulation.ordered-window`).prop( + oracleReplayTest(fcTest, `includes-cross-formulation.ordered-window`).prop( [windowedScenarioArbitrary], { ...oraclePropertyOptions(12, `includes-cross-formulation.ordered-window`), - examples: replay.replayPath === undefined ? windowedExamples : [], + examples: oracleReplayRequested() ? [] : windowedExamples, }, )( `matches recomputation for ordered offset and limit child windows for a random or replayed seed`, diff --git a/packages/db/tests/query/includes-optimistic-oracle.property.test.ts b/packages/db/tests/query/includes-optimistic-oracle.property.test.ts index 1a2013145d..9190d819d3 100644 --- a/packages/db/tests/query/includes-optimistic-oracle.property.test.ts +++ b/packages/db/tests/query/includes-optimistic-oracle.property.test.ts @@ -14,7 +14,7 @@ import { import { withHistoryCleanup } from '../optimistic-history-oracle.js' import { flushPromises, withExpectedRejection } from '../utils.js' import { runTrace } from '../trace-runner.js' -import { oraclePropertyOptions, readOracleRunConfig } from '../oracle-config.js' +import { oraclePropertyOptions, oracleReplayTest } from '../oracle-config.js' import { createControlledCollection as createOracleControlledCollection } from './includes-oracle-helpers.js' import type { TraceDriver, TraceProjection } from '../trace-runner.js' import type { OracleSyncChange as SyncChange } from './includes-oracle-helpers.js' @@ -646,22 +646,17 @@ function optimisticProperty( name: string, run: (routes: RouteValues) => Promise, ): void { - const { replayPath, replayProperty } = readOracleRunConfig() - if (replayPath !== undefined && replayProperty !== property) return - const options = oraclePropertyOptions(12, property) - // Both normal campaigns use the same arbitrary, driver, checker, and budget. - // A seed-and-path replay registers only its requested property and lane. - if (replayPath !== undefined) { - fcTest.prop([routeValuesArbitrary], options)(name, run) - return - } - - fcTest.prop([routeValuesArbitrary], { ...options, seed: 1_658_013 })( - `${name} (fixed seed)`, + // Both campaigns use the same arbitrary, driver, checker, and budget. A path + // replay of this property runs only the random campaign, from its path. + oracleReplayTest(fcTest).prop([routeValuesArbitrary], { + ...options, + seed: 1_658_013, + })(`${name} (fixed seed)`, run) + oracleReplayTest(fcTest, property).prop([routeValuesArbitrary], options)( + `${name} (random seed)`, run, ) - fcTest.prop([routeValuesArbitrary], options)(`${name} (random seed)`, run) } describe(`optimistic relationship-transition oracle`, () => { diff --git a/packages/db/tests/query/includes-oracle.property.test.ts b/packages/db/tests/query/includes-oracle.property.test.ts index d9da48f4ab..5ea782c83f 100644 --- a/packages/db/tests/query/includes-oracle.property.test.ts +++ b/packages/db/tests/query/includes-oracle.property.test.ts @@ -9,7 +9,12 @@ import { toArray, } from '../../src/query/index.js' import { flushPromises, withExpectedRejection } from '../utils.js' -import { oraclePropertyOptions, oracleRuns } from '../oracle-config.js' +import { + oraclePropertyOptions, + oracleReplayRequested, + oracleReplaySkips, + oracleRuns, +} from '../oracle-config.js' import { runTrace } from '../trace-runner.js' import { createControlledCollection as createOracleControlledCollection } from './includes-oracle-helpers.js' import { @@ -214,11 +219,10 @@ const depthArbitrary = fc.constantFrom(1, 2, 3, 4) // a fresh seed at the same budget. A seed-and-path replay runs only its named // property; other matrix cells cannot delay or obscure that reproduction. function generatedCampaigns(runs: number, property: string, fixedSeed: number) { - const replayProperty = process.env.TANSTACK_DB_ORACLE_PROPERTY - if (process.env.TANSTACK_DB_ORACLE_PATH !== undefined) { - return replayProperty === property - ? [{ label: `replay`, options: oraclePropertyOptions(runs, property) }] - : [] + if (oracleReplayRequested()) { + return oracleReplaySkips(property) + ? [] + : [{ label: `replay`, options: oraclePropertyOptions(runs, property) }] } return [ @@ -4986,7 +4990,6 @@ describe(`includes recompute oracle`, () => { ) { continue } - for (const sourceBranch of [0, 1] as const) { const property = `includes.matrix.transition-${depth}-${firstTransition}-${secondTransition}-${sourceBranch}` const arbitrary = transitionHistoryScenariosArbitrary( diff --git a/packages/db/tests/query/includes-temporal-oracle.test.ts b/packages/db/tests/query/includes-temporal-oracle.test.ts index f146360acd..5e7b182af1 100644 --- a/packages/db/tests/query/includes-temporal-oracle.test.ts +++ b/packages/db/tests/query/includes-temporal-oracle.test.ts @@ -19,8 +19,8 @@ import { import { runTrace } from '../trace-runner.js' import { oraclePropertyOptions, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../oracle-config.js' import { captureMicrotaskErrors, flushPromises } from '../utils.js' import type { Collection } from '../../src/collection/index.js' @@ -3343,19 +3343,15 @@ async function expectLaggingSourceTraceMatches( // Each generated lane uses one grammar and one check in both campaigns. A // seed-and-path replay selects only its requested random lane. function temporalCampaigns(property: string, fixedSeed: number) { - const replay = readOracleRunConfig() return [ { label: `fixed seed ${fixedSeed}`, - test: replay.replayPath === undefined ? fcTest : fcTest.skip, + test: oracleReplayTest(fcTest), options: { numRuns: oracleRuns(20), seed: fixedSeed }, }, { label: `random or replayed seed`, - test: - replay.replayPath === undefined || replay.replayProperty === property - ? fcTest - : fcTest.skip, + test: oracleReplayTest(fcTest, property), options: oraclePropertyOptions(20, property), }, ] diff --git a/packages/db/tests/query/ir-stable-identity-oracle.test.ts b/packages/db/tests/query/ir-stable-identity-oracle.test.ts index f475760351..297bc50f60 100644 --- a/packages/db/tests/query/ir-stable-identity-oracle.test.ts +++ b/packages/db/tests/query/ir-stable-identity-oracle.test.ts @@ -53,6 +53,7 @@ import { getRuntimeReferenceIdentity, } from '../../src/query/runtime-reference-identity.js' import { createValueIdentity } from '../../src/query/equality-value-identity.js' +import { oracleRuns } from '../oracle-config.js' import type { BasicExpression, OrderBy, QueryIR } from '../../src/query/ir.js' import type { LoadSubsetOptions } from '../../src/types.js' @@ -1539,7 +1540,7 @@ describe(`stable QueryIR identity smoke test`, () => { `matches output denotation through composed contexts with a fixed seed`, (program) => assertOutputProgram(observeOutputProgram(program)), ) - fcTest.prop([outputProgramArbitrary], { numRuns: 200 })( + fcTest.prop([outputProgramArbitrary], { numRuns: oracleRuns(200) })( `matches output denotation through random composed contexts`, (program) => assertOutputProgram(observeOutputProgram(program)), ) diff --git a/packages/db/tests/query/join-result-key-oracle.property.test.ts b/packages/db/tests/query/join-result-key-oracle.property.test.ts index 7fcfafc4e3..faec3dd771 100644 --- a/packages/db/tests/query/join-result-key-oracle.property.test.ts +++ b/packages/db/tests/query/join-result-key-oracle.property.test.ts @@ -46,13 +46,13 @@ import { createCollection } from '../../src/collection/index.js' import { createLiveQueryCollection, eq } from '../../src/query/index.js' import { oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../oracle-config.js' import { mockSyncCollectionOptions, withOracleCleanup } from '../utils.js' const property = `join-result-key.pairs` -const requestedReplayProperty = readOracleRunConfig().replayProperty type Key = string | number type Row = { id: Key; g: number } @@ -260,7 +260,10 @@ async function runHistory(history: History): Promise { } describe(`joined result key oracle`, () => { - if (requestedReplayProperty === undefined) { + // The pinned histories and the fixed campaign stand aside during any path + // replay. The random campaign below is the one that a replay of this + // property selects. + if (!oracleReplayRequested()) { for (const history of pinnedHistories) { it(`keeps every ${history.join} join pair for ${history.left .map((row) => @@ -273,17 +276,10 @@ describe(`joined result key oracle`, () => { seed: 44_501_962, numRuns: oracleRuns(80), })(`publishes one row per joined pair (fixed)`, runHistory) - - fcTest.prop([historyArbitrary], oraclePropertyOptions(80, property))( - `publishes one row per joined pair (random)`, - runHistory, - ) - } else if (requestedReplayProperty === property) { - fcTest.prop([historyArbitrary], oraclePropertyOptions(80, property))( - `publishes one row per joined pair (replay)`, - runHistory, - ) - } else { - it.skip(`runs only when its replay property is selected`, () => {}) } + + oracleReplayTest(fcTest, property).prop( + [historyArbitrary], + oraclePropertyOptions(80, property), + )(`publishes one row per joined pair (random)`, runHistory) }) diff --git a/packages/db/tests/query/pooled-live-query-oracle.property.test.ts b/packages/db/tests/query/pooled-live-query-oracle.property.test.ts index 694f1ab8a1..2fd7e2fc5e 100644 --- a/packages/db/tests/query/pooled-live-query-oracle.property.test.ts +++ b/packages/db/tests/query/pooled-live-query-oracle.property.test.ts @@ -80,8 +80,9 @@ import { createPooledLiveQuery } from '../../src/query/pooled-live-query.js' import { Func, PropRef, Value } from '../../src/query/ir.js' import { oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../oracle-config.js' import { flushPromises, @@ -92,7 +93,6 @@ import { import type { ChangeMessage } from '../../src/types.js' const property = `pooled-live-query.publication` -const requestedReplayProperty = readOracleRunConfig().replayProperty const MISSING = Symbol(`missing`) type FieldValue = string | number | boolean | Date | null | typeof MISSING @@ -896,7 +896,10 @@ async function runHistory(history: History): Promise { } describe(`pooled live query oracle`, () => { - if (requestedReplayProperty === undefined) { + // The pinned histories and the fixed campaign stand aside during any path + // replay. The random campaign below is the one that a replay of this + // property selects. + if (!oracleReplayRequested()) { // Generated fields are plain values; this pins a field whose getter // throws, which both paths treat as a row that does not match. it(`matches the live-query Collection when an eq path getter throws`, async () => { @@ -942,16 +945,10 @@ describe(`pooled live query oracle`, () => { seed: 44_502_001, numRuns: oracleRuns(80), })(`matches the live-query Collection (fixed)`, runHistory) - fcTest.prop([historyArbitrary], oraclePropertyOptions(80, property))( - `matches the live-query Collection (random)`, - runHistory, - ) - } else if (requestedReplayProperty === property) { - fcTest.prop([historyArbitrary], oraclePropertyOptions(80, property))( - `matches the live-query Collection (replay)`, - runHistory, - ) - } else { - it.skip(`runs only when its replay property is selected`, () => {}) } + + oracleReplayTest(fcTest, property).prop( + [historyArbitrary], + oraclePropertyOptions(80, property), + )(`matches the live-query Collection (random)`, runHistory) }) diff --git a/packages/db/tests/query/where-predicate-publication-oracle.property.test.ts b/packages/db/tests/query/where-predicate-publication-oracle.property.test.ts index 67f05b11e7..e1740dd552 100644 --- a/packages/db/tests/query/where-predicate-publication-oracle.property.test.ts +++ b/packages/db/tests/query/where-predicate-publication-oracle.property.test.ts @@ -92,8 +92,9 @@ import { Func, PropRef, Value } from '../../src/query/ir.js' import { createLiveQueryCollection } from '../../src/query/index.js' import { oraclePropertyOptions, + oracleReplayRequested, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../oracle-config.js' import { mockSyncCollectionOptions, @@ -104,7 +105,6 @@ import type { BasicExpression } from '../../src/query/ir.js' import type { ChangeMessage } from '../../src/types.js' const property = `where-predicate.publication` -const requestedReplayProperty = readOracleRunConfig().replayProperty // A string whose content equals an internal normalized-key prefix must still // compare as an ordinary string. @@ -863,7 +863,10 @@ function describePredicate(predicate: Predicate): string { // --------------------------------------------------------------------------- describe(`WHERE predicate publication oracle`, () => { - if (requestedReplayProperty === undefined) { + // The pinned histories and the fixed campaign stand aside during any path + // replay. The random campaign below is the one that a replay of this + // property selects. + if (!oracleReplayRequested()) { it(`pinned snapshot histories distinguish Kleene logic from two-valued logic`, () => { // A checker that folds UNKNOWN into FALSE before NOT must disagree with // the model on each pinned snapshot, or the history does not calibrate. @@ -1116,17 +1119,10 @@ describe(`WHERE predicate publication oracle`, () => { seed: 44_500_301, numRuns: oracleRuns(80), })(`publishes exactly the TRUE rows (fixed)`, runHistory) - - fcTest.prop([historyArbitrary], oraclePropertyOptions(80, property))( - `publishes exactly the TRUE rows (random)`, - runHistory, - ) - } else if (requestedReplayProperty === property) { - fcTest.prop([historyArbitrary], oraclePropertyOptions(80, property))( - `publishes exactly the TRUE rows (replay)`, - runHistory, - ) - } else { - it.skip(`runs only when its replay property is selected`, () => {}) } + + oracleReplayTest(fcTest, property).prop( + [historyArbitrary], + oraclePropertyOptions(80, property), + )(`publishes exactly the TRUE rows (random)`, runHistory) }) diff --git a/packages/db/tests/oracle-replay.fixture.test.ts b/packages/db/tests/replay-guard-calibration.fixture.test.ts similarity index 65% rename from packages/db/tests/oracle-replay.fixture.test.ts rename to packages/db/tests/replay-guard-calibration.fixture.test.ts index 23e3edb958..aac54ee832 100644 --- a/packages/db/tests/oracle-replay.fixture.test.ts +++ b/packages/db/tests/replay-guard-calibration.fixture.test.ts @@ -8,7 +8,9 @@ import { oraclePropertyOptions } from './oracle-config.js' * Ordinary discovery runs one real positive property and one unrelated test. * A child process may inject exactly one fault: setup, property, unrelated test, * skip, zero runs, expected failure, precondition exhaustion, or wrong replay - * coordinates. The parent uses these variants to prove it neither accepts a + * coordinates. The `sync` variant registers the property as a synchronous + * fast-check property, which the guarded runner witnesses in its synchronous + * mode. The parent uses these variants to prove it neither accepts a * false-green run nor rewrites fast-check's real failure. */ const fault = process.env.TANSTACK_DB_ORACLE_REPLAY_CALIBRATION @@ -31,15 +33,26 @@ fcTest.prop([fc.constant(1)], { numRuns: 1, seed: 20260930 })( process.stdout.write(`unrelated fixed campaign ran\n`) }, ) -propertyTest.prop([fc.integer({ min: 0, max: 100 })], options)( - `executes the replay calibration property`, - (value) => { - if (fault === `precondition`) fc.pre(false) - if (fault === `property` || fault === `expected`) - throw new Error(`replay property sentinel`) - expect(value + 1).toBeGreaterThan(value) - }, -) +if (fault === `sync`) { + it(`executes the replay calibration property`, () => { + fc.assert( + fc.property(fc.integer({ min: 0, max: 100 }), (value) => { + expect(value + 1).toBeGreaterThan(value) + }), + options, + ) + }) +} else { + propertyTest.prop([fc.integer({ min: 0, max: 100 })], options)( + `executes the replay calibration property`, + (value) => { + if (fault === `precondition`) fc.pre(false) + if (fault === `property` || fault === `expected`) + throw new Error(`replay property sentinel`) + expect(value + 1).toBeGreaterThan(value) + }, + ) +} it(`executes an unrelated calibration assertion`, () => { if (fault === `unrelated`) throw new Error(`replay unrelated sentinel`) diff --git a/packages/db/tests/utils-oracle.property.test.ts b/packages/db/tests/utils-oracle.property.test.ts index 2ad77a4632..050b9bd6cc 100644 --- a/packages/db/tests/utils-oracle.property.test.ts +++ b/packages/db/tests/utils-oracle.property.test.ts @@ -2,6 +2,11 @@ import { describe, expect, it } from 'vitest' import { fc, test as fcTest } from '@fast-check/vitest' import { Temporal } from 'temporal-polyfill' import { deepEquals, equalPersistedSnapshotValues } from '../src/utils' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from './oracle-config.js' /** * The `deepEquals` API contract in `../src/utils.ts`, reinforced by the @@ -24,61 +29,35 @@ import { deepEquals, equalPersistedSnapshotValues } from '../src/utils' * reference relation does not define. */ // Every generated law below uses the same property in a fixed and a fresh -// campaign. Set UTILS_ORACLE_PROPERTY to the numbered test label and provide -// UTILS_ORACLE_SEED and UTILS_ORACLE_PATH to replay its fast-check shrink -// directly, for example with `vitest run tests/utils-oracle.property.test.ts`. +// campaign. Set TANSTACK_DB_ORACLE_PROPERTY to `utils.` from the test +// label, with TANSTACK_DB_ORACLE_SEED and TANSTACK_DB_ORACLE_PATH, to replay +// its fast-check shrink directly, for example with +// `vitest run tests/utils-oracle.property.test.ts`. const fixedSeed = 20260911 const campaignRuns = 100 -const requestedProperty = process.env.UTILS_ORACLE_PROPERTY -const replaySeed = process.env.UTILS_ORACLE_SEED -const replayPath = process.env.UTILS_ORACLE_PATH +// Each property registers as `utils.`, in file order. A path replay +// selects one with TANSTACK_DB_ORACLE_PROPERTY=utils.. let nextPropertyId = 0 -if ( - requestedProperty === undefined && - (replaySeed !== undefined || replayPath !== undefined) -) { - throw new Error(`utils oracle replay needs a property number`) -} - function utilsProperty]>(arbitraries: { [K in keyof Ts]: fc.Arbitrary }) { return (name: string, check: (...values: Ts) => void): void => { - const id = String(++nextPropertyId) + const id = ++nextPropertyId + const property = `utils.${id}` const label = `${id}: ${name}` // fast-check's variadic tuple overload widens a generic tuple to any[]. // The mapped arbitraries preserve each position's Ts member. const argumentsArbitrary = fc.tuple(...arbitraries) as fc.Arbitrary - if (requestedProperty !== undefined) { - if (requestedProperty !== id) { - it.skip(`${label} skipped for oracle replay`, () => {}) - return - } - if (replaySeed === undefined || replayPath === undefined) { - throw new Error(`utils oracle replay needs a seed and shrink path`) - } - const seed = Number(replaySeed) - if (!Number.isSafeInteger(seed)) { - throw new Error(`utils oracle replay seed must be an integer`) - } - fcTest.prop([argumentsArbitrary], { - seed, - path: replayPath, - numRuns: campaignRuns, - })(`${label} replay`, (values) => check(...values)) - return - } - - fcTest.prop([argumentsArbitrary], { + oracleReplayTest(fcTest).prop([argumentsArbitrary], { seed: fixedSeed, - numRuns: campaignRuns, + numRuns: oracleRuns(campaignRuns), })(`${label} fixed`, (values) => check(...values)) - fcTest.prop([argumentsArbitrary], { numRuns: campaignRuns })( - `${label} random`, - (values) => check(...values), - ) + oracleReplayTest(fcTest, property).prop( + [argumentsArbitrary], + oraclePropertyOptions(campaignRuns, property), + )(`${label} random`, (values) => check(...values)) } } @@ -1199,7 +1178,8 @@ describe(`deepEquals property-based tests`, () => { }) // Keep newly added generated laws after the original registration order: - // UTILS_ORACLE_PROPERTY uses ordinal selectors for saved seed/path replays. + // `utils.` names properties in registration order, so saved + // seed/path replays keep their numbers. describe(`additional non-ring value boundaries`, () => { utilsProperty([arbitraryRegExp])( `RegExp source and flags determine equality`, @@ -1339,13 +1319,3 @@ describe(`persisted-snapshot prototype equality`, () => { ).toBe(false) }) }) - -if (requestedProperty !== undefined) { - it(`selects a known utils oracle property`, () => { - const id = Number(requestedProperty) - expect( - Number.isSafeInteger(id) && id >= 1 && id <= nextPropertyId, - `unknown utils oracle property: ${requestedProperty}`, - ).toBe(true) - }) -} diff --git a/packages/electric-db-collection/tests/electric-descriptor-isolation-oracle.test.ts b/packages/electric-db-collection/tests/electric-descriptor-isolation-oracle.test.ts index 6a4507b50a..4ddecf84e0 100644 --- a/packages/electric-db-collection/tests/electric-descriptor-isolation-oracle.test.ts +++ b/packages/electric-db-collection/tests/electric-descriptor-isolation-oracle.test.ts @@ -15,8 +15,8 @@ import { electricCollectionOptions } from '../src/electric' import { StreamAbortedError } from '../src/errors' import { oraclePropertyOptions, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../../db/tests/oracle-config' import { atCheckpoint } from './electric-oracle-lifecycle' import { storageTargetId, tagPersistence } from './electric-persistence-fixture' @@ -51,9 +51,9 @@ import type { ElectricCollectionUtils } from '../src/electric' * framing or service behavior; those have separate owners. */ -const { replayPath, replayProperty } = readOracleRunConfig() -const fixedCase = replayPath === undefined ? it : it.skip -const fixedDescribe = replayPath === undefined ? describe : describe.skip +// Ordinary tests stand aside during any path replay, which runs one property. +const fixedCase = oracleReplayTest(it) +const fixedDescribe = oracleReplayTest(describe) // The durable fixture is a driver, not the independent tag model. It must // preserve the SQLite adapter's mutation algebra: an insert replaces the old @@ -607,32 +607,25 @@ fixedCase.each([ }, ) -// The two campaigns use the same grammar, checker, and budget. An explicit -// replay registers only its selected property. Reproduce a saved failure with: +// The two campaigns use the same grammar, checker, and budget. A path replay +// of the property runs only the random campaign. Reproduce a saved failure with: // TANSTACK_DB_ORACLE_PROPERTY=electric.persisted-tag-history // TANSTACK_DB_ORACLE_SEED= TANSTACK_DB_ORACLE_PATH= // pnpm exec vitest run tests/electric-descriptor-isolation-oracle.test.ts -if (replayPath === undefined) { - fcTest.prop([tagHistory], { seed: 42712, numRuns: oracleRuns(10) })( - `persisted tag histories preserve membership across warm and cold restart (fixed)`, - (history) => runTagHistory(history), - ) - fcTest.prop( - [tagHistory], - oraclePropertyOptions(10, `electric.persisted-tag-history`), - )( - `persisted tag histories preserve membership across warm and cold restart (random)`, - (history) => runTagHistory(history), - ) -} else if (replayProperty === `electric.persisted-tag-history`) { - fcTest.prop( - [tagHistory], - oraclePropertyOptions(10, `electric.persisted-tag-history`), - )( - `persisted tag histories preserve membership across warm and cold restart (replay)`, - (history) => runTagHistory(history), - ) -} +oracleReplayTest(fcTest).prop([tagHistory], { + seed: 42712, + numRuns: oracleRuns(10), +})( + `persisted tag histories preserve membership across warm and cold restart (fixed)`, + (history) => runTagHistory(history), +) +oracleReplayTest(fcTest, `electric.persisted-tag-history`).prop( + [tagHistory], + oraclePropertyOptions(10, `electric.persisted-tag-history`), +)( + `persisted tag histories preserve membership across warm and cold restart (random)`, + (history) => runTagHistory(history), +) beforeEach(() => { streams.length = 0 @@ -1748,27 +1741,20 @@ async function runOwnerHistory(history: { } } -if (replayPath === undefined) { - fcTest.prop([ownerHistory], { seed: 42711, numRuns: oracleRuns(12) })( - `config derivation preserves independent owner histories (fixed)`, - runOwnerHistory, - ) - fcTest.prop( - [ownerHistory], - oraclePropertyOptions(20, `electric.bound-descriptor-history`), - )( - `config derivation preserves independent owner histories (random)`, - runOwnerHistory, - ) -} else if (replayProperty === `electric.bound-descriptor-history`) { - fcTest.prop( - [ownerHistory], - oraclePropertyOptions(20, `electric.bound-descriptor-history`), - )( - `config derivation preserves independent owner histories (replay)`, - runOwnerHistory, - ) -} +oracleReplayTest(fcTest).prop([ownerHistory], { + seed: 42711, + numRuns: oracleRuns(12), +})( + `config derivation preserves independent owner histories (fixed)`, + runOwnerHistory, +) +oracleReplayTest(fcTest, `electric.bound-descriptor-history`).prop( + [ownerHistory], + oraclePropertyOptions(20, `electric.bound-descriptor-history`), +)( + `config derivation preserves independent owner histories (random)`, + runOwnerHistory, +) // The Electric descriptor may be reused across independent persisted wrappers. // The wrappers may address the same or different durable Collection IDs; that diff --git a/packages/electric-db-collection/tests/electric-oracle.property.test.ts b/packages/electric-db-collection/tests/electric-oracle.property.test.ts index fa62022c93..7eb9e01591 100644 --- a/packages/electric-db-collection/tests/electric-oracle.property.test.ts +++ b/packages/electric-db-collection/tests/electric-oracle.property.test.ts @@ -17,8 +17,8 @@ import { queryCollectionOptions } from '../../query-db-collection/src/query' import { electricCollectionOptions } from '../src/electric' import { oraclePropertyOptions, + oracleReplayTest, oracleRuns, - readOracleRunConfig, } from '../../db/tests/oracle-config' import { atCheckpoint, withElectricCleanup } from './electric-oracle-lifecycle' import { @@ -43,46 +43,12 @@ import type { RetryWait, } from './electric-retry-decision-model' -// Replays one generated law directly with -// TANSTACK_DB_ELECTRIC_HISTORY_PROPERTY= -// TANSTACK_DB_ELECTRIC_HISTORY_SEED= -// TANSTACK_DB_ELECTRIC_HISTORY_PATH= +// Replays one generated law directly with the shared oracle variables: +// TANSTACK_DB_ORACLE_PROPERTY= +// TANSTACK_DB_ORACLE_SEED= +// TANSTACK_DB_ORACLE_PATH= // pnpm exec vitest run tests/electric-oracle.property.test.ts -t ' \(replay\)' -const historyReplayProperty = process.env.TANSTACK_DB_ELECTRIC_HISTORY_PROPERTY -const historyReplaySeed = process.env.TANSTACK_DB_ELECTRIC_HISTORY_SEED -const historyReplayPath = process.env.TANSTACK_DB_ELECTRIC_HISTORY_PATH const registeredHistoryProperties = new Set() -const historyPropertyIds = new Set([ - `electric.initial-tagged-move-outs`, - `electric.process-grammar`, - `electric.design-convergence`, - `electric.synthetic-batch-partition`, - `electric.valid-batch-partition`, - `electric.invalid-resume-partition`, - `electric.subset-request-row-validity`, - `electric.resume-metadata-merge`, - `electric.metadata-merge-complete-state`, - `electric.retry-decision`, -]) - -if ( - historyReplayProperty === undefined && - (historyReplaySeed !== undefined || historyReplayPath !== undefined) -) { - throw new Error(`Electric history replay requires a property`) -} -if ( - historyReplayProperty !== undefined && - (historyReplaySeed === undefined || historyReplayPath === undefined) -) { - throw new Error(`Electric history replay requires both seed and path`) -} -if ( - historyReplayProperty !== undefined && - !historyPropertyIds.has(historyReplayProperty) -) { - throw new Error(`unknown Electric history property: ${historyReplayProperty}`) -} function electricHistoryProperty( property: string, @@ -98,35 +64,17 @@ function electricHistoryProperty( throw new Error(`duplicate Electric history property: ${property}`) } registeredHistoryProperties.add(property) - const numRuns = oracleRuns(baseRuns) - if (historyReplayProperty === undefined) { - fcTest.prop([arbitrary], { numRuns, seed: fixedSeed, examples })( - `${name} (fixed)`, - check, - timeout, - ) - fcTest.prop([arbitrary], { numRuns, examples })( - `${name} (random)`, - check, - timeout, - ) - return - } - if (historyReplayProperty !== property) return - const seed = Number(historyReplaySeed) - if (!Number.isSafeInteger(seed)) { - throw new Error(`Electric history replay seed must be an integer`) - } - if (!/^\d+(?::\d+)*$/.test(historyReplayPath!)) { - throw new Error( - `Electric history replay path must contain colon-separated nonnegative integers`, - ) - } - fcTest.prop([arbitrary], { numRuns, seed, path: historyReplayPath })( - `${name} (replay)`, - check, - timeout, - ) + // A path replay of this property runs only the random campaign, from its + // path. The fixed campaign stands aside during every path replay. + oracleReplayTest(fcTest).prop([arbitrary], { + numRuns: oracleRuns(baseRuns), + seed: fixedSeed, + examples, + })(`${name} (fixed)`, check, timeout) + oracleReplayTest(fcTest, property).prop([arbitrary], { + ...oraclePropertyOptions(baseRuns, property), + examples, + })(`${name} (random)`, check, timeout) } /** @@ -383,58 +331,29 @@ function foldSourceLedgerThrough( throw new Error(`resume offset ${resumeOffset} is absent from source ledger`) } +const durablePrefixProperty = `electric.durable-prefix` +const publicationBeforePersistenceProperty = `electric.publication-before-persistence` +const adapterFailurePolicyProperty = `electric.adapter-failure-policy` + function durablePrefixPropertyOptions(mode: `fixed` | `random`): { numRuns: number seed?: number path?: string } { - const numRuns = oracleRuns(10) - if (mode === `fixed`) return { numRuns, seed: 20260921 } - const seedValue = process.env.TANSTACK_DB_ELECTRIC_DURABILITY_SEED - const path = process.env.TANSTACK_DB_ELECTRIC_DURABILITY_PATH - if (path !== undefined && seedValue === undefined) { - throw new Error( - `TANSTACK_DB_ELECTRIC_DURABILITY_PATH requires TANSTACK_DB_ELECTRIC_DURABILITY_SEED`, - ) - } - if (seedValue === undefined) return { numRuns } - const seed = Number(seedValue) - if (!Number.isSafeInteger(seed)) { - throw new Error(`TANSTACK_DB_ELECTRIC_DURABILITY_SEED must be an integer`) - } - return { - numRuns, - seed, - ...(path === undefined ? {} : { path }), - } + if (mode === `fixed`) return { numRuns: oracleRuns(10), seed: 20260921 } + return oraclePropertyOptions(10, durablePrefixProperty) } -function persistencePolicyPropertyOptions(mode: `fixed` | `random`): { +function persistencePolicyPropertyOptions( + mode: `fixed` | `random`, + property: string, +): { numRuns: number seed?: number path?: string } { - const numRuns = oracleRuns(8) - if (mode === `fixed`) return { numRuns, seed: 20260923 } - const seedValue = process.env.TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_SEED - const path = process.env.TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_PATH - if (path !== undefined && seedValue === undefined) { - throw new Error( - `TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_PATH requires TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_SEED`, - ) - } - if (seedValue === undefined) return { numRuns } - const seed = Number(seedValue) - if (!Number.isSafeInteger(seed)) { - throw new Error( - `TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_SEED must be an integer`, - ) - } - return { - numRuns, - seed, - ...(path === undefined ? {} : { path }), - } + if (mode === `fixed`) return { numRuns: oracleRuns(8), seed: 20260923 } + return oraclePropertyOptions(8, property) } async function runRejectedDurablePrefixWitness( @@ -711,48 +630,20 @@ const queuedPresenceArbitraries: [ }), fc.tuple(fc.string({ maxLength: 12 }), fc.string({ maxLength: 12 })), ] -const requestedQueuedPresenceProperty = - process.env.TANSTACK_DB_ELECTRIC_ORACLE_PROPERTY const persistenceInterleavingProperty = `electric.persistence-interleaving` -const requestedPersistenceInterleavingProperty = - readOracleRunConfig().replayProperty - -if ( - requestedQueuedPresenceProperty !== undefined && - requestedQueuedPresenceProperty !== queuedPresenceProperty -) { - throw new Error( - `unknown Electric oracle replay property: ${requestedQueuedPresenceProperty}`, - ) -} -function queuedPresencePropertyOptions(mode: `fixed` | `random` | `replay`): { +function queuedPresencePropertyOptions(mode: `fixed` | `random`): { numRuns: number seed?: number path?: string } { if (mode === `fixed`) { - return { numRuns: queuedPresenceRuns, seed: queuedPresenceFixedSeed } - } - if (mode === `random`) return { numRuns: queuedPresenceRuns } - - const seedValue = process.env.TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_SEED - const path = process.env.TANSTACK_DB_ELECTRIC_PERSISTENCE_POLICY_PATH - if (seedValue === undefined || path === undefined || path.trim() === ``) { - throw new Error( - `Electric queued-presence replay requires both seed and path`, - ) - } - const seed = Number(seedValue) - if (!Number.isSafeInteger(seed)) { - throw new Error(`Electric queued-presence replay seed must be an integer`) - } - if (!/^\d+(?::\d+)*$/.test(path)) { - throw new Error( - `Electric queued-presence replay path must contain colon-separated nonnegative integers`, - ) + return { + numRuns: oracleRuns(queuedPresenceRuns), + seed: queuedPresenceFixedSeed, + } } - return { numRuns: queuedPresenceRuns, seed, path } + return oraclePropertyOptions(queuedPresenceRuns, queuedPresenceProperty) } function reconstructQueuedPresenceCampaign( @@ -2866,12 +2757,6 @@ async function runSchedulerPermutation( } } -const describeUnlessQueuedPresenceReplay = - requestedQueuedPresenceProperty || - requestedPersistenceInterleavingProperty === persistenceInterleavingProperty - ? describe.skip - : describe - function resetElectricOracleMocks(): void { vi.clearAllMocks() // clearAllMocks retains one-shot implementations from a failed earlier @@ -2883,7 +2768,9 @@ function resetElectricOracleMocks(): void { mockStream.lastOffset = `20_0` } -describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { +// Each registration below says which path replays stand it aside. A path +// replay runs only the selected property, and the sibling suites register it. +describe(`Electric adapter laws`, () => { let processGrammarRun = 0 let persistencePolicyRun = 0 @@ -2891,7 +2778,7 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { resetElectricOracleMocks() }) - it.each([false, true])( + oracleReplayTest(it).each([false, true])( `cleans direct traces after delivery failure and preserves the primary error (cleanup failure: %s)`, async (failCleanup) => { const primary = new Error(`delivery sentinel`) @@ -2929,7 +2816,7 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it.each( + oracleReplayTest(it).each( ([`eager`, `on-demand`, `progressive`] as const).flatMap((syncMode) => [0, 1, 2].map((removals) => ({ syncMode, removals })), ), @@ -2968,42 +2855,45 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it(`delivers retired callbacks without advancing a failed lifecycle`, async () => { - const result = await runProcessGrammar(`process-grammar-terminal-error`, [ - { - kind: `batch`, - slot: `a`, - operation: `update`, - id: 1, - name: `unseen`, - txid: 55, - }, - { kind: `snapshot`, slot: `a`, id: 1, name: `stale callback` }, - { - kind: `retired-callback`, - slot: `a`, - id: 1, - name: `actually delivered`, - }, - ]) - expect( - result.reached - .filter(({ command }) => command.kind === `retired-callback`) - .map(({ result: reached }) => reached), - ).toEqual([ - `delivered: retired callback`, - `delivered: retired callback`, - `delivered: retired callback`, - ]) - expect( - result.reached.find( - ({ command }) => - command.kind === `snapshot` && command.name === `stale callback`, - )?.result, - ).toBe(`skipped: terminal error`) - }) + oracleReplayTest(it)( + `delivers retired callbacks without advancing a failed lifecycle`, + async () => { + const result = await runProcessGrammar(`process-grammar-terminal-error`, [ + { + kind: `batch`, + slot: `a`, + operation: `update`, + id: 1, + name: `unseen`, + txid: 55, + }, + { kind: `snapshot`, slot: `a`, id: 1, name: `stale callback` }, + { + kind: `retired-callback`, + slot: `a`, + id: 1, + name: `actually delivered`, + }, + ]) + expect( + result.reached + .filter(({ command }) => command.kind === `retired-callback`) + .map(({ result: reached }) => reached), + ).toEqual([ + `delivered: retired callback`, + `delivered: retired callback`, + `delivered: retired callback`, + ]) + expect( + result.reached.find( + ({ command }) => + command.kind === `snapshot` && command.name === `stale callback`, + )?.result, + ).toBe(`skipped: terminal error`) + }, + ) - it.each([`invalid-resume`, `cleanup`] as const)( + oracleReplayTest(it).each([`invalid-resume`, `cleanup`] as const)( `records an initial preload's %s rejection before continuing the grammar`, async (failure) => { const result = await runProcessGrammar(`preload-${failure}`, [ @@ -3049,18 +2939,21 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it(`preserves unexpected preload failures instead of treating observation as success`, async () => { - const failure = new Error(`test-owned preload failure`) - const observed = observePreload(Promise.reject(failure)) - const outcome = await observed.outcome - expect(outcome).toEqual({ status: `rejected`, reason: failure }) - if (outcome.status === `rejected`) expect(outcome.reason).toBe(failure) - expect(() => expectPreloadOutcome(observed, outcome)).toThrow(failure) - const success = observePreload(Promise.resolve()) - expectPreloadOutcome(success, await success.outcome) - }) + oracleReplayTest(it)( + `preserves unexpected preload failures instead of treating observation as success`, + async () => { + const failure = new Error(`test-owned preload failure`) + const observed = observePreload(Promise.reject(failure)) + const outcome = await observed.outcome + expect(outcome).toEqual({ status: `rejected`, reason: failure }) + if (outcome.status === `rejected`) expect(outcome.reason).toBe(failure) + expect(() => expectPreloadOutcome(observed, outcome)).toThrow(failure) + const success = observePreload(Promise.resolve()) + expectPreloadOutcome(success, await success.outcome) + }, + ) - it.each([`reset`, `delete`, `move-out`] as const)( + oracleReplayTest(it).each([`reset`, `delete`, `move-out`] as const)( `keeps $0 removal authoritative across an optimistic write and a new acquisition`, async (removal) => { const trace = createOracleCollection( @@ -3121,217 +3014,232 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it(`publishes an Electric source commit before its adapter persistence settles`, async () => { - await runPublicationBeforePersistenceWitness( - `publication-before-persistence-fixed`, - { id: 41, name: `visible first`, stable: `stable-41` }, - 741, - ) - }) + oracleReplayTest(it)( + `publishes an Electric source commit before its adapter persistence settles`, + async () => { + await runPublicationBeforePersistenceWitness( + `publication-before-persistence-fixed`, + { id: 41, name: `visible first`, stable: `stable-41` }, + 741, + ) + }, + ) - it(`publishes later Electric callbacks while an earlier durability turn is held`, async () => { - let subscriber: ((messages: Array>) => void) | undefined - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const firstPersistenceEntered = createDeferred() - const releaseFirstPersistence = createDeferred() - const persistedMetadata = new Map() - const persistedRows = new Map() - const adapter = createPersistedAdapter(persistedMetadata, persistedRows) - const applyCommittedTx = adapter.applyCommittedTx.bind(adapter) - let heldFirstRow = false - let durabilityCalls = 0 - adapter.applyCommittedTx = async (...args) => { - durabilityCalls++ - if ( - !heldFirstRow && - args[1].mutations.some((mutation) => mutation.key === 1) - ) { - heldFirstRow = true - firstPersistenceEntered.resolve() - await releaseFirstPersistence.promise + oracleReplayTest(it)( + `publishes later Electric callbacks while an earlier durability turn is held`, + async () => { + let subscriber: + ((messages: Array>) => void) | undefined + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const firstPersistenceEntered = createDeferred() + const releaseFirstPersistence = createDeferred() + const persistedMetadata = new Map() + const persistedRows = new Map() + const adapter = createPersistedAdapter(persistedMetadata, persistedRows) + const applyCommittedTx = adapter.applyCommittedTx.bind(adapter) + let heldFirstRow = false + let durabilityCalls = 0 + adapter.applyCommittedTx = async (...args) => { + durabilityCalls++ + if ( + !heldFirstRow && + args[1].mutations.some((mutation) => mutation.key === 1) + ) { + heldFirstRow = true + firstPersistenceEntered.resolve() + await releaseFirstPersistence.promise + } + await applyCommittedTx(...args) } - await applyCommittedTx(...args) - } - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricCollectionOptions({ - id: `queued-persisted-presence`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - syncMode: `progressive`, - getKey: (row) => row.id, - startSync: true, + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricCollectionOptions({ + id: `queued-persisted-presence`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + syncMode: `progressive`, + getKey: (row) => row.id, + startSync: true, + }), + persistence: { adapter }, }), - persistence: { adapter }, - }), - ) - const publications = observePublications(collection) - - await withElectricCleanup(async () => { - await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`), { - interval: 1, - timeout: 250, - }) - mockStream.lastOffset = `90_0` - subscriber!([upToDate]) - await vi.waitFor(() => expect(collection.status).toBe(`ready`), { - interval: 1, - timeout: 250, - }) - - const first = change(`insert`, 1, `holds persistence`) - first.value = { id: 1, name: `holds persistence`, stable: `stable-1` } - mockStream.lastOffset = `91_0` - subscriber!([first, upToDate]) - await atCheckpoint( - firstPersistenceEntered.promise, - `first Electric persistence entered`, ) - const durabilityCallsAtHold = durabilityCalls + const publications = observePublications(collection) - const queued = change(`insert`, 2, `queued insert`) - queued.value = { id: 2, name: `queued insert`, stable: `stable-2` } - mockStream.lastOffset = `92_0` - subscriber!([queued, upToDate]) + await withElectricCleanup(async () => { + await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`), { + interval: 1, + timeout: 250, + }) + mockStream.lastOffset = `90_0` + subscriber!([upToDate]) + await vi.waitFor(() => expect(collection.status).toBe(`ready`), { + interval: 1, + timeout: 250, + }) - const laterUpdate = change(`update`, 2, `later update`) - laterUpdate.value = { id: 2, name: `later update`, stable: `stable-2` } - mockStream.lastOffset = `93_0` - subscriber!([laterUpdate, upToDate]) + const first = change(`insert`, 1, `holds persistence`) + first.value = { id: 1, name: `holds persistence`, stable: `stable-1` } + mockStream.lastOffset = `91_0` + subscriber!([first, upToDate]) + await atCheckpoint( + firstPersistenceEntered.promise, + `first Electric persistence entered`, + ) + const durabilityCallsAtHold = durabilityCalls - const expected = rowsFromMap( - new Map([ - [1, first.value], - [2, laterUpdate.value], - ]), - ) - expect({ - publicRows: rowsFromCollection(collection), - durableRows: rowsFromMap(persistedRows), - noLaterDurabilityStarted: durabilityCalls === durabilityCallsAtHold, - published: publications.entries.some( - ({ cut, rows }) => - cut === `event` && isDeepStrictEqual(rows, expected), - ), - }).toEqual({ - publicRows: rowsFromMap(new Map([[1, first.value]])), - durableRows: [], - noLaterDurabilityStarted: true, - published: false, - }) + const queued = change(`insert`, 2, `queued insert`) + queued.value = { id: 2, name: `queued insert`, stable: `stable-2` } + mockStream.lastOffset = `92_0` + subscriber!([queued, upToDate]) - releaseFirstPersistence.resolve() - await vi.waitFor( - () => - expect(rowsFromMap(persistedRows)).toEqual( - rowsFromMap( - new Map([ - [1, first.value], - [2, laterUpdate.value], - ]), - ), - ), - { interval: 1, timeout: 250 }, - ) - expect(rowsFromCollection(collection)).toEqual( - rowsFromMap( + const laterUpdate = change(`update`, 2, `later update`) + laterUpdate.value = { id: 2, name: `later update`, stable: `stable-2` } + mockStream.lastOffset = `93_0` + subscriber!([laterUpdate, upToDate]) + + const expected = rowsFromMap( new Map([ [1, first.value], [2, laterUpdate.value], ]), - ), - ) - expect(collection.status).toBe(`ready`) - }, [ - () => releaseFirstPersistence.resolve(), - () => publications.stop(), - () => collection.cleanup(), - ]) - }) + ) + expect({ + publicRows: rowsFromCollection(collection), + durableRows: rowsFromMap(persistedRows), + noLaterDurabilityStarted: durabilityCalls === durabilityCallsAtHold, + published: publications.entries.some( + ({ cut, rows }) => + cut === `event` && isDeepStrictEqual(rows, expected), + ), + }).toEqual({ + publicRows: rowsFromMap(new Map([[1, first.value]])), + durableRows: [], + noLaterDurabilityStarted: true, + published: false, + }) - it(`reconstructs the queued-presence grammar and rejects ablated, out-of-range, and foreign campaigns`, () => { - const campaigns = fc.sample(fc.tuple(...queuedPresenceArbitraries), { - seed: queuedPresenceFixedSeed, - numRuns: 20, - }) - for (const [histories, names] of campaigns) { - expect(reconstructQueuedPresenceCampaign({ histories, names })).toEqual({ - histories, - names, + releaseFirstPersistence.resolve() + await vi.waitFor( + () => + expect(rowsFromMap(persistedRows)).toEqual( + rowsFromMap( + new Map([ + [1, first.value], + [2, laterUpdate.value], + ]), + ), + ), + { interval: 1, timeout: 250 }, + ) + expect(rowsFromCollection(collection)).toEqual( + rowsFromMap( + new Map([ + [1, first.value], + [2, laterUpdate.value], + ]), + ), + ) + expect(collection.status).toBe(`ready`) + }, [ + () => releaseFirstPersistence.resolve(), + () => publications.stop(), + () => collection.cleanup(), + ]) + }, + ) + + oracleReplayTest(it)( + `reconstructs the queued-presence grammar and rejects ablated, out-of-range, and foreign campaigns`, + () => { + const campaigns = fc.sample(fc.tuple(...queuedPresenceArbitraries), { + seed: queuedPresenceFixedSeed, + numRuns: 20, }) - } + for (const [histories, names] of campaigns) { + expect(reconstructQueuedPresenceCampaign({ histories, names })).toEqual( + { + histories, + names, + }, + ) + } - const witness = { - histories: [...queuedPresenceHistories], - names: [``, `x`.repeat(12)] as [string, string], - } - expect(reconstructQueuedPresenceCampaign(witness)).toEqual(witness) - expect(() => - reconstructQueuedPresenceCampaign({ names: witness.names }), - ).toThrow() - expect(() => - reconstructQueuedPresenceCampaign({ histories: witness.histories }), - ).toThrow() - expect(() => - reconstructQueuedPresenceCampaign({ - ...witness, - histories: witness.histories.slice(1), - }), - ).toThrow() - expect(() => - reconstructQueuedPresenceCampaign({ - ...witness, - histories: [ - `insert-update`, - `insert-update`, - `delete-update`, - `truncate-update`, - ], - }), - ).toThrow() - expect(() => - reconstructQueuedPresenceCampaign({ - ...witness, - histories: [ - `insert-update`, - `update-update`, - `delete-update`, - `replace-update`, - ], - }), - ).toThrow() - expect(() => - reconstructQueuedPresenceCampaign({ - ...witness, - names: [`x`.repeat(13), ``], - }), - ).toThrow() - }) + const witness = { + histories: [...queuedPresenceHistories], + names: [``, `x`.repeat(12)] as [string, string], + } + expect(reconstructQueuedPresenceCampaign(witness)).toEqual(witness) + expect(() => + reconstructQueuedPresenceCampaign({ names: witness.names }), + ).toThrow() + expect(() => + reconstructQueuedPresenceCampaign({ histories: witness.histories }), + ).toThrow() + expect(() => + reconstructQueuedPresenceCampaign({ + ...witness, + histories: witness.histories.slice(1), + }), + ).toThrow() + expect(() => + reconstructQueuedPresenceCampaign({ + ...witness, + histories: [ + `insert-update`, + `insert-update`, + `delete-update`, + `truncate-update`, + ], + }), + ).toThrow() + expect(() => + reconstructQueuedPresenceCampaign({ + ...witness, + histories: [ + `insert-update`, + `update-update`, + `delete-update`, + `replace-update`, + ], + }), + ).toThrow() + expect(() => + reconstructQueuedPresenceCampaign({ + ...witness, + names: [`x`.repeat(13), ``], + }), + ).toThrow() + }, + ) - it(`rejects named wrong answer: later callback publishes before durability`, () => { - expect(() => - expectQueuedPresenceAtDurabilityHold( - [ - [1, `holds persistence`, `stable-1`], - [2, `later`, `stable-2`], - ], - `insert-update`, - { id: 1, name: `holds persistence`, stable: `stable-1` }, - ), - ).toThrow() - }) + oracleReplayTest(it)( + `rejects named wrong answer: later callback publishes before durability`, + () => { + expect(() => + expectQueuedPresenceAtDurabilityHold( + [ + [1, `holds persistence`, `stable-1`], + [2, `later`, `stable-2`], + ], + `insert-update`, + { id: 1, name: `holds persistence`, stable: `stable-1` }, + ), + ).toThrow() + }, + ) - fcTest.prop( + oracleReplayTest(fcTest).prop( queuedPresenceArbitraries, queuedPresencePropertyOptions(`fixed`), )( @@ -3339,14 +3247,6 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { runQueuedPresenceCampaign, ) - fcTest.prop( - queuedPresenceArbitraries, - queuedPresencePropertyOptions(`random`), - )( - `generated queued Electric histories preserve staged presence and reset fences (random)`, - runQueuedPresenceCampaign, - ) - const publicationPolicyArbitraries: [ fc.Arbitrary, fc.Arbitrary, @@ -3359,29 +3259,37 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { 800 + id, ) } - fcTest.prop( + oracleReplayTest(fcTest).prop( publicationPolicyArbitraries, - persistencePolicyPropertyOptions(`fixed`), + persistencePolicyPropertyOptions( + `fixed`, + publicationBeforePersistenceProperty, + ), )( `generated Electric commits publish before adapter persistence settles (fixed)`, checkPublicationPolicy, ) - // Replay the random lane with the legacy seed/path variables and -t 'random'. - fcTest.prop( + oracleReplayTest(fcTest, publicationBeforePersistenceProperty).prop( publicationPolicyArbitraries, - persistencePolicyPropertyOptions(`random`), + persistencePolicyPropertyOptions( + `random`, + publicationBeforePersistenceProperty, + ), )( `generated Electric commits publish before adapter persistence settles (random)`, checkPublicationPolicy, ) - it(`turns adapter rejection into a named terminal Electric persistence error`, async () => { - await runPersistenceFailureCrashOnlyWitness( - `terminal-persistence-error-fixed`, - { id: 51, name: `published before failure`, stable: `stable-51` }, - { id: 52, name: `must not be admitted`, stable: `stable-52` }, - ) - }) + oracleReplayTest(it)( + `turns adapter rejection into a named terminal Electric persistence error`, + async () => { + await runPersistenceFailureCrashOnlyWitness( + `terminal-persistence-error-fixed`, + { id: 51, name: `published before failure`, stable: `stable-51` }, + { id: 52, name: `must not be admitted`, stable: `stable-52` }, + ) + }, + ) const failurePolicyArbitraries: [ fc.Arbitrary, @@ -3398,16 +3306,16 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { { id: id + 100, name: names[1], stable: `stable-${id + 100}` }, ) } - fcTest.prop( + oracleReplayTest(fcTest).prop( failurePolicyArbitraries, - persistencePolicyPropertyOptions(`fixed`), + persistencePolicyPropertyOptions(`fixed`, adapterFailurePolicyProperty), )( `generated adapter failures reject publicly and admit no later Electric work (fixed)`, checkFailurePolicy, ) - fcTest.prop( + oracleReplayTest(fcTest, adapterFailurePolicyProperty).prop( failurePolicyArbitraries, - persistencePolicyPropertyOptions(`random`), + persistencePolicyPropertyOptions(`random`, adapterFailurePolicyProperty), )( `generated adapter failures reject publicly and admit no later Electric work (random)`, checkFailurePolicy, @@ -3450,147 +3358,163 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it(`distinguishes plausible unseen-update and reset-visibility semantics`, async () => { - const completeUnseenUpdate: ChangeMessage = { - key: `1`, - value: { id: 1, name: `complete`, stable: `stable-1` }, - headers: { operation: `update` }, - } - const updateHistory = [[completeUnseenUpdate, upToDate]] - expect(recomputeCommittedRows(updateHistory, `ignore`)).toEqual([]) - expect(recomputeCommittedRows(updateHistory, `promote-complete`)).toEqual([ - [1, `complete`, `stable-1`], - ]) - const updateActual = await runTrace( - `design-unseen-update`, - `eager`, - [], - updateHistory, - ) - expect(updateActual.rows).toEqual( - recomputeCommittedRows(updateHistory, `ignore`), - ) - - const resetHistory = [ - [change(`insert`, 1, `committed`), upToDate], - [mustRefetch], - ] - const atomicReset = recomputedSnapshots([], resetHistory) - const immediateReset: typeof atomicReset = [atomicReset[0]!, []] - expect(atomicReset).not.toEqual(immediateReset) - const resetActual = await runTrace( - `design-reset-visibility`, - `eager`, - [], - resetHistory, - ) - expect(resetActual.snapshots).toEqual(atomicReset) - }) - - it(`keeps SDK reset callbacks separate from every neighboring message kind`, () => { - const neighbors: Array> = [ - change(`insert`, 1, `row`), - change(`update`, 1, `row`), - change(`delete`, 1, `row`), - { headers: { event: `move-out`, patterns: [{ pos: 0, value: `left` }] } }, - upToDate, - subsetEnd, - mustRefetch, - ] - expect(isSdkResetFramedPartition([[mustRefetch]])).toBe(true) - for (const neighbor of neighbors) { - for (const messages of [ - [neighbor, mustRefetch], - [mustRefetch, neighbor], - ]) { - expect( - isSdkResetFramedPartition([messages]), - JSON.stringify(messages), - ).toBe(false) - expect( - isSdkResetFramedPartition(messages.map((message) => [message])), - ).toBe(true) + oracleReplayTest(it)( + `distinguishes plausible unseen-update and reset-visibility semantics`, + async () => { + const completeUnseenUpdate: ChangeMessage = { + key: `1`, + value: { id: 1, name: `complete`, stable: `stable-1` }, + headers: { operation: `update` }, } - } - }) - - it(`distinguishes callback-atomic and subset publication semantics`, async () => { - const callbackHistory = [ - [ - change(`insert`, 1, `before-control`), - upToDate, - change(`update`, 1, `after-control`), - ], - ] - const callbackAtomic = recomputeCommittedRows(callbackHistory) - const freezeAtControl: typeof callbackAtomic = [ - [1, `before-control`, `stable-1`], - ] - expect(callbackAtomic).toEqual([[1, `after-control`, `stable-1`]]) - expect(callbackAtomic).not.toEqual(freezeAtControl) - expect(isSdkResetFramedPartition([[upToDate, mustRefetch]])).toBe(false) - expect( - isSdkResetFramedPartition([[mustRefetch, subsetEnd, mustRefetch]]), - ).toBe(false) - - const readyPrefix = [[upToDate]] - const subsetHistory = [ - [change(`insert`, 1, `subset-publication`)], - [subsetEnd], - ] - const subsetPublishes = recomputedSnapshots(readyPrefix, subsetHistory) - const upToDateOnly: typeof subsetPublishes = [[], []] - expect(subsetPublishes).not.toEqual(upToDateOnly) - - for (const syncMode of [`eager`, `on-demand`, `progressive`] as const) { - const callbackActual = await runTrace( - `design-callback-atomic-${syncMode}`, - syncMode, + const updateHistory = [[completeUnseenUpdate, upToDate]] + expect(recomputeCommittedRows(updateHistory, `ignore`)).toEqual([]) + expect(recomputeCommittedRows(updateHistory, `promote-complete`)).toEqual( + [[1, `complete`, `stable-1`]], + ) + const updateActual = await runTrace( + `design-unseen-update`, + `eager`, [], - callbackHistory, + updateHistory, + ) + expect(updateActual.rows).toEqual( + recomputeCommittedRows(updateHistory, `ignore`), ) - expect(callbackActual.rows).toEqual(callbackAtomic) - const subsetActual = await runTrace( - `design-subset-publication-${syncMode}`, - syncMode, - readyPrefix, - subsetHistory, + const resetHistory = [ + [change(`insert`, 1, `committed`), upToDate], + [mustRefetch], + ] + const atomicReset = recomputedSnapshots([], resetHistory) + const immediateReset: typeof atomicReset = [atomicReset[0]!, []] + expect(atomicReset).not.toEqual(immediateReset) + const resetActual = await runTrace( + `design-reset-visibility`, + `eager`, + [], + resetHistory, ) - expect(subsetActual.snapshots).toEqual(subsetPublishes) - } - }) + expect(resetActual.snapshots).toEqual(atomicReset) + }, + ) - it(`settles every startup, hydration, snapshot availability, persistence-apply release, and cleanup permutation`, async () => { - const events: Array = [ - `startup-promise`, - `hydration`, - `snapshot-available`, - `release-persistence-apply`, - `cleanup`, - ] - let permutationIndex = 0 - const outcomes = new Set<`resolved` | `aborted`>() - let sawAcknowledgementBeforeDurability = false - for (const order of permutations(events)) { - const currentIndex = permutationIndex++ - try { - const result = await runSchedulerPermutation( - `scheduler-permutation-${currentIndex}`, - order, - ) - outcomes.add(result.outcome) - sawAcknowledgementBeforeDurability ||= result.acknowledgedBeforeDurable - } catch (error) { - throw new Error( - `scheduler permutation ${currentIndex} failed: ${order.join(` → `)}`, - { cause: error }, - ) + oracleReplayTest(it)( + `keeps SDK reset callbacks separate from every neighboring message kind`, + () => { + const neighbors: Array> = [ + change(`insert`, 1, `row`), + change(`update`, 1, `row`), + change(`delete`, 1, `row`), + { + headers: { event: `move-out`, patterns: [{ pos: 0, value: `left` }] }, + }, + upToDate, + subsetEnd, + mustRefetch, + ] + expect(isSdkResetFramedPartition([[mustRefetch]])).toBe(true) + for (const neighbor of neighbors) { + for (const messages of [ + [neighbor, mustRefetch], + [mustRefetch, neighbor], + ]) { + expect( + isSdkResetFramedPartition([messages]), + JSON.stringify(messages), + ).toBe(false) + expect( + isSdkResetFramedPartition(messages.map((message) => [message])), + ).toBe(true) + } } - } - expect(outcomes).toEqual(new Set([`resolved`, `aborted`])) - expect(sawAcknowledgementBeforeDurability).toBe(true) - }, 30_000) + }, + ) + + oracleReplayTest(it)( + `distinguishes callback-atomic and subset publication semantics`, + async () => { + const callbackHistory = [ + [ + change(`insert`, 1, `before-control`), + upToDate, + change(`update`, 1, `after-control`), + ], + ] + const callbackAtomic = recomputeCommittedRows(callbackHistory) + const freezeAtControl: typeof callbackAtomic = [ + [1, `before-control`, `stable-1`], + ] + expect(callbackAtomic).toEqual([[1, `after-control`, `stable-1`]]) + expect(callbackAtomic).not.toEqual(freezeAtControl) + expect(isSdkResetFramedPartition([[upToDate, mustRefetch]])).toBe(false) + expect( + isSdkResetFramedPartition([[mustRefetch, subsetEnd, mustRefetch]]), + ).toBe(false) + + const readyPrefix = [[upToDate]] + const subsetHistory = [ + [change(`insert`, 1, `subset-publication`)], + [subsetEnd], + ] + const subsetPublishes = recomputedSnapshots(readyPrefix, subsetHistory) + const upToDateOnly: typeof subsetPublishes = [[], []] + expect(subsetPublishes).not.toEqual(upToDateOnly) + + for (const syncMode of [`eager`, `on-demand`, `progressive`] as const) { + const callbackActual = await runTrace( + `design-callback-atomic-${syncMode}`, + syncMode, + [], + callbackHistory, + ) + expect(callbackActual.rows).toEqual(callbackAtomic) + + const subsetActual = await runTrace( + `design-subset-publication-${syncMode}`, + syncMode, + readyPrefix, + subsetHistory, + ) + expect(subsetActual.snapshots).toEqual(subsetPublishes) + } + }, + ) + + oracleReplayTest(it)( + `settles every startup, hydration, snapshot availability, persistence-apply release, and cleanup permutation`, + async () => { + const events: Array = [ + `startup-promise`, + `hydration`, + `snapshot-available`, + `release-persistence-apply`, + `cleanup`, + ] + let permutationIndex = 0 + const outcomes = new Set<`resolved` | `aborted`>() + let sawAcknowledgementBeforeDurability = false + for (const order of permutations(events)) { + const currentIndex = permutationIndex++ + try { + const result = await runSchedulerPermutation( + `scheduler-permutation-${currentIndex}`, + order, + ) + outcomes.add(result.outcome) + sawAcknowledgementBeforeDurability ||= + result.acknowledgedBeforeDurable + } catch (error) { + throw new Error( + `scheduler permutation ${currentIndex} failed: ${order.join(` → `)}`, + { cause: error }, + ) + } + } + expect(outcomes).toEqual(new Set([`resolved`, `aborted`])) + expect(sawAcknowledgementBeforeDurability).toBe(true) + }, + 30_000, + ) electricHistoryProperty( `electric.synthetic-batch-partition`, @@ -3798,107 +3722,110 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it(`keeps pending row presence across callbacks until one persisted subset commit`, async () => { - let subscriber!: (messages: Array>) => void - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const persistedRows = new Map() - const persistedMetadata = new Map() - const adapter = createPersistedAdapter(persistedMetadata, persistedRows) - const applyCommittedTx = adapter.applyCommittedTx.bind(adapter) - const durabilityEntered = createDeferred() - const releaseDurability = createDeferred() - const durabilitySettled = createDeferred() - let persistedTx: PersistedTx | undefined - adapter.applyCommittedTx = async (...args) => { - persistedTx = structuredClone(args[1]) - durabilityEntered.resolve() - await releaseDurability.promise - await applyCommittedTx(...args) - durabilitySettled.resolve() - } - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricCollectionOptions({ - id: `persisted-cross-callback-presence`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - syncMode: `eager`, - getKey: (row) => row.id, - startSync: true, + oracleReplayTest(it)( + `keeps pending row presence across callbacks until one persisted subset commit`, + async () => { + let subscriber!: (messages: Array>) => void + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const persistedRows = new Map() + const persistedMetadata = new Map() + const adapter = createPersistedAdapter(persistedMetadata, persistedRows) + const applyCommittedTx = adapter.applyCommittedTx.bind(adapter) + const durabilityEntered = createDeferred() + const releaseDurability = createDeferred() + const durabilitySettled = createDeferred() + let persistedTx: PersistedTx | undefined + adapter.applyCommittedTx = async (...args) => { + persistedTx = structuredClone(args[1]) + durabilityEntered.resolve() + await releaseDurability.promise + await applyCommittedTx(...args) + durabilitySettled.resolve() + } + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricCollectionOptions({ + id: `persisted-cross-callback-presence`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + syncMode: `eager`, + getKey: (row) => row.id, + startSync: true, + }), + persistence: { adapter }, }), - persistence: { adapter }, - }), - ) - - collection.startSyncImmediate() - await withElectricCleanup(async () => { - await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) - - subscriber([change(`insert`, 1, ``)]) - subscriber([change(`update`, 1, `updated across callbacks`)]) - subscriber([subsetEnd]) - await atCheckpoint( - durabilityEntered.promise, - `cross-callback durability entered`, ) - expect({ - publicRows: rowsFromCollection(collection), - durableRows: rowsFromMap(persistedRows), - persistedMutations: persistedTx?.mutations.map((mutation) => ({ - type: mutation.type, - key: mutation.key, - value: mutation.type === `delete` ? undefined : mutation.value, - })), - }).toEqual({ - publicRows: [[1, `updated across callbacks`, `stable-1`]], - durableRows: [], - persistedMutations: [ - { - type: `update`, - key: 1, - value: { - id: 1, - name: ``, - stable: `stable-1`, + collection.startSyncImmediate() + await withElectricCleanup(async () => { + await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) + + subscriber([change(`insert`, 1, ``)]) + subscriber([change(`update`, 1, `updated across callbacks`)]) + subscriber([subsetEnd]) + await atCheckpoint( + durabilityEntered.promise, + `cross-callback durability entered`, + ) + + expect({ + publicRows: rowsFromCollection(collection), + durableRows: rowsFromMap(persistedRows), + persistedMutations: persistedTx?.mutations.map((mutation) => ({ + type: mutation.type, + key: mutation.key, + value: mutation.type === `delete` ? undefined : mutation.value, + })), + }).toEqual({ + publicRows: [[1, `updated across callbacks`, `stable-1`]], + durableRows: [], + persistedMutations: [ + { + type: `update`, + key: 1, + value: { + id: 1, + name: ``, + stable: `stable-1`, + }, }, - }, - { - type: `update`, - key: 1, - value: { - id: 1, - name: `updated across callbacks`, + { + type: `update`, + key: 1, + value: { + id: 1, + name: `updated across callbacks`, + }, }, - }, - ], - }) + ], + }) - releaseDurability.resolve() - await atCheckpoint( - durabilitySettled.promise, - `cross-callback durability settled`, - ) - expect(rowsFromMap(persistedRows)).toEqual([ - [1, `updated across callbacks`, `stable-1`], - ]) - }, [ - async () => { releaseDurability.resolve() - await collection.cleanup() - }, - ]) - }) + await atCheckpoint( + durabilitySettled.promise, + `cross-callback durability settled`, + ) + expect(rowsFromMap(persistedRows)).toEqual([ + [1, `updated across callbacks`, `stable-1`], + ]) + }, [ + async () => { + releaseDurability.resolve() + await collection.cleanup() + }, + ]) + }, + ) const publicationEpochTokensArbitrary = fc.array(designTokenArb, { minLength: 1, @@ -3969,13 +3896,13 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { ], ] - it.each(publicationEpochExamples)( + oracleReplayTest(it).each(publicationEpochExamples)( `reconstructs the authored publication-epoch history`, assertPublicationEpochHistory, 30_000, ) - fcTest.prop( + oracleReplayTest(fcTest).prop( [publicationEpochTokensArbitrary, publicationEpochPartitionArbitrary], { seed: 42714, @@ -3987,7 +3914,7 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { 30_000, ) - fcTest.prop( + oracleReplayTest(fcTest, `electric.publication-epoch-convergence`).prop( [publicationEpochTokensArbitrary, publicationEpochPartitionArbitrary], oraclePropertyOptions(20, `electric.publication-epoch-convergence`), )( @@ -3996,176 +3923,182 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { 30_000, ) - it(`retains a wrong intermediate publication even when the final read repairs it`, async () => { - const trace = createOracleCollection( - `publication-recorder-control`, - `eager`, - createMetadata(new Map()).api, - ) - trace.subscriber([change(`insert`, 1, `old`), upToDate]) - const observed = observePublications(trace.collection) - try { - observed.record(`before`) - trace.subscriber([change(`update`, 1, `wrong`), upToDate]) - trace.subscriber([change(`update`, 1, `correct`), upToDate]) - observed.record(`final`) - expect(rowsFromCollection(trace.collection)).toEqual([ - [1, `correct`, `stable-1`], - ]) - expect(observed.entries[0]!.rows).toEqual([[1, `old`, `stable-1`]]) - expect( - observed.entries.some( - ({ cut, rows }) => cut === `event` && rows[0]?.[1] === `wrong`, - ), - ).toBe(true) - expect(() => + oracleReplayTest(it)( + `retains a wrong intermediate publication even when the final read repairs it`, + async () => { + const trace = createOracleCollection( + `publication-recorder-control`, + `eager`, + createMetadata(new Map()).api, + ) + trace.subscriber([change(`insert`, 1, `old`), upToDate]) + const observed = observePublications(trace.collection) + try { + observed.record(`before`) + trace.subscriber([change(`update`, 1, `wrong`), upToDate]) + trace.subscriber([change(`update`, 1, `correct`), upToDate]) + observed.record(`final`) + expect(rowsFromCollection(trace.collection)).toEqual([ + [1, `correct`, `stable-1`], + ]) + expect(observed.entries[0]!.rows).toEqual([[1, `old`, `stable-1`]]) + expect( + observed.entries.some( + ({ cut, rows }) => cut === `event` && rows[0]?.[1] === `wrong`, + ), + ).toBe(true) + expect(() => + expectWholePublications(observed.entries, [ + [[1, `old`, `stable-1`]], + [[1, `correct`, `stable-1`]], + ]), + ).toThrow() expectWholePublications(observed.entries, [ [[1, `old`, `stable-1`]], + [[1, `wrong`, `stable-1`]], [[1, `correct`, `stable-1`]], - ]), - ).toThrow() - expectWholePublications(observed.entries, [ - [[1, `old`, `stable-1`]], - [[1, `wrong`, `stable-1`]], - [[1, `correct`, `stable-1`]], - ]) - } finally { - observed.stop() - await trace.collection.cleanup() - } - }) + ]) + } finally { + observed.stop() + await trace.collection.cleanup() + } + }, + ) - it(`checks an authored HTTP transcript and separate complete Query snapshots`, async () => { - const { ShapeStream: RealShapeStream } = await vi.importActual<{ - ShapeStream: typeof ShapeStream - }>(`@electric-sql/client`) - const ready: Message = { - headers: { control: `up-to-date`, global_last_seen_lsn: `2` }, - } - const authored: Array>> = [ - [ - change(`insert`, 1, `old`), - { headers: { control: `up-to-date`, global_last_seen_lsn: `1` } }, - ], - [mustRefetch], - [ - change(`insert`, 2, `replacement`), - subsetEnd, - change(`insert`, 3, `after-control`), - ready, - ], - ] - // These complete snapshots are authored directly, not computed by either - // reference reducer or from any adapter result. A post-control row belongs - // to the same delivered callback's atomic publication. - const complete: TraceResult[`snapshots`] = [ - [[1, `old`, `stable-1`]], - [[1, `old`, `stable-1`]], - [ - [2, `replacement`, `stable-2`], - [3, `after-control`, `stable-3`], - ], - ] - const controller = new AbortController() - const headers = { - 'electric-handle': `authored-old`, - 'electric-offset': `1_0`, - // Without this completion header the SDK can prefetch the next chunk; - // a positional mock-response queue would not describe the live request. - 'electric-up-to-date': `true`, - 'electric-schema': JSON.stringify({ - id: { type: `int4` }, - name: { type: `text` }, - stable: { type: `text` }, - }), - } - const wire = (messages: Array>) => - JSON.stringify( - messages.map((message) => - `value` in message - ? { - ...message, - value: { ...message.value, id: String(message.value.id) }, - } - : message, - ), - ) - const responses = [ - new Response(wire(authored[0]!), { headers }), - new Response(wire(authored[1]!), { - status: 409, - headers: { 'electric-handle': `authored-new` }, - }), - new Response(wire(authored[2]!), { - headers: { - ...headers, - 'electric-handle': `authored-new`, - 'electric-offset': `2_0`, - 'electric-cursor': `1`, + oracleReplayTest(it)( + `checks an authored HTTP transcript and separate complete Query snapshots`, + async () => { + const { ShapeStream: RealShapeStream } = await vi.importActual<{ + ShapeStream: typeof ShapeStream + }>(`@electric-sql/client`) + const ready: Message = { + headers: { control: `up-to-date`, global_last_seen_lsn: `2` }, + } + const authored: Array>> = [ + [ + change(`insert`, 1, `old`), + { headers: { control: `up-to-date`, global_last_seen_lsn: `1` } }, + ], + [mustRefetch], + [ + change(`insert`, 2, `replacement`), + subsetEnd, + change(`insert`, 3, `after-control`), + ready, + ], + ] + // These complete snapshots are authored directly, not computed by either + // reference reducer or from any adapter result. A post-control row belongs + // to the same delivered callback's atomic publication. + const complete: TraceResult[`snapshots`] = [ + [[1, `old`, `stable-1`]], + [[1, `old`, `stable-1`]], + [ + [2, `replacement`, `stable-2`], + [3, `after-control`, `stable-3`], + ], + ] + const controller = new AbortController() + const headers = { + 'electric-handle': `authored-old`, + 'electric-offset': `1_0`, + // Without this completion header the SDK can prefetch the next chunk; + // a positional mock-response queue would not describe the live request. + 'electric-up-to-date': `true`, + 'electric-schema': JSON.stringify({ + id: { type: `int4` }, + name: { type: `text` }, + stable: { type: `text` }, + }), + } + const wire = (messages: Array>) => + JSON.stringify( + messages.map((message) => + `value` in message + ? { + ...message, + value: { ...message.value, id: String(message.value.id) }, + } + : message, + ), + ) + const responses = [ + new Response(wire(authored[0]!), { headers }), + new Response(wire(authored[1]!), { + status: 409, + headers: { 'electric-handle': `authored-new` }, + }), + new Response(wire(authored[2]!), { + headers: { + ...headers, + 'electric-handle': `authored-new`, + 'electric-offset': `2_0`, + 'electric-cursor': `1`, + }, + }), + ] + const received: Array>> = [] + const stream = new RealShapeStream({ + url: `http://test-url/v1/authored-publication-transcript`, + params: { table: `rows` }, + signal: controller.signal, + fetchClient: () => { + const response = responses.shift() + if (response) return Promise.resolve(response) + return new Promise((_resolve, reject) => { + if (controller.signal.aborted) reject(controller.signal.reason) + else + controller.signal.addEventListener( + `abort`, + () => reject(controller.signal.reason), + { once: true }, + ) + }) }, - }), - ] - const received: Array>> = [] - const stream = new RealShapeStream({ - url: `http://test-url/v1/authored-publication-transcript`, - params: { table: `rows` }, - signal: controller.signal, - fetchClient: () => { - const response = responses.shift() - if (response) return Promise.resolve(response) - return new Promise((_resolve, reject) => { - if (controller.signal.aborted) reject(controller.signal.reason) - else - controller.signal.addEventListener( - `abort`, - () => reject(controller.signal.reason), - { once: true }, - ) - }) - }, - }) - const unsubscribe = stream.subscribe((messages) => { - received.push(structuredClone(messages)) - }) - try { - await vi.waitFor(() => - expect( - received, - JSON.stringify({ received, remainingResponses: responses.length }), - ).toHaveLength(3), - ) - expect(received).toEqual(authored) - expect(isSdkResetFramedPartition(received)).toBe(true) - // This validates this installed SDK's controlled-HTTP antecedent only. - // It is not proof that all servers or SSE streams emit this transcript. - for (const mode of [`eager`, `on-demand`, `progressive`] as const) { - const direct = await runTrace( - `authored-direct-${mode}`, - mode, - [], - structuredClone(received), + }) + const unsubscribe = stream.subscribe((messages) => { + received.push(structuredClone(messages)) + }) + try { + await vi.waitFor(() => + expect( + received, + JSON.stringify({ received, remainingResponses: responses.length }), + ).toHaveLength(3), ) - const persisted = await runPersistedTrace( - `authored-persisted-${mode}`, - mode, + expect(received).toEqual(authored) + expect(isSdkResetFramedPartition(received)).toBe(true) + // This validates this installed SDK's controlled-HTTP antecedent only. + // It is not proof that all servers or SSE streams emit this transcript. + for (const mode of [`eager`, `on-demand`, `progressive`] as const) { + const direct = await runTrace( + `authored-direct-${mode}`, + mode, + [], + structuredClone(received), + ) + const persisted = await runPersistedTrace( + `authored-persisted-${mode}`, + mode, + structuredClone(received), + complete, + ) + expect(direct.snapshots).toEqual(complete) + expect(persisted.snapshots).toEqual(complete) + expect(persisted.durableRows).toEqual(complete[2]) + } + const query = await runQueryTrace( + `authored-query`, structuredClone(received), complete, ) - expect(direct.snapshots).toEqual(complete) - expect(persisted.snapshots).toEqual(complete) - expect(persisted.durableRows).toEqual(complete[2]) + expect(query.snapshots).toEqual(complete) + } finally { + unsubscribe() + controller.abort() } - const query = await runQueryTrace( - `authored-query`, - structuredClone(received), - complete, - ) - expect(query.snapshots).toEqual(complete) - } finally { - unsubscribe() - controller.abort() - } - }) + }, + ) electricHistoryProperty( `electric.invalid-resume-partition`, @@ -4318,356 +4251,340 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }) } - it(`keeps stream cleanup and stale callbacks scoped to their lifecycle`, async () => { - const subscribers: Array<(messages: Array>) => void> = [] - const unsubscribes = [vi.fn(), vi.fn()] - mockSubscribe.mockImplementation((callback) => { - const index = subscribers.length - subscribers.push(callback) - return unsubscribes[index]! - }) - const options = electricCollectionOptions({ - id: `reused-sync-config`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: true, - }) - const createLifecycle = (id: string) => { - const metadata = createMetadata(resumeState()) - const lifecycleSync = options.sync - return createCollection({ - ...options, - id, - sync: { - ...lifecycleSync, - sync: (params: Parameters[0]) => - lifecycleSync.sync({ ...params, metadata: metadata.api }), + oracleReplayTest(it)( + `keeps stream cleanup and stale callbacks scoped to their lifecycle`, + async () => { + const subscribers: Array<(messages: Array>) => void> = + [] + const unsubscribes = [vi.fn(), vi.fn()] + mockSubscribe.mockImplementation((callback) => { + const index = subscribers.length + subscribers.push(callback) + return unsubscribes[index]! + }) + const options = electricCollectionOptions({ + id: `reused-sync-config`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, }, + getKey: (row) => row.id, + startSync: true, }) - } + const createLifecycle = (id: string) => { + const metadata = createMetadata(resumeState()) + const lifecycleSync = options.sync + return createCollection({ + ...options, + id, + sync: { + ...lifecycleSync, + sync: (params: Parameters[0]) => + lifecycleSync.sync({ ...params, metadata: metadata.api }), + }, + }) + } - const oldCollection = createLifecycle(`old-sync-lifecycle`) - const currentCollection = createLifecycle(`current-sync-lifecycle`) - - subscribers[0]!([change(`insert`, 1, `first row`), upToDate]) - subscribers[1]!([change(`insert`, 2, `second row`), upToDate]) - expect(oldCollection.get(1)?.name).toBe(`first row`) - expect(currentCollection.get(2)?.name).toBe(`second row`) - subscribers[0]!([change(`update`, 1, `first row updated`), upToDate]) - const currentMatch = currentCollection.utils.awaitMatch( - (message) => `value` in message && message.value.id === 3, - 100, - ) - const currentTxid = currentCollection.utils.awaitTxId(42, 100) - const currentSnapshotTxid = currentCollection.utils.awaitTxId(50, 100) - let currentMatchResolved = false - let currentTxidResolved = false - let currentSnapshotTxidResolved = false - void currentMatch.then(() => { - currentMatchResolved = true - }) - void currentTxid.then(() => { - currentTxidResolved = true - }) - void currentSnapshotTxid.then(() => { - currentSnapshotTxidResolved = true - }) - subscribers[0]!([ - change(`insert`, 3, `wrong lifecycle`), - { - headers: { - control: `snapshot-end`, - xmin: `100`, - xmax: `150`, - xip_list: [], + const oldCollection = createLifecycle(`old-sync-lifecycle`) + const currentCollection = createLifecycle(`current-sync-lifecycle`) + + subscribers[0]!([change(`insert`, 1, `first row`), upToDate]) + subscribers[1]!([change(`insert`, 2, `second row`), upToDate]) + expect(oldCollection.get(1)?.name).toBe(`first row`) + expect(currentCollection.get(2)?.name).toBe(`second row`) + subscribers[0]!([change(`update`, 1, `first row updated`), upToDate]) + const currentMatch = currentCollection.utils.awaitMatch( + (message) => `value` in message && message.value.id === 3, + 100, + ) + const currentTxid = currentCollection.utils.awaitTxId(42, 100) + const currentSnapshotTxid = currentCollection.utils.awaitTxId(50, 100) + let currentMatchResolved = false + let currentTxidResolved = false + let currentSnapshotTxidResolved = false + void currentMatch.then(() => { + currentMatchResolved = true + }) + void currentTxid.then(() => { + currentTxidResolved = true + }) + void currentSnapshotTxid.then(() => { + currentSnapshotTxidResolved = true + }) + subscribers[0]!([ + change(`insert`, 3, `wrong lifecycle`), + { + headers: { + control: `snapshot-end`, + xmin: `100`, + xmax: `150`, + xip_list: [], + }, }, - }, - { headers: { control: `up-to-date`, txids: [42] } }, - ]) - await Promise.resolve() - expect(currentMatchResolved).toBe(false) - expect(currentTxidResolved).toBe(false) - expect(currentSnapshotTxidResolved).toBe(false) + { headers: { control: `up-to-date`, txids: [42] } }, + ]) + await Promise.resolve() + expect(currentMatchResolved).toBe(false) + expect(currentTxidResolved).toBe(false) + expect(currentSnapshotTxidResolved).toBe(false) - await oldCollection.cleanup() + await oldCollection.cleanup() - expect(unsubscribes[0]).toHaveBeenCalledOnce() - expect(unsubscribes[1]).not.toHaveBeenCalled() + expect(unsubscribes[0]).toHaveBeenCalledOnce() + expect(unsubscribes[1]).not.toHaveBeenCalled() - subscribers[0]!([change(`update`, 1, `stale`)]) - expect(unsubscribes[1]).not.toHaveBeenCalled() - subscribers[1]!([ - change(`insert`, 3, `still live`), - { - headers: { - control: `snapshot-end`, - xmin: `100`, - xmax: `150`, - xip_list: [], + subscribers[0]!([change(`update`, 1, `stale`)]) + expect(unsubscribes[1]).not.toHaveBeenCalled() + subscribers[1]!([ + change(`insert`, 3, `still live`), + { + headers: { + control: `snapshot-end`, + xmin: `100`, + xmax: `150`, + xip_list: [], + }, }, - }, - { headers: { control: `up-to-date`, txids: [42] } }, - ]) - await expect(currentMatch).resolves.toBe(true) - await expect(currentTxid).resolves.toBe(true) - await expect(currentSnapshotTxid).resolves.toBe(true) - expect(currentCollection.get(3)?.name).toBe(`still live`) - await currentCollection.cleanup() - expect(unsubscribes[1]).toHaveBeenCalledOnce() - }) - - it(`binds sync metadata import and export to the receiving collection`, async () => { - const subscribers: Array<(messages: Array>) => void> = [] - mockSubscribe.mockImplementation((callback) => { - subscribers.push(callback) - return vi.fn() - }) - const options = electricCollectionOptions({ - id: `shared-metadata-config`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: true, - }) - const first = createCollection({ ...options, id: `metadata-first` }) - const second = createCollection({ ...options, id: `metadata-second` }) - - subscribers[0]!([{ headers: { control: `up-to-date`, txids: [11] } }]) - subscribers[1]!([{ headers: { control: `up-to-date`, txids: [22] } }]) - - expect(first.config.sync.exportSyncMeta?.()).toMatchObject({ - version: 1, - seenTxids: [11], - }) - expect(second.config.sync.exportSyncMeta?.()).toMatchObject({ - version: 1, - seenTxids: [22], - }) + { headers: { control: `up-to-date`, txids: [42] } }, + ]) + await expect(currentMatch).resolves.toBe(true) + await expect(currentTxid).resolves.toBe(true) + await expect(currentSnapshotTxid).resolves.toBe(true) + expect(currentCollection.get(3)?.name).toBe(`still live`) + await currentCollection.cleanup() + expect(unsubscribes[1]).toHaveBeenCalledOnce() + }, + ) - const importedResume = { - kind: `resume` as const, - requiresTagState: false, - offset: `7_0` as const, - handle: `shape-7`, - shapeId, - updatedAt: 7, - } - first.config.sync.importSyncMeta?.({ - version: 1, - resume: importedResume, - seenTxids: [99], - }) + oracleReplayTest(it)( + `binds sync metadata import and export to the receiving collection`, + async () => { + const subscribers: Array<(messages: Array>) => void> = + [] + mockSubscribe.mockImplementation((callback) => { + subscribers.push(callback) + return vi.fn() + }) + const options = electricCollectionOptions({ + id: `shared-metadata-config`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + startSync: true, + }) + const first = createCollection({ ...options, id: `metadata-first` }) + const second = createCollection({ ...options, id: `metadata-second` }) - await expect(first.utils.awaitTxId(99, 50)).resolves.toBe(true) - await expect(second.utils.awaitTxId(99, 10)).rejects.toThrow() - expect(first.config.sync.exportSyncMeta?.()).toEqual({ - version: 1, - resume: importedResume, - seenTxids: [99], - }) - expect(second.config.sync.exportSyncMeta?.()).toMatchObject({ - version: 1, - seenTxids: [22], - }) - expect(second.config.sync.exportSyncMeta?.()).not.toMatchObject({ - resume: importedResume, - }) + subscribers[0]!([{ headers: { control: `up-to-date`, txids: [11] } }]) + subscribers[1]!([{ headers: { control: `up-to-date`, txids: [22] } }]) - const third = createCollection({ ...options, id: `metadata-third` }) - await expect(third.utils.awaitTxId(99, 10)).rejects.toThrow() - expect(third.config.sync.exportSyncMeta?.()).not.toMatchObject({ - resume: importedResume, - }) + expect(first.config.sync.exportSyncMeta?.()).toMatchObject({ + version: 1, + seenTxids: [11], + }) + expect(second.config.sync.exportSyncMeta?.()).toMatchObject({ + version: 1, + seenTxids: [22], + }) - await first.cleanup() - await second.cleanup() - await third.cleanup() - }) + const importedResume = { + kind: `resume` as const, + requiresTagState: false, + offset: `7_0` as const, + handle: `shape-7`, + shapeId, + updatedAt: 7, + } + first.config.sync.importSyncMeta?.({ + version: 1, + resume: importedResume, + seenTxids: [99], + }) - it(`consumes raw sync metadata when the next collection is materialized`, async () => { - mockSubscribe.mockImplementation(() => vi.fn()) - const options = electricCollectionOptions({ - id: `seeded-metadata-config`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: true, - }) - const importedResume = { - kind: `resume` as const, - requiresTagState: false, - offset: `8_0` as const, - handle: `shape-8`, - shapeId, - updatedAt: 8, - } - options.sync.importSyncMeta?.({ - version: 1, - resume: importedResume, - seenTxids: [55], - }) + await expect(first.utils.awaitTxId(99, 50)).resolves.toBe(true) + await expect(second.utils.awaitTxId(99, 10)).rejects.toThrow() + expect(first.config.sync.exportSyncMeta?.()).toEqual({ + version: 1, + resume: importedResume, + seenTxids: [99], + }) + expect(second.config.sync.exportSyncMeta?.()).toMatchObject({ + version: 1, + seenTxids: [22], + }) + expect(second.config.sync.exportSyncMeta?.()).not.toMatchObject({ + resume: importedResume, + }) - const seeded = createCollection({ ...options, id: `metadata-seeded` }) - await expect(seeded.utils.awaitTxId(55, 50)).resolves.toBe(true) - expect(seeded.config.sync.exportSyncMeta?.()).toMatchObject({ - resume: importedResume, - seenTxids: [55], - }) + const third = createCollection({ ...options, id: `metadata-third` }) + await expect(third.utils.awaitTxId(99, 10)).rejects.toThrow() + expect(third.config.sync.exportSyncMeta?.()).not.toMatchObject({ + resume: importedResume, + }) - const unseeded = createCollection({ ...options, id: `metadata-unseeded` }) - await expect(unseeded.utils.awaitTxId(55, 10)).rejects.toThrow() - expect(unseeded.config.sync.exportSyncMeta?.()).toEqual({ - version: 1, - seenTxids: [], - }) + await first.cleanup() + await second.cleanup() + await third.cleanup() + }, + ) - await seeded.cleanup() - await unseeded.cleanup() - }) + oracleReplayTest(it)( + `consumes raw sync metadata when the next collection is materialized`, + async () => { + mockSubscribe.mockImplementation(() => vi.fn()) + const options = electricCollectionOptions({ + id: `seeded-metadata-config`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + startSync: true, + }) + const importedResume = { + kind: `resume` as const, + requiresTagState: false, + offset: `8_0` as const, + handle: `shape-8`, + shapeId, + updatedAt: 8, + } + options.sync.importSyncMeta?.({ + version: 1, + resume: importedResume, + seenTxids: [55], + }) - it(`binds imported evidence and pending matches before lazy sync starts`, async () => { - let subscriber!: (messages: Array>) => void - mockSubscribe.mockImplementation((callback) => { - subscriber = callback - return vi.fn() - }) - const options = electricCollectionOptions({ - id: `lazy-bound-evidence`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: false, - }) - const collection = createCollection(options) - const importedResume = { - kind: `resume` as const, - requiresTagState: false, - offset: `9_0` as const, - handle: `shape-9`, - shapeId, - updatedAt: 9, - } - collection.config.sync.importSyncMeta?.({ - version: 1, - resume: importedResume, - seenTxids: [33], - }) + const seeded = createCollection({ ...options, id: `metadata-seeded` }) + await expect(seeded.utils.awaitTxId(55, 50)).resolves.toBe(true) + expect(seeded.config.sync.exportSyncMeta?.()).toMatchObject({ + resume: importedResume, + seenTxids: [55], + }) - await expect(collection.utils.awaitTxId(33, 20)).resolves.toBe(true) - const pendingMatch = collection.utils.awaitMatch( - (message) => `value` in message && message.value.id === 5, - 100, - ) - const pendingTxid = collection.utils.awaitTxId(34, 100) - const preload = collection.preload() - expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ - offset: `9_0`, - handle: `shape-9`, - }) - const evidenceChange = change(`insert`, 5, `matched after start`) - subscriber([ - { - ...evidenceChange, - headers: { operation: `insert`, txids: [34] }, - }, - upToDate, - ]) + const unseeded = createCollection({ ...options, id: `metadata-unseeded` }) + await expect(unseeded.utils.awaitTxId(55, 10)).rejects.toThrow() + expect(unseeded.config.sync.exportSyncMeta?.()).toEqual({ + version: 1, + seenTxids: [], + }) - await expect(pendingMatch).resolves.toBe(true) - await expect(pendingTxid).resolves.toBe(true) - await preload - expect(collection.get(5)?.name).toBe(`matched after start`) - await collection.cleanup() - }) + await seeded.cleanup() + await unseeded.cleanup() + }, + ) - it(`keeps descriptor utilities captured before collection startup live`, async () => { - let subscriber!: (messages: Array>) => void - mockSubscribe.mockImplementation((callback) => { - subscriber = callback - return vi.fn() - }) - const options = electricCollectionOptions({ - id: `captured-descriptor-utilities`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: false, - }) - const { awaitMatch, awaitTxId } = options.utils - const collection = createCollection(options) - const pendingMatch = awaitMatch( - (message) => `value` in message && message.value.id === 77, - 100, - ) - const pendingTxid = awaitTxId(77, 100) + oracleReplayTest(it)( + `binds imported evidence and pending matches before lazy sync starts`, + async () => { + let subscriber!: (messages: Array>) => void + mockSubscribe.mockImplementation((callback) => { + subscriber = callback + return vi.fn() + }) + const options = electricCollectionOptions({ + id: `lazy-bound-evidence`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + startSync: false, + }) + const collection = createCollection(options) + const importedResume = { + kind: `resume` as const, + requiresTagState: false, + offset: `9_0` as const, + handle: `shape-9`, + shapeId, + updatedAt: 9, + } + collection.config.sync.importSyncMeta?.({ + version: 1, + resume: importedResume, + seenTxids: [33], + }) - const preload = collection.preload() - const evidenceChange = change(`insert`, 77, `captured utilities`) - subscriber([ - { - ...evidenceChange, - headers: { operation: `insert`, txids: [77] }, - }, - upToDate, - ]) + await expect(collection.utils.awaitTxId(33, 20)).resolves.toBe(true) + const pendingMatch = collection.utils.awaitMatch( + (message) => `value` in message && message.value.id === 5, + 100, + ) + const pendingTxid = collection.utils.awaitTxId(34, 100) + const preload = collection.preload() + expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ + offset: `9_0`, + handle: `shape-9`, + }) + const evidenceChange = change(`insert`, 5, `matched after start`) + subscriber([ + { + ...evidenceChange, + headers: { operation: `insert`, txids: [34] }, + }, + upToDate, + ]) - await expect(pendingMatch).resolves.toBe(true) - await expect(pendingTxid).resolves.toBe(true) - await preload - await collection.cleanup() - }) + await expect(pendingMatch).resolves.toBe(true) + await expect(pendingTxid).resolves.toBe(true) + await preload + expect(collection.get(5)?.name).toBe(`matched after start`) + await collection.cleanup() + }, + ) - it(`retires a pending pre-start match when a lazy collection is cleaned up`, async () => { - mockSubscribe.mockImplementation(() => vi.fn()) - const collection = createCollection( - electricCollectionOptions({ - id: `lazy-pre-start-cleanup`, + oracleReplayTest(it)( + `keeps descriptor utilities captured before collection startup live`, + async () => { + let subscriber!: (messages: Array>) => void + mockSubscribe.mockImplementation((callback) => { + subscriber = callback + return vi.fn() + }) + const options = electricCollectionOptions({ + id: `captured-descriptor-utilities`, shapeOptions: { url: `http://test-url`, params: { table: `test_table` }, }, getKey: (row) => row.id, startSync: false, - }), - ) - const pendingMatch = collection.utils.awaitMatch(() => false, 100) + }) + const { awaitMatch, awaitTxId } = options.utils + const collection = createCollection(options) + const pendingMatch = awaitMatch( + (message) => `value` in message && message.value.id === 77, + 100, + ) + const pendingTxid = awaitTxId(77, 100) - await collection.cleanup() + const preload = collection.preload() + const evidenceChange = change(`insert`, 77, `captured utilities`) + subscriber([ + { + ...evidenceChange, + headers: { operation: `insert`, txids: [77] }, + }, + upToDate, + ]) - await expect(pendingMatch).rejects.toThrow(/aborted/i) - expect(mockSubscribe).not.toHaveBeenCalled() - }) + await expect(pendingMatch).resolves.toBe(true) + await expect(pendingTxid).resolves.toBe(true) + await preload + await collection.cleanup() + }, + ) - it(`retires pre-start waiters when persisted metadata startup is interrupted`, async () => { - const metadataStarted = createDeferred() - const metadataGate = createDeferred() - const adapter = createPersistedAdapter(new Map(), new Map()) - const loadResumeSnapshot = adapter.loadResumeSnapshot - adapter.loadResumeSnapshot = async (...args) => { - metadataStarted.resolve() - await metadataGate.promise - return loadResumeSnapshot(args[0], args[1]) - } - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricCollectionOptions({ - id: `persisted-startup-waiter-cleanup`, + oracleReplayTest(it)( + `retires a pending pre-start match when a lazy collection is cleaned up`, + async () => { + mockSubscribe.mockImplementation(() => vi.fn()) + const collection = createCollection( + electricCollectionOptions({ + id: `lazy-pre-start-cleanup`, shapeOptions: { url: `http://test-url`, params: { table: `test_table` }, @@ -4675,230 +4592,291 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { getKey: (row) => row.id, startSync: false, }), - persistence: { adapter }, - }), - ) - const pendingMatch = collection.utils.awaitMatch(() => false, 100) - const pendingTxid = collection.utils.awaitTxId(702, 100) - const preload = expect(collection.preload()).rejects.toMatchObject({ - name: `AbortError`, - }) - await metadataStarted.promise - const matchOutcome = expect(pendingMatch).rejects.toThrow(/aborted/i) - const txidOutcome = expect(pendingTxid).rejects.toThrow(/aborted/i) + ) + const pendingMatch = collection.utils.awaitMatch(() => false, 100) - await collection.cleanup() + await collection.cleanup() - await matchOutcome - await txidOutcome - expect(mockSubscribe).not.toHaveBeenCalled() - metadataGate.resolve() - await preload - }) + await expect(pendingMatch).rejects.toThrow(/aborted/i) + expect(mockSubscribe).not.toHaveBeenCalled() + }, + ) - it(`retires pre-start waiters through automatic collection GC`, async () => { - const metadataGate = createDeferred() - const adapter = createPersistedAdapter(new Map(), new Map()) - const loadResumeSnapshot = adapter.loadResumeSnapshot - adapter.loadResumeSnapshot = vi.fn(async (...args) => { - await metadataGate.promise - return loadResumeSnapshot(args[0], args[1]) - }) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricCollectionOptions({ - id: `persisted-gc-waiter-cleanup`, + oracleReplayTest(it)( + `retires pre-start waiters when persisted metadata startup is interrupted`, + async () => { + const metadataStarted = createDeferred() + const metadataGate = createDeferred() + const adapter = createPersistedAdapter(new Map(), new Map()) + const loadResumeSnapshot = adapter.loadResumeSnapshot + adapter.loadResumeSnapshot = async (...args) => { + metadataStarted.resolve() + await metadataGate.promise + return loadResumeSnapshot(args[0], args[1]) + } + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricCollectionOptions({ + id: `persisted-startup-waiter-cleanup`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + startSync: false, + }), + persistence: { adapter }, + }), + ) + const pendingMatch = collection.utils.awaitMatch(() => false, 100) + const pendingTxid = collection.utils.awaitTxId(702, 100) + const preload = expect(collection.preload()).rejects.toMatchObject({ + name: `AbortError`, + }) + await metadataStarted.promise + const matchOutcome = expect(pendingMatch).rejects.toThrow(/aborted/i) + const txidOutcome = expect(pendingTxid).rejects.toThrow(/aborted/i) + + await collection.cleanup() + + await matchOutcome + await txidOutcome + expect(mockSubscribe).not.toHaveBeenCalled() + metadataGate.resolve() + await preload + }, + ) + + oracleReplayTest(it)( + `retires pre-start waiters through automatic collection GC`, + async () => { + const metadataGate = createDeferred() + const adapter = createPersistedAdapter(new Map(), new Map()) + const loadResumeSnapshot = adapter.loadResumeSnapshot + adapter.loadResumeSnapshot = vi.fn(async (...args) => { + await metadataGate.promise + return loadResumeSnapshot(args[0], args[1]) + }) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricCollectionOptions({ + id: `persisted-gc-waiter-cleanup`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + startSync: false, + gcTime: 10, + }), + persistence: { adapter }, + }), + ) + const pendingMatch = collection.utils.awaitMatch(() => false, 5000) + const pendingTxid = collection.utils.awaitTxId(703, 5000) + // A pending preload owns retention; exercise unowned sync for automatic GC. + collection.startSyncImmediate() + await vi.waitFor( + () => expect(adapter.loadResumeSnapshot).toHaveBeenCalledOnce(), + { interval: 1, timeout: 250 }, + ) + const subscription = collection.subscribeChanges(() => {}) + const matchOutcome = pendingMatch.catch((error: unknown) => error) + const txidOutcome = pendingTxid.catch((error: unknown) => error) + + subscription.unsubscribe() + await vi.waitFor(() => expect(collection.status).toBe(`cleaned-up`), { + interval: 10, + timeout: 1500, + }) + const pendingSentinel = Symbol(`pending`) + const matchResult = await Promise.race([ + matchOutcome, + new Promise((resolve) => + setTimeout(() => resolve(pendingSentinel), 50), + ), + ]) + const txidResult = await Promise.race([ + txidOutcome, + new Promise((resolve) => + setTimeout(() => resolve(pendingSentinel), 50), + ), + ]) + + expect(matchResult).toEqual( + expect.objectContaining({ + message: expect.stringMatching(/aborted/i), + }), + ) + expect(txidResult).toEqual( + expect.objectContaining({ + message: expect.stringMatching(/aborted/i), + }), + ) + expect(mockSubscribe).not.toHaveBeenCalled() + metadataGate.resolve() + }, + ) + + oracleReplayTest(it)( + `retires every pending waiter when its collection lifecycle is cleaned up`, + async () => { + mockSubscribe.mockImplementation(() => vi.fn()) + const lazy = createCollection( + electricCollectionOptions({ + id: `lazy-waiter-cleanup`, shapeOptions: { url: `http://test-url`, params: { table: `test_table` }, }, getKey: (row) => row.id, startSync: false, - gcTime: 10, }), - persistence: { adapter }, - }), - ) - const pendingMatch = collection.utils.awaitMatch(() => false, 5000) - const pendingTxid = collection.utils.awaitTxId(703, 5000) - // A pending preload owns retention; exercise unowned sync for automatic GC. - collection.startSyncImmediate() - await vi.waitFor( - () => expect(adapter.loadResumeSnapshot).toHaveBeenCalledOnce(), - { interval: 1, timeout: 250 }, - ) - const subscription = collection.subscribeChanges(() => {}) - const matchOutcome = pendingMatch.catch((error: unknown) => error) - const txidOutcome = pendingTxid.catch((error: unknown) => error) - - subscription.unsubscribe() - await vi.waitFor(() => expect(collection.status).toBe(`cleaned-up`), { - interval: 10, - timeout: 1500, - }) - const pendingSentinel = Symbol(`pending`) - const matchResult = await Promise.race([ - matchOutcome, - new Promise((resolve) => setTimeout(() => resolve(pendingSentinel), 50)), - ]) - const txidResult = await Promise.race([ - txidOutcome, - new Promise((resolve) => setTimeout(() => resolve(pendingSentinel), 50)), - ]) + ) + const lazyTxid = lazy.utils.awaitTxId(700, 100) + await lazy.cleanup() + await expect(lazyTxid).rejects.toThrow(/aborted/i) - expect(matchResult).toEqual( - expect.objectContaining({ - message: expect.stringMatching(/aborted/i), - }), - ) - expect(txidResult).toEqual( - expect.objectContaining({ - message: expect.stringMatching(/aborted/i), - }), - ) - expect(mockSubscribe).not.toHaveBeenCalled() - metadataGate.resolve() - }) + const active = createOracleCollection( + `active-waiter-cleanup`, + `eager`, + createMetadata(new Map()).api, + ) + const activeMatch = active.collection.utils.awaitMatch(() => false, 100) + const activeTxid = active.collection.utils.awaitTxId(701, 100) + await active.collection.cleanup() - it(`retires every pending waiter when its collection lifecycle is cleaned up`, async () => { - mockSubscribe.mockImplementation(() => vi.fn()) - const lazy = createCollection( - electricCollectionOptions({ - id: `lazy-waiter-cleanup`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: false, - }), - ) - const lazyTxid = lazy.utils.awaitTxId(700, 100) - await lazy.cleanup() - await expect(lazyTxid).rejects.toThrow(/aborted/i) + await expect(activeMatch).rejects.toThrow(/aborted/i) + await expect(activeTxid).rejects.toThrow(/aborted/i) + }, + ) - const active = createOracleCollection( - `active-waiter-cleanup`, - `eager`, - createMetadata(new Map()).api, - ) - const activeMatch = active.collection.utils.awaitMatch(() => false, 100) - const activeTxid = active.collection.utils.awaitTxId(701, 100) - await active.collection.cleanup() + oracleReplayTest(it)( + `settles waiters according to whether evidence or cleanup wins`, + async () => { + for (const cleanupWins of [true, false]) { + const trace = createOracleCollection( + `waiter-race-${cleanupWins}`, + `eager`, + createMetadata(new Map()).api, + ) + const pendingMatch = trace.collection.utils.awaitMatch( + (message) => `value` in message && message.value.id === 91, + 100, + ) + const pendingTxid = trace.collection.utils.awaitTxId(91, 100) + const evidenceChange = change(`insert`, 91, `race winner`) + const evidence: Array> = [ + { + ...evidenceChange, + headers: { operation: `insert`, txids: [91] }, + }, + upToDate, + ] - await expect(activeMatch).rejects.toThrow(/aborted/i) - await expect(activeTxid).rejects.toThrow(/aborted/i) - }) + if (cleanupWins) { + await trace.collection.cleanup() + trace.subscriber(evidence) + await expect(pendingMatch).rejects.toThrow(/aborted/i) + await expect(pendingTxid).rejects.toThrow(/aborted/i) + expect(trace.collection.get(91)).toBeUndefined() + } else { + trace.subscriber(evidence) + await trace.collection.cleanup() + await expect(pendingMatch).resolves.toBe(true) + await expect(pendingTxid).resolves.toBe(true) + } + } + }, + ) - it(`settles waiters according to whether evidence or cleanup wins`, async () => { - for (const cleanupWins of [true, false]) { + oracleReplayTest(it)( + `keeps committed match evidence across newer writer batches`, + async () => { + const metadata = createMetadata(new Map()) const trace = createOracleCollection( - `waiter-race-${cleanupWins}`, + `await-match-batch-generation`, `eager`, - createMetadata(new Map()).api, - ) - const pendingMatch = trace.collection.utils.awaitMatch( - (message) => `value` in message && message.value.id === 91, - 100, + metadata.api, ) - const pendingTxid = trace.collection.utils.awaitTxId(91, 100) - const evidenceChange = change(`insert`, 91, `race winner`) - const evidence: Array> = [ - { - ...evidenceChange, - headers: { operation: `insert`, txids: [91] }, - }, - upToDate, - ] - - if (cleanupWins) { - await trace.collection.cleanup() - trace.subscriber(evidence) - await expect(pendingMatch).rejects.toThrow(/aborted/i) - await expect(pendingTxid).rejects.toThrow(/aborted/i) - expect(trace.collection.get(91)).toBeUndefined() - } else { - trace.subscriber(evidence) - await trace.collection.cleanup() - await expect(pendingMatch).resolves.toBe(true) - await expect(pendingTxid).resolves.toBe(true) - } - } - }) - it(`keeps committed match evidence across newer writer batches`, async () => { - const metadata = createMetadata(new Map()) - const trace = createOracleCollection( - `await-match-batch-generation`, - `eager`, - metadata.api, - ) + trace.subscriber([change(`insert`, 1, `old batch`), upToDate]) + trace.subscriber([change(`insert`, 2, `current batch`), upToDate]) - trace.subscriber([change(`insert`, 1, `old batch`), upToDate]) - trace.subscriber([change(`insert`, 2, `current batch`), upToDate]) - - await expect( - trace.collection.utils.awaitMatch( - (message) => `value` in message && message.value.name === `old batch`, - 20, - ), - ).resolves.toBe(true) - await trace.collection.cleanup() - }) + await expect( + trace.collection.utils.awaitMatch( + (message) => `value` in message && message.value.name === `old batch`, + 20, + ), + ).resolves.toBe(true) + await trace.collection.cleanup() + }, + ) - it(`keeps committed match evidence across callbacks with no new data`, async () => { - const trace = createOracleCollection( - `await-match-neutral-callback`, - `eager`, - createMetadata(new Map()).api, - ) + oracleReplayTest(it)( + `keeps committed match evidence across callbacks with no new data`, + async () => { + const trace = createOracleCollection( + `await-match-neutral-callback`, + `eager`, + createMetadata(new Map()).api, + ) - trace.subscriber([change(`insert`, 1, `committed`), upToDate]) - trace.subscriber([ - { - headers: { - control: `snapshot-end`, - xmin: `100`, - xmax: `150`, - xip_list: [], + trace.subscriber([change(`insert`, 1, `committed`), upToDate]) + trace.subscriber([ + { + headers: { + control: `snapshot-end`, + xmin: `100`, + xmax: `150`, + xip_list: [], + }, }, - }, - ]) - trace.subscriber([]) + ]) + trace.subscriber([]) - await expect( - trace.collection.utils.awaitMatch( - (message) => `value` in message && message.value.id === 1, - 20, - ), - ).resolves.toBe(true) - await trace.collection.cleanup() - }) + await expect( + trace.collection.utils.awaitMatch( + (message) => `value` in message && message.value.id === 1, + 20, + ), + ).resolves.toBe(true) + await trace.collection.cleanup() + }, + ) - it(`clears committed match evidence when the stream must refetch`, async () => { - const trace = createOracleCollection( - `await-match-reset-generation`, - `eager`, - createMetadata(new Map()).api, - ) + oracleReplayTest(it)( + `clears committed match evidence when the stream must refetch`, + async () => { + const trace = createOracleCollection( + `await-match-reset-generation`, + `eager`, + createMetadata(new Map()).api, + ) - trace.subscriber([change(`insert`, 1, `old generation`), upToDate]) - trace.subscriber([mustRefetch, upToDate]) + trace.subscriber([change(`insert`, 1, `old generation`), upToDate]) + trace.subscriber([mustRefetch, upToDate]) - await expect( - trace.collection.utils.awaitMatch( - (message) => - `value` in message && message.value.name === `old generation`, - 20, - ), - ).rejects.toThrow(/Timeout waiting for custom match function/) - await trace.collection.cleanup() - }) + await expect( + trace.collection.utils.awaitMatch( + (message) => + `value` in message && message.value.name === `old generation`, + 20, + ), + ).rejects.toThrow(/Timeout waiting for custom match function/) + await trace.collection.cleanup() + }, + ) const reentryHistory = fc.record({ txid: fc.integer({ min: 1, max: 10000 }), @@ -5001,11 +4979,14 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { } } - fcTest.prop([reentryHistory], { seed: 42713, numRuns: oracleRuns(10) })( + oracleReplayTest(fcTest).prop([reentryHistory], { + seed: 42713, + numRuns: oracleRuns(10), + })( `replacement sync runs reject evidence from callback reentry histories (fixed)`, runReentryHistory, ) - fcTest.prop( + oracleReplayTest(fcTest, `electric.match-reentry`).prop( [reentryHistory], oraclePropertyOptions(10, `electric.match-reentry`), )( @@ -5013,38 +4994,42 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { runReentryHistory, ) - it(`does not let a committed message satisfy awaitMatch after restart`, async () => { - const subscribers: Array<(messages: Array>) => void> = [] - mockSubscribe.mockImplementation((callback) => { - subscribers.push(callback) - return vi.fn() - }) - const collection = createCollection( - electricCollectionOptions({ - id: `await-match-lifecycle-generation`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: true, - }), - ) + oracleReplayTest(it)( + `does not let a committed message satisfy awaitMatch after restart`, + async () => { + const subscribers: Array<(messages: Array>) => void> = + [] + mockSubscribe.mockImplementation((callback) => { + subscribers.push(callback) + return vi.fn() + }) + const collection = createCollection( + electricCollectionOptions({ + id: `await-match-lifecycle-generation`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + startSync: true, + }), + ) - subscribers[0]!([change(`insert`, 1, `old lifecycle`), upToDate]) - await collection.cleanup() - collection.startSyncImmediate() - await vi.waitFor(() => expect(subscribers).toHaveLength(2)) + subscribers[0]!([change(`insert`, 1, `old lifecycle`), upToDate]) + await collection.cleanup() + collection.startSyncImmediate() + await vi.waitFor(() => expect(subscribers).toHaveLength(2)) - await expect( - collection.utils.awaitMatch( - (message) => - `value` in message && message.value.name === `old lifecycle`, - 20, - ), - ).rejects.toThrow(/Timeout waiting for custom match function/) - await collection.cleanup() - }) + await expect( + collection.utils.awaitMatch( + (message) => + `value` in message && message.value.name === `old lifecycle`, + 20, + ), + ).rejects.toThrow(/Timeout waiting for custom match function/) + await collection.cleanup() + }, + ) for (const syncMode of [`eager`, `on-demand`, `progressive`] as const) { it(`does not apply an unseen ${syncMode} update after must-refetch`, async () => { @@ -5102,80 +5087,86 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }) } - it(`does not let an older applied receipt finish a newer reset`, async () => { - const metadata = createMetadata(resumeState()) - const trace = createOracleCollection( - `overlapping-reset-generations`, - `eager`, - metadata.api, - ) - const persistence = createDeferred() - const transaction = createTransaction({ - mutationFn: () => persistence.promise, - }) + oracleReplayTest(it)( + `does not let an older applied receipt finish a newer reset`, + async () => { + const metadata = createMetadata(resumeState()) + const trace = createOracleCollection( + `overlapping-reset-generations`, + `eager`, + metadata.api, + ) + const persistence = createDeferred() + const transaction = createTransaction({ + mutationFn: () => persistence.promise, + }) - trace.subscriber([ - mustRefetch, - change(`insert`, 1, `first replacement`), - subsetEnd, - ]) - transaction.mutate(() => - trace.collection.insert({ - id: 99, - name: `optimistic`, - stable: `stable-99`, - }), - ) - trace.subscriber([upToDate]) - trace.subscriber([mustRefetch]) - await Promise.resolve() + trace.subscriber([ + mustRefetch, + change(`insert`, 1, `first replacement`), + subsetEnd, + ]) + transaction.mutate(() => + trace.collection.insert({ + id: 99, + name: `optimistic`, + stable: `stable-99`, + }), + ) + trace.subscriber([upToDate]) + trace.subscriber([mustRefetch]) + await Promise.resolve() - trace.subscriber([change(`update`, 2, `unseen`), upToDate]) + trace.subscriber([change(`update`, 2, `unseen`), upToDate]) - expect(trace.collection.status).not.toBe(`error`) - expect(trace.collection.has(2)).toBe(false) + expect(trace.collection.status).not.toBe(`error`) + expect(trace.collection.has(2)).toBe(false) - persistence.resolve() - await transaction.isPersisted.promise - await trace.collection.cleanup() - }) + persistence.resolve() + await transaction.isPersisted.promise + await trace.collection.cleanup() + }, + ) // Readiness counts accepted rows; they publish when the mutation settles. - it(`publishes readiness once a held receipt is accepted`, async () => { - const metadata = createMetadata(new Map()) - const trace = createOracleCollection( - `deferred-applied-receipt`, - `eager`, - metadata.api, - ) - const persistence = createDeferred() - const transaction = createTransaction({ - mutationFn: () => persistence.promise, - }) - transaction.mutate(() => - trace.collection.insert({ - id: 99, - name: `optimistic`, - stable: `stable-99`, - }), - ) + oracleReplayTest(it)( + `publishes readiness once a held receipt is accepted`, + async () => { + const metadata = createMetadata(new Map()) + const trace = createOracleCollection( + `deferred-applied-receipt`, + `eager`, + metadata.api, + ) + const persistence = createDeferred() + const transaction = createTransaction({ + mutationFn: () => persistence.promise, + }) + transaction.mutate(() => + trace.collection.insert({ + id: 99, + name: `optimistic`, + stable: `stable-99`, + }), + ) - trace.subscriber([change(`insert`, 1, `synced`), upToDate]) - await Promise.resolve() + trace.subscriber([change(`insert`, 1, `synced`), upToDate]) + await Promise.resolve() - expect(trace.collection.status).toBe(`ready`) - expect(trace.collection.has(1)).toBe(false) + expect(trace.collection.status).toBe(`ready`) + expect(trace.collection.has(1)).toBe(false) - persistence.resolve() - await transaction.isPersisted.promise - await trace.collection.stateWhenReady() + persistence.resolve() + await transaction.isPersisted.promise + await trace.collection.stateWhenReady() - expect(trace.collection.status).toBe(`ready`) - expect(trace.collection.get(1)).toEqual( - expect.objectContaining({ stable: `stable-1` }), - ) - await trace.collection.cleanup() - }) + expect(trace.collection.status).toBe(`ready`) + expect(trace.collection.get(1)).toEqual( + expect.objectContaining({ stable: `stable-1` }), + ) + await trace.collection.cleanup() + }, + ) /** * Full-mode recovery can accept its complete source snapshot while an @@ -5189,7 +5180,7 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { * before the acquisition's own commit cursor. The transaction promise * controls application; yielding a turn only drains already queued work. */ - it.each([`before-ready`, `after-ready`] as const)( + oracleReplayTest(it).each([`before-ready`, `after-ready`] as const)( `full recovery subset waits for application when acquired %s`, async (acquireAt) => { const trace = createOracleCollection( @@ -5249,43 +5240,46 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { // A full stream keeps its capability across a same-run must-refetch. Its // previous applied snapshot cannot establish newly requested replacement // coverage. This continuation rejects a permanently fulfilled startup gate. - it(`full recovery demand waits again after a same-run reset`, async () => { - const trace = createOracleCollection( - `full-recovery-reset`, - `on-demand`, - createMetadata( - new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), - ).api, - ) - await withElectricCleanup(async () => { - trace.subscriber([change(`insert`, 1, `first`), upToDate]) - await trace.collection._sync.loadSubset({ limit: 1 }) - trace.subscriber([mustRefetch]) - let outcome = `pending` - const loading = Promise.resolve( - trace.collection._sync.loadSubset({ limit: 2 }), - ).then( - () => { - outcome = `fulfilled` - }, - (error: unknown) => { - outcome = `rejected` - throw error - }, - ) - void loading.catch(() => undefined) - await new Promise((resolve) => setTimeout(resolve, 0)) - expect(outcome, `prior readiness is not replacement coverage`).toBe( - `pending`, + oracleReplayTest(it)( + `full recovery demand waits again after a same-run reset`, + async () => { + const trace = createOracleCollection( + `full-recovery-reset`, + `on-demand`, + createMetadata( + new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), + ).api, ) - trace.subscriber([change(`insert`, 2, `second`), upToDate]) - await atCheckpoint(loading, `same-run replacement applied`) - expect(outcome).toBe(`fulfilled`) - expect(rowsFromCollection(trace.collection)).toEqual([ - [2, `second`, `stable-2`], - ]) - }, [() => trace.collection.cleanup()]) - }) + await withElectricCleanup(async () => { + trace.subscriber([change(`insert`, 1, `first`), upToDate]) + await trace.collection._sync.loadSubset({ limit: 1 }) + trace.subscriber([mustRefetch]) + let outcome = `pending` + const loading = Promise.resolve( + trace.collection._sync.loadSubset({ limit: 2 }), + ).then( + () => { + outcome = `fulfilled` + }, + (error: unknown) => { + outcome = `rejected` + throw error + }, + ) + void loading.catch(() => undefined) + await new Promise((resolve) => setTimeout(resolve, 0)) + expect(outcome, `prior readiness is not replacement coverage`).toBe( + `pending`, + ) + trace.subscriber([change(`insert`, 2, `second`), upToDate]) + await atCheckpoint(loading, `same-run replacement applied`) + expect(outcome).toBe(`fulfilled`) + expect(rowsFromCollection(trace.collection)).toEqual([ + [2, `second`, `stable-2`], + ]) + }, [() => trace.collection.cleanup()]) + }, + ) /** * ## Does a source that still tries ever look failed? @@ -5901,76 +5895,82 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { * public visibility at demand settlement. Full-mode replay's distinct * up-to-date obligation is checked by neighboring laws. */ - it(`applies a scoped reset subset before its demand succeeds`, async () => { - let subscriber: ((messages: Array>) => void) | undefined - mockSubscribe.mockImplementation((callback) => { - subscriber = callback - return vi.fn() - }) - const adapter = createPersistedAdapter( - new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), - new Map(), - ) - // This fixture abstracts physical cache IDs because the law here concerns - // source settlement. The SQLite generation oracle owns physical isolation. - let generation = 0 - const claim = (id: string) => ({ - storageCollectionId: `${id}:generation-${generation}`, - claimId: `claim-${generation}`, - }) - adapter.claimCacheGeneration = async (id) => claim(id) - adapter.rotateCacheGeneration = async (id) => { - generation++ - return claim(id) - } - adapter.renewCacheGenerationClaim = async () => Date.now() + 60_000 - adapter.releaseCacheGenerationClaim = async () => {} - const options = electricCollectionOptions({ - id: `scoped-reset-applied-settlement`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - syncMode: `on-demand`, - getKey: (row) => row.id, - startSync: true, - }) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ ...options, persistence: { adapter } }), - ) - const snapshot = createDeferred() - mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) - - await withElectricCleanup(async () => { - collection.startSyncImmediate() - await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) - const demand = Promise.resolve(collection._sync.loadSubset({ limit: 1 })) - void demand.catch(() => undefined) - await vi.waitFor(() => - expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + oracleReplayTest(it)( + `applies a scoped reset subset before its demand succeeds`, + async () => { + let subscriber: + ((messages: Array>) => void) | undefined + mockSubscribe.mockImplementation((callback) => { + subscriber = callback + return vi.fn() + }) + const adapter = createPersistedAdapter( + new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), + new Map(), ) - subscriber!([mustRefetch]) - subscriber!([change(`insert`, 1, `scoped replacement`), subsetEnd]) - snapshot.resolve() - await atCheckpoint(demand, `scoped reset applied settlement`) - expect(collection.get(1)).toMatchObject({ - id: 1, - name: `scoped replacement`, - stable: `stable-1`, + // This fixture abstracts physical cache IDs because the law here concerns + // source settlement. The SQLite generation oracle owns physical isolation. + let generation = 0 + const claim = (id: string) => ({ + storageCollectionId: `${id}:generation-${generation}`, + claimId: `claim-${generation}`, }) - }, [() => snapshot.resolve(), () => collection.cleanup()]) - }) + adapter.claimCacheGeneration = async (id) => claim(id) + adapter.rotateCacheGeneration = async (id) => { + generation++ + return claim(id) + } + adapter.renewCacheGenerationClaim = async () => Date.now() + 60_000 + adapter.releaseCacheGenerationClaim = async () => {} + const options = electricCollectionOptions({ + id: `scoped-reset-applied-settlement`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + syncMode: `on-demand`, + getKey: (row) => row.id, + startSync: true, + }) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ ...options, persistence: { adapter } }), + ) + const snapshot = createDeferred() + mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) + + await withElectricCleanup(async () => { + collection.startSyncImmediate() + await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) + const demand = Promise.resolve( + collection._sync.loadSubset({ limit: 1 }), + ) + void demand.catch(() => undefined) + await vi.waitFor(() => + expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + ) + subscriber!([mustRefetch]) + subscriber!([change(`insert`, 1, `scoped replacement`), subsetEnd]) + snapshot.resolve() + await atCheckpoint(demand, `scoped reset applied settlement`) + expect(collection.get(1)).toMatchObject({ + id: 1, + name: `scoped replacement`, + stable: `stable-1`, + }) + }, [() => snapshot.resolve(), () => collection.cleanup()]) + }, + ) // Initial failure rejects all current waiters with the original error. // Cleanup quietly settles adapter loads and invalidates later callbacks. // A provider error alone does not make its future callbacks obsolete; // retry semantics are checked by the neighboring recovery laws. - it.each([`failure`, `cleanup`] as const)( + oracleReplayTest(it).each([`failure`, `cleanup`] as const)( `full recovery retires concurrent demand on %s`, async (terminal) => { const loggedError = vi @@ -6027,264 +6027,279 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { // shared. Aborting one demand ends that load before up-to-date; its sibling // and a replacement of the canceled demand still wait for the source // snapshot and succeed after the row applies. - it(`cancels one full recovery demand without settling its sibling`, async () => { - const trace = createOracleCollection( - `full-recovery-one-abort`, - `on-demand`, - createMetadata( - new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), - ).api, - ) - const controller = new AbortController() - const canceled = Promise.resolve( - trace.collection._sync.loadSubset({ - limit: 1, - signal: controller.signal, - }), - ) - void canceled.catch(() => undefined) - let siblingOutcome = `pending` - const sibling = Promise.resolve( - trace.collection._sync.loadSubset({ limit: 2 }), - ).then( - () => { - siblingOutcome = `fulfilled` - }, - () => { - siblingOutcome = `rejected` - }, - ) - - await withElectricCleanup(async () => { - controller.abort() - await expect( - atCheckpoint(canceled, `canceled full recovery demand`), - ).rejects.toMatchObject({ name: `AbortError` }) - expect(siblingOutcome).toBe(`pending`) - - // Cancellation cannot mark this predicate complete or poison a later - // acquisition of the same demand key. - const replacement = Promise.resolve( - trace.collection._sync.loadSubset({ limit: 1 }), + oracleReplayTest(it)( + `cancels one full recovery demand without settling its sibling`, + async () => { + const trace = createOracleCollection( + `full-recovery-one-abort`, + `on-demand`, + createMetadata( + new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), + ).api, + ) + const controller = new AbortController() + const canceled = Promise.resolve( + trace.collection._sync.loadSubset({ + limit: 1, + signal: controller.signal, + }), ) - let replacementOutcome = `pending` - const observedReplacement = replacement.then( + void canceled.catch(() => undefined) + let siblingOutcome = `pending` + const sibling = Promise.resolve( + trace.collection._sync.loadSubset({ limit: 2 }), + ).then( () => { - replacementOutcome = `fulfilled` + siblingOutcome = `fulfilled` }, () => { - replacementOutcome = `rejected` + siblingOutcome = `rejected` }, ) - await Promise.resolve() - expect(replacementOutcome).toBe(`pending`) - trace.subscriber([change(`insert`, 1, `source row`), upToDate]) - await atCheckpoint( - Promise.all([sibling, observedReplacement]), - `remaining full recovery demands`, - ) - expect(siblingOutcome).toBe(`fulfilled`) - expect(replacementOutcome).toBe(`fulfilled`) - expect(trace.collection.get(1)).toMatchObject({ name: `source row` }) - }, [() => trace.collection.cleanup()]) - }) + await withElectricCleanup(async () => { + controller.abort() + await expect( + atCheckpoint(canceled, `canceled full recovery demand`), + ).rejects.toMatchObject({ name: `AbortError` }) + expect(siblingOutcome).toBe(`pending`) + + // Cancellation cannot mark this predicate complete or poison a later + // acquisition of the same demand key. + const replacement = Promise.resolve( + trace.collection._sync.loadSubset({ limit: 1 }), + ) + let replacementOutcome = `pending` + const observedReplacement = replacement.then( + () => { + replacementOutcome = `fulfilled` + }, + () => { + replacementOutcome = `rejected` + }, + ) + await Promise.resolve() + expect(replacementOutcome).toBe(`pending`) + + trace.subscriber([change(`insert`, 1, `source row`), upToDate]) + await atCheckpoint( + Promise.all([sibling, observedReplacement]), + `remaining full recovery demands`, + ) + expect(siblingOutcome).toBe(`fulfilled`) + expect(replacementOutcome).toBe(`fulfilled`) + expect(trace.collection.get(1)).toMatchObject({ name: `source row` }) + }, [() => trace.collection.cleanup()]) + }, + ) // An external ShapeStream abort is a provider failure, not Collection // cleanup. It cannot fulfill an in-flight demand or certify its key as // loaded when no source snapshot arrived. A second identical demand must // therefore remain a failing acquisition rather than deduplicated success. - it(`does not certify a full recovery demand after external stream abort`, async () => { - const external = new AbortController() - const trace = createOracleCollection( - `full-recovery-external-abort`, - `on-demand`, - createMetadata( - new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), - ).api, - { signal: external.signal }, - ) - const first = Promise.resolve( - trace.collection._sync.loadSubset({ limit: 1 }), - ) - void first.catch(() => undefined) - - await withElectricCleanup(async () => { - external.abort() - await expect( - atCheckpoint(first, `externally aborted full recovery demand`), - ).rejects.toMatchObject({ name: `StreamAbortedError` }) - const second = trace.collection._sync.loadSubset({ limit: 1 }) - expect(second).not.toBe(true) - await expect(Promise.resolve(second)).rejects.toMatchObject({ - name: `AbortError`, - }) - expect(trace.collection.size).toBe(0) - }, [() => trace.collection.cleanup()]) - }) + oracleReplayTest(it)( + `does not certify a full recovery demand after external stream abort`, + async () => { + const external = new AbortController() + const trace = createOracleCollection( + `full-recovery-external-abort`, + `on-demand`, + createMetadata( + new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), + ).api, + { signal: external.signal }, + ) + const first = Promise.resolve( + trace.collection._sync.loadSubset({ limit: 1 }), + ) + void first.catch(() => undefined) + + await withElectricCleanup(async () => { + external.abort() + await expect( + atCheckpoint(first, `externally aborted full recovery demand`), + ).rejects.toMatchObject({ name: `StreamAbortedError` }) + const second = trace.collection._sync.loadSubset({ limit: 1 }) + expect(second).not.toBe(true) + await expect(Promise.resolve(second)).rejects.toMatchObject({ + name: `AbortError`, + }) + expect(trace.collection.size).toBe(0) + }, [() => trace.collection.cleanup()]) + }, + ) // A subset transport may return after the shared stream retires. Its // messages cannot apply to that lifecycle, so transport settlement alone // cannot certify the demand or its deduplication key. - it(`does not certify a subset snapshot after external stream abort`, async () => { - const external = new AbortController() - const trace = createOracleCollection( - `subset-external-abort`, - `on-demand`, - createMetadata(new Map()).api, - { signal: external.signal }, - ) - const snapshot = createDeferred() - mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) - const first = Promise.resolve( - trace.collection._sync.loadSubset({ limit: 1 }), - ) - void first.catch(() => undefined) - - await withElectricCleanup(async () => { - await vi.waitFor(() => - expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + oracleReplayTest(it)( + `does not certify a subset snapshot after external stream abort`, + async () => { + const external = new AbortController() + const trace = createOracleCollection( + `subset-external-abort`, + `on-demand`, + createMetadata(new Map()).api, + { signal: external.signal }, ) - external.abort() - snapshot.resolve() - await expect( - atCheckpoint(first, `externally aborted subset snapshot`), - ).rejects.toMatchObject({ name: `AbortError` }) - const second = trace.collection._sync.loadSubset({ limit: 1 }) - expect(second).not.toBe(true) - await expect(Promise.resolve(second)).rejects.toMatchObject({ - name: `AbortError`, - }) - expect(trace.collection.size).toBe(0) - }, [() => snapshot.resolve(), () => trace.collection.cleanup()]) - }) + const snapshot = createDeferred() + mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) + const first = Promise.resolve( + trace.collection._sync.loadSubset({ limit: 1 }), + ) + void first.catch(() => undefined) + + await withElectricCleanup(async () => { + await vi.waitFor(() => + expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + ) + external.abort() + snapshot.resolve() + await expect( + atCheckpoint(first, `externally aborted subset snapshot`), + ).rejects.toMatchObject({ name: `AbortError` }) + const second = trace.collection._sync.loadSubset({ limit: 1 }) + expect(second).not.toBe(true) + await expect(Promise.resolve(second)).rejects.toMatchObject({ + name: `AbortError`, + }) + expect(trace.collection.size).toBe(0) + }, [() => snapshot.resolve(), () => trace.collection.cleanup()]) + }, + ) // Even a completed transport cannot certify a demand while its accepted // source commit is waiting for application. Retiring the stream during that // wait leaves no visible source row, so the demand must fail when the held // applied receipt finally settles. - it(`does not certify an externally aborted subset during applied settlement`, async () => { - const external = new AbortController() - const trace = createOracleCollection( - `subset-held-external-abort`, - `on-demand`, - createMetadata(new Map()).api, - { signal: external.signal }, - ) - const persistence = createDeferred() - const transaction = createTransaction({ - mutationFn: () => persistence.promise, - }) - transaction.mutate(() => - trace.collection.insert({ - id: 99, - name: `optimistic`, - stable: `stable-99`, - }), - ) - const snapshot = createDeferred() - mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) - const first = Promise.resolve( - trace.collection._sync.loadSubset({ limit: 1 }), - ) - void first.catch(() => undefined) - - await withElectricCleanup(async () => { - await vi.waitFor(() => - expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + oracleReplayTest(it)( + `does not certify an externally aborted subset during applied settlement`, + async () => { + const external = new AbortController() + const trace = createOracleCollection( + `subset-held-external-abort`, + `on-demand`, + createMetadata(new Map()).api, + { signal: external.signal }, ) - trace.subscriber([change(`insert`, 1, `source row`), subsetEnd]) - snapshot.resolve() - await new Promise((resolve) => setTimeout(resolve, 0)) - expect(trace.collection.has(1), `applied receipt is held`).toBe(false) - external.abort() - persistence.resolve() - await transaction.isPersisted.promise - await expect( - atCheckpoint(first, `externally aborted held subset`), - ).rejects.toMatchObject({ name: `AbortError` }) - const second = trace.collection._sync.loadSubset({ limit: 1 }) - expect(second).not.toBe(true) - await expect(Promise.resolve(second)).rejects.toMatchObject({ - name: `AbortError`, + const persistence = createDeferred() + const transaction = createTransaction({ + mutationFn: () => persistence.promise, }) - expect(mockStream.requestSnapshot).toHaveBeenCalledOnce() - }, [ - () => snapshot.resolve(), - () => persistence.resolve(), - () => trace.collection.cleanup(), - ]) - }) + transaction.mutate(() => + trace.collection.insert({ + id: 99, + name: `optimistic`, + stable: `stable-99`, + }), + ) + const snapshot = createDeferred() + mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) + const first = Promise.resolve( + trace.collection._sync.loadSubset({ limit: 1 }), + ) + void first.catch(() => undefined) + + await withElectricCleanup(async () => { + await vi.waitFor(() => + expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + ) + trace.subscriber([change(`insert`, 1, `source row`), subsetEnd]) + snapshot.resolve() + await new Promise((resolve) => setTimeout(resolve, 0)) + expect(trace.collection.has(1), `applied receipt is held`).toBe(false) + external.abort() + persistence.resolve() + await transaction.isPersisted.promise + await expect( + atCheckpoint(first, `externally aborted held subset`), + ).rejects.toMatchObject({ name: `AbortError` }) + const second = trace.collection._sync.loadSubset({ limit: 1 }) + expect(second).not.toBe(true) + await expect(Promise.resolve(second)).rejects.toMatchObject({ + name: `AbortError`, + }) + expect(mockStream.requestSnapshot).toHaveBeenCalledOnce() + }, [ + () => snapshot.resolve(), + () => persistence.resolve(), + () => trace.collection.cleanup(), + ]) + }, + ) // Application can outlive the SDK request. A caller that releases its // demand during that interval must reject promptly, while the source // receipt remains held and a later acquisition can still wait for it. - it(`cancels a subset demand while its source receipt is held`, async () => { - const trace = createOracleCollection( - `subset-held-demand-abort`, - `on-demand`, - createMetadata(new Map()).api, - ) - const persistence = createDeferred() - const transaction = createTransaction({ - mutationFn: () => persistence.promise, - }) - transaction.mutate(() => - trace.collection.insert({ - id: 99, - name: `optimistic`, - stable: `stable-99`, - }), - ) - const snapshot = createDeferred() - mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) - const controller = new AbortController() - const first = Promise.resolve( - trace.collection._sync.loadSubset({ - limit: 1, - signal: controller.signal, - }), - ) - let firstOutcome: `pending` | `fulfilled` | `rejected` = `pending` - let firstError: unknown - const observedFirst = first.then( - () => { - firstOutcome = `fulfilled` - }, - (error: unknown) => { - firstOutcome = `rejected` - firstError = error - }, - ) - - await withElectricCleanup(async () => { - await vi.waitFor(() => - expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + oracleReplayTest(it)( + `cancels a subset demand while its source receipt is held`, + async () => { + const trace = createOracleCollection( + `subset-held-demand-abort`, + `on-demand`, + createMetadata(new Map()).api, ) - trace.subscriber([change(`insert`, 1, `source row`), subsetEnd]) - snapshot.resolve() - await new Promise((resolve) => setTimeout(resolve, 0)) - expect(trace.collection.has(1)).toBe(false) - controller.abort() - await new Promise((resolve) => setTimeout(resolve, 0)) - expect(firstOutcome).toBe(`rejected`) - expect(firstError).toMatchObject({ name: `AbortError` }) - persistence.resolve() - await transaction.isPersisted.promise - await observedFirst - expect(trace.collection.has(1)).toBe(true) - }, [ - () => snapshot.resolve(), - () => persistence.resolve(), - () => trace.collection.cleanup(), - ]) - }) + const persistence = createDeferred() + const transaction = createTransaction({ + mutationFn: () => persistence.promise, + }) + transaction.mutate(() => + trace.collection.insert({ + id: 99, + name: `optimistic`, + stable: `stable-99`, + }), + ) + const snapshot = createDeferred() + mockStream.requestSnapshot.mockReturnValueOnce(snapshot.promise) + const controller = new AbortController() + const first = Promise.resolve( + trace.collection._sync.loadSubset({ + limit: 1, + signal: controller.signal, + }), + ) + let firstOutcome: `pending` | `fulfilled` | `rejected` = `pending` + let firstError: unknown + const observedFirst = first.then( + () => { + firstOutcome = `fulfilled` + }, + (error: unknown) => { + firstOutcome = `rejected` + firstError = error + }, + ) + + await withElectricCleanup(async () => { + await vi.waitFor(() => + expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + ) + trace.subscriber([change(`insert`, 1, `source row`), subsetEnd]) + snapshot.resolve() + await new Promise((resolve) => setTimeout(resolve, 0)) + expect(trace.collection.has(1)).toBe(false) + controller.abort() + await new Promise((resolve) => setTimeout(resolve, 0)) + expect(firstOutcome).toBe(`rejected`) + expect(firstError).toMatchObject({ name: `AbortError` }) + persistence.resolve() + await transaction.isPersisted.promise + await observedFirst + expect(trace.collection.has(1)).toBe(true) + }, [ + () => snapshot.resolve(), + () => persistence.resolve(), + () => trace.collection.cleanup(), + ]) + }, + ) // A demand canceled before its subset transport settles has no successful // acquisition, whether that transport later resolves or rejects. The source // may still apply a late SDK response, but this caller observes AbortError and // an identical later demand remains eligible for a new acquisition. - it.each([`resolve`, `reject`] as const)( + oracleReplayTest(it).each([`resolve`, `reject`] as const)( `rejects a canceled subset demand when its transport later %ss`, async (transportOutcome) => { const trace = createOracleCollection( @@ -6330,229 +6345,248 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { // cancellation independently of the request ahead of it. At the next task // boundary after abort, this caller must have rejected without invoking a // second SDK request; the first demand remains pending until its response. - it(`cancels a queued subset demand while an earlier snapshot is held`, async () => { - const trace = createOracleCollection( - `queued-subset-abort`, - `on-demand`, - createMetadata(new Map()).api, - ) - const held = createDeferred() - mockStream.requestSnapshot.mockReturnValueOnce(held.promise) - const first = Promise.resolve( - trace.collection._sync.loadSubset({ limit: 1 }), - ) - const controller = new AbortController() - let secondOutcome: `pending` | `fulfilled` | `rejected` = `pending` - let secondError: unknown - const second = Promise.resolve( - trace.collection._sync.loadSubset({ - limit: 2, - signal: controller.signal, - }), - ).then( - () => { - secondOutcome = `fulfilled` - }, - (error: unknown) => { - secondOutcome = `rejected` - secondError = error - }, - ) - - await withElectricCleanup(async () => { - await vi.waitFor(() => - expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + oracleReplayTest(it)( + `cancels a queued subset demand while an earlier snapshot is held`, + async () => { + const trace = createOracleCollection( + `queued-subset-abort`, + `on-demand`, + createMetadata(new Map()).api, + ) + const held = createDeferred() + mockStream.requestSnapshot.mockReturnValueOnce(held.promise) + const first = Promise.resolve( + trace.collection._sync.loadSubset({ limit: 1 }), + ) + const controller = new AbortController() + let secondOutcome: `pending` | `fulfilled` | `rejected` = `pending` + let secondError: unknown + const second = Promise.resolve( + trace.collection._sync.loadSubset({ + limit: 2, + signal: controller.signal, + }), + ).then( + () => { + secondOutcome = `fulfilled` + }, + (error: unknown) => { + secondOutcome = `rejected` + secondError = error + }, ) - controller.abort() - await new Promise((resolve) => setTimeout(resolve, 0)) - expect(secondOutcome).toBe(`rejected`) - expect(secondError).toMatchObject({ name: `AbortError` }) - expect(mockStream.requestSnapshot).toHaveBeenCalledOnce() - held.resolve() - await atCheckpoint(Promise.all([first, second]), `queued abort drain`) - }, [() => held.resolve(), () => trace.collection.cleanup()]) - }) - it(`drops a held receipt's rows when cleanup abandons it`, async () => { - const metadata = createMetadata(new Map()) - const trace = createOracleCollection( - `rejected-applied-receipt`, - `eager`, - metadata.api, - ) - const persistence = createDeferred() - const transaction = createTransaction({ - mutationFn: () => persistence.promise, - }) - transaction.mutate(() => - trace.collection.insert({ - id: 99, - name: `optimistic`, - stable: `stable-99`, - }), - ) - trace.subscriber([change(`insert`, 1, `synced`), upToDate]) - await Promise.resolve() - expect(trace.collection.status).toBe(`ready`) + await withElectricCleanup(async () => { + await vi.waitFor(() => + expect(mockStream.requestSnapshot).toHaveBeenCalledOnce(), + ) + controller.abort() + await new Promise((resolve) => setTimeout(resolve, 0)) + expect(secondOutcome).toBe(`rejected`) + expect(secondError).toMatchObject({ name: `AbortError` }) + expect(mockStream.requestSnapshot).toHaveBeenCalledOnce() + held.resolve() + await atCheckpoint(Promise.all([first, second]), `queued abort drain`) + }, [() => held.resolve(), () => trace.collection.cleanup()]) + }, + ) - await trace.collection.cleanup() - persistence.resolve() - await transaction.isPersisted.promise - await Promise.resolve() + oracleReplayTest(it)( + `drops a held receipt's rows when cleanup abandons it`, + async () => { + const metadata = createMetadata(new Map()) + const trace = createOracleCollection( + `rejected-applied-receipt`, + `eager`, + metadata.api, + ) + const persistence = createDeferred() + const transaction = createTransaction({ + mutationFn: () => persistence.promise, + }) + transaction.mutate(() => + trace.collection.insert({ + id: 99, + name: `optimistic`, + stable: `stable-99`, + }), + ) + trace.subscriber([change(`insert`, 1, `synced`), upToDate]) + await Promise.resolve() + expect(trace.collection.status).toBe(`ready`) - expect(trace.collection.status).toBe(`cleaned-up`) - expect(trace.collection._lifecycle.getSyncError()).toBeUndefined() - expect(trace.collection.has(1)).toBe(false) - }) + await trace.collection.cleanup() + persistence.resolve() + await transaction.isPersisted.promise + await Promise.resolve() - it(`accepts partial resumed updates for rows restored by persistence`, async () => { - let subscriber!: (messages: Array>) => void - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const collectionMetadata = new Map(resumeState()) - const persistedRows = new Map([ - [ - 1, - { - id: 1, - name: `persisted`, - stable: `stable-1`, - }, - ], - ]) - const electricOptions = electricCollectionOptions({ - id: `persisted-resume-oracle`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - syncMode: `progressive`, - getKey: (row) => row.id, - startSync: true, - }) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricOptions, - persistence: { - adapter: createPersistedAdapter(collectionMetadata, persistedRows), + expect(trace.collection.status).toBe(`cleaned-up`) + expect(trace.collection._lifecycle.getSyncError()).toBeUndefined() + expect(trace.collection.has(1)).toBe(false) + }, + ) + + oracleReplayTest(it)( + `accepts partial resumed updates for rows restored by persistence`, + async () => { + let subscriber!: (messages: Array>) => void + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const collectionMetadata = new Map(resumeState()) + const persistedRows = new Map([ + [ + 1, + { + id: 1, + name: `persisted`, + stable: `stable-1`, + }, + ], + ]) + const electricOptions = electricCollectionOptions({ + id: `persisted-resume-oracle`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, }, - }), - ) + syncMode: `progressive`, + getKey: (row) => row.id, + startSync: true, + }) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricOptions, + persistence: { + adapter: createPersistedAdapter(collectionMetadata, persistedRows), + }, + }), + ) - collection.startSyncImmediate() - await collection._sync.loadSubset({ limit: 10 }) - expect(collection.get(1)).toEqual( - expect.objectContaining({ name: `persisted`, stable: `stable-1` }), - ) - expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ - offset: `10_0`, - handle: `shape-1`, - }) + collection.startSyncImmediate() + await collection._sync.loadSubset({ limit: 10 }) + expect(collection.get(1)).toEqual( + expect.objectContaining({ name: `persisted`, stable: `stable-1` }), + ) + expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ + offset: `10_0`, + handle: `shape-1`, + }) - subscriber([change(`update`, 1, `resumed update`), upToDate]) - await vi.waitFor(() => expect(collection.status).toBe(`ready`)) + subscriber([change(`update`, 1, `resumed update`), upToDate]) + await vi.waitFor(() => expect(collection.status).toBe(`ready`)) - expect(collection.get(1)).toEqual( - expect.objectContaining({ - name: `resumed update`, - stable: `stable-1`, - }), - ) - await vi.waitFor(() => { - expect(persistedRows.get(1)).toEqual( + expect(collection.get(1)).toEqual( expect.objectContaining({ name: `resumed update`, stable: `stable-1`, }), ) - }) - await collection.cleanup() - }) + await vi.waitFor(() => { + expect(persistedRows.get(1)).toEqual( + expect.objectContaining({ + name: `resumed update`, + stable: `stable-1`, + }), + ) + }) + await collection.cleanup() + }, + ) - it(`buffers resumed updates that arrive while persisted rows are hydrating`, async () => { - let subscriber: ((messages: Array>) => void) | undefined - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const hydration = createDeferred() - const collectionMetadata = new Map(resumeState()) - const persistedRows = new Map([ - [ - 1, - { - id: 1, - name: `persisted`, - stable: `stable-1`, - }, - ], - ]) - const electricOptions = electricCollectionOptions({ - id: `concurrent-persisted-resume-oracle`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - syncMode: `progressive`, - getKey: (row) => row.id, - startSync: true, - }) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricOptions, - persistence: { - adapter: createPersistedAdapter( - collectionMetadata, - persistedRows, - hydration.promise, - ), + oracleReplayTest(it)( + `buffers resumed updates that arrive while persisted rows are hydrating`, + async () => { + let subscriber: + ((messages: Array>) => void) | undefined + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const hydration = createDeferred() + const collectionMetadata = new Map(resumeState()) + const persistedRows = new Map([ + [ + 1, + { + id: 1, + name: `persisted`, + stable: `stable-1`, + }, + ], + ]) + const electricOptions = electricCollectionOptions({ + id: `concurrent-persisted-resume-oracle`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, }, - }), - ) + syncMode: `progressive`, + getKey: (row) => row.id, + startSync: true, + }) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricOptions, + persistence: { + adapter: createPersistedAdapter( + collectionMetadata, + persistedRows, + hydration.promise, + ), + }, + }), + ) - collection.startSyncImmediate() - await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) - expect(collection.status).not.toBe(`error`) - subscriber!([change(`update`, 1, `concurrent update`), upToDate]) + collection.startSyncImmediate() + await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) + expect(collection.status).not.toBe(`error`) + subscriber!([change(`update`, 1, `concurrent update`), upToDate]) - expect(collection.status).not.toBe(`error`) - hydration.resolve() - await vi.waitFor(() => expect(collection.status).toBe(`ready`)) - expect(collection.get(1)).toEqual( - expect.objectContaining({ - name: `concurrent update`, - stable: `stable-1`, - }), - ) - await collection.cleanup() - }) + expect(collection.status).not.toBe(`error`) + hydration.resolve() + await vi.waitFor(() => expect(collection.status).toBe(`ready`)) + expect(collection.get(1)).toEqual( + expect.objectContaining({ + name: `concurrent update`, + stable: `stable-1`, + }), + ) + await collection.cleanup() + }, + ) - it(`does not advance durable Electric resume metadata past a rejected row commit`, async () => { - await runRejectedDurablePrefixWitness(`durable-prefix-fixed`, [ - `first`, - `second`, - ]) - }) + oracleReplayTest(it)( + `does not advance durable Electric resume metadata past a rejected row commit`, + async () => { + await runRejectedDurablePrefixWitness(`durable-prefix-fixed`, [ + `first`, + `second`, + ]) + }, + ) - it(`retains an accepted Electric source prefix when the next row commit rejects`, async () => { - await runRejectedDurablePrefixWitness( - `durable-prefix-after-success`, - [`accepted`, `rejected`], - true, - ) - }) + oracleReplayTest(it)( + `retains an accepted Electric source prefix when the next row commit rejects`, + async () => { + await runRejectedDurablePrefixWitness( + `durable-prefix-after-success`, + [`accepted`, `rejected`], + true, + ) + }, + ) const durablePrefixArbitrary = fc.tuple( fc.string({ maxLength: 12 }), @@ -6564,17 +6598,23 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { names, ) } - fcTest.prop([durablePrefixArbitrary], durablePrefixPropertyOptions(`fixed`))( + oracleReplayTest(fcTest).prop( + [durablePrefixArbitrary], + durablePrefixPropertyOptions(`fixed`), + )( `a rejected initial Electric row commit creates no durable source prefix (fixed)`, checkDurablePrefix, ) - // Replay the random lane with the legacy seed/path variables and -t 'random'. - fcTest.prop([durablePrefixArbitrary], durablePrefixPropertyOptions(`random`))( + // A shared seed reseeds this random lane; a path replay selects it by property. + oracleReplayTest(fcTest, durablePrefixProperty).prop( + [durablePrefixArbitrary], + durablePrefixPropertyOptions(`random`), + )( `a rejected initial Electric row commit creates no durable source prefix (random)`, checkDurablePrefix, ) - fcTest.prop( + oracleReplayTest(fcTest).prop( [ fc.uniqueArray(fc.integer({ min: 1, max: 20 }), { minLength: 2, @@ -6607,168 +6647,180 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { ) }) - it(`rehydrates persisted rows and resume metadata after cleanup and restart`, async () => { - const subscribers: Array<(messages: Array>) => void> = [] - mockSubscribe.mockImplementation((callback) => { - subscribers.push(callback) - return vi.fn() - }) - const collectionMetadata = new Map(resumeState()) - const persistedRows = new Map([ - [1, { id: 1, name: `persisted`, stable: `stable-1` }], - ]) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricCollectionOptions({ - id: `persisted-restart-oracle`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, + oracleReplayTest(it)( + `rehydrates persisted rows and resume metadata after cleanup and restart`, + async () => { + const subscribers: Array<(messages: Array>) => void> = + [] + mockSubscribe.mockImplementation((callback) => { + subscribers.push(callback) + return vi.fn() + }) + const collectionMetadata = new Map(resumeState()) + const persistedRows = new Map([ + [1, { id: 1, name: `persisted`, stable: `stable-1` }], + ]) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricCollectionOptions({ + id: `persisted-restart-oracle`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + syncMode: `progressive`, + getKey: (row) => row.id, + startSync: true, + }), + persistence: { + adapter: createPersistedAdapter(collectionMetadata, persistedRows), }, - syncMode: `progressive`, - getKey: (row) => row.id, - startSync: true, }), - persistence: { - adapter: createPersistedAdapter(collectionMetadata, persistedRows), - }, - }), - ) + ) - collection.startSyncImmediate() - await vi.waitFor(() => expect(collection.get(1)?.name).toBe(`persisted`)) - await vi.waitFor(() => expect(subscribers).toHaveLength(1)) - subscribers[0]!([upToDate]) - await collection.stateWhenReady() + collection.startSyncImmediate() + await vi.waitFor(() => expect(collection.get(1)?.name).toBe(`persisted`)) + await vi.waitFor(() => expect(subscribers).toHaveLength(1)) + subscribers[0]!([upToDate]) + await collection.stateWhenReady() - await collection.cleanup() - collection.startSyncImmediate() - await vi.waitFor(() => expect(subscribers).toHaveLength(2)) - await vi.waitFor(() => expect(collection.get(1)?.name).toBe(`persisted`)) - expect(collectionMetadata.get(`electric:resume`)).toEqual( - expect.objectContaining({ kind: `resume` }), - ) - await collection.cleanup() - }) + await collection.cleanup() + collection.startSyncImmediate() + await vi.waitFor(() => expect(subscribers).toHaveLength(2)) + await vi.waitFor(() => expect(collection.get(1)?.name).toBe(`persisted`)) + expect(collectionMetadata.get(`electric:resume`)).toEqual( + expect.objectContaining({ kind: `resume` }), + ) + await collection.cleanup() + }, + ) - it(`does not let hydration from a cleaned-up lifecycle poison restart`, async () => { - const subscribers: Array<(messages: Array>) => void> = [] - mockSubscribe.mockImplementation((callback) => { - subscribers.push(callback) - return vi.fn() - }) - const firstHydration = createDeferred() - const collectionMetadata = new Map(resumeState()) - const persistedRows = new Map([ - [1, { id: 1, name: `current`, stable: `stable-1` }], - ]) - const adapter = createPersistedAdapter(collectionMetadata, persistedRows) - const loadResumeSnapshot = adapter.loadResumeSnapshot - let hydrationCall = 0 - adapter.loadResumeSnapshot = vi.fn(async (collectionId, ctx) => { - const snapshot = await loadResumeSnapshot(collectionId, ctx) - if (ctx?.includeRows === false) return snapshot - hydrationCall++ - if (hydrationCall === 1) { - await firstHydration.promise - return { - ...snapshot, - rows: [ - { - key: 1, - value: { id: 1, name: `stale`, stable: `stable-1` }, - }, - ], + oracleReplayTest(it)( + `does not let hydration from a cleaned-up lifecycle poison restart`, + async () => { + const subscribers: Array<(messages: Array>) => void> = + [] + mockSubscribe.mockImplementation((callback) => { + subscribers.push(callback) + return vi.fn() + }) + const firstHydration = createDeferred() + const collectionMetadata = new Map(resumeState()) + const persistedRows = new Map([ + [1, { id: 1, name: `current`, stable: `stable-1` }], + ]) + const adapter = createPersistedAdapter(collectionMetadata, persistedRows) + const loadResumeSnapshot = adapter.loadResumeSnapshot + let hydrationCall = 0 + adapter.loadResumeSnapshot = vi.fn(async (collectionId, ctx) => { + const snapshot = await loadResumeSnapshot(collectionId, ctx) + if (ctx?.includeRows === false) return snapshot + hydrationCall++ + if (hydrationCall === 1) { + await firstHydration.promise + return { + ...snapshot, + rows: [ + { + key: 1, + value: { id: 1, name: `stale`, stable: `stable-1` }, + }, + ], + } } - } - return snapshot - }) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricCollectionOptions({ - id: `persisted-stale-hydration-oracle`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - syncMode: `progressive`, - getKey: (row) => row.id, - startSync: true, + return snapshot + }) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricCollectionOptions({ + id: `persisted-stale-hydration-oracle`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + syncMode: `progressive`, + getKey: (row) => row.id, + startSync: true, + }), + persistence: { adapter }, }), - persistence: { adapter }, - }), - ) - - collection.startSyncImmediate() - await vi.waitFor(() => expect(hydrationCall).toBe(1)) - await collection.cleanup() - collection.startSyncImmediate() - await vi.waitFor(() => expect(subscribers).toHaveLength(2)) + ) - firstHydration.resolve() - await vi.waitFor(() => expect(hydrationCall).toBe(2)) - await vi.waitFor(() => expect(collection.get(1)?.name).toBe(`current`)) - subscribers[1]!([upToDate]) - await collection.stateWhenReady() - await collection.cleanup() - }) + collection.startSyncImmediate() + await vi.waitFor(() => expect(hydrationCall).toBe(1)) + await collection.cleanup() + collection.startSyncImmediate() + await vi.waitFor(() => expect(subscribers).toHaveLength(2)) + + firstHydration.resolve() + await vi.waitFor(() => expect(hydrationCall).toBe(2)) + await vi.waitFor(() => expect(collection.get(1)?.name).toBe(`current`)) + subscribers[1]!([upToDate]) + await collection.stateWhenReady() + await collection.cleanup() + }, + ) - it(`does not hydrate persisted on-demand rows before subset demand`, async () => { - let subscriber: ((messages: Array>) => void) | undefined - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const collectionMetadata = new Map() - const persistedRows = new Map([ - [1, { id: 1, name: `persisted`, stable: `stable-1` }], - ]) - const adapter = createPersistedAdapter(collectionMetadata, persistedRows) - const loadSubset = vi.fn(adapter.loadSubset) - adapter.loadSubset = loadSubset - const electricOptions = electricCollectionOptions({ - id: `persisted-on-demand-oracle`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - syncMode: `on-demand`, - getKey: (row) => row.id, - startSync: true, - }) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricOptions, - persistence: { adapter }, - }), - ) + oracleReplayTest(it)( + `does not hydrate persisted on-demand rows before subset demand`, + async () => { + let subscriber: + ((messages: Array>) => void) | undefined + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const collectionMetadata = new Map() + const persistedRows = new Map([ + [1, { id: 1, name: `persisted`, stable: `stable-1` }], + ]) + const adapter = createPersistedAdapter(collectionMetadata, persistedRows) + const loadSubset = vi.fn(adapter.loadSubset) + adapter.loadSubset = loadSubset + const electricOptions = electricCollectionOptions({ + id: `persisted-on-demand-oracle`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + syncMode: `on-demand`, + getKey: (row) => row.id, + startSync: true, + }) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricOptions, + persistence: { adapter }, + }), + ) - collection.startSyncImmediate() - await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) - subscriber!([upToDate]) - await vi.waitFor(() => expect(collection.status).toBe(`ready`)) + collection.startSyncImmediate() + await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) + subscriber!([upToDate]) + await vi.waitFor(() => expect(collection.status).toBe(`ready`)) - expect(loadSubset).not.toHaveBeenCalled() - expect(collection.has(1)).toBe(false) - await collection.cleanup() - }) + expect(loadSubset).not.toHaveBeenCalled() + expect(collection.has(1)).toBe(false) + await collection.cleanup() + }, + ) - it.each( + oracleReplayTest(it).each( [`reset`, `delete`, `move-out`].flatMap((removal) => [`none`, `before-commit`, `after-commit`].map((acquire) => ({ removal, @@ -6920,55 +6972,58 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { ], ) - it(`applies on-demand catch-up updates to hydrated persisted rows`, async () => { - let subscriber!: (messages: Array>) => void - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const collectionMetadata = new Map(resumeState()) - const persistedRows = new Map([ - [1, { id: 1, name: `persisted`, stable: `stable-1` }], - ]) - const collection = createCollection( - persistedCollectionOptions< - OracleRow, - string | number, - never, - ElectricCollectionUtils - >({ - ...electricCollectionOptions({ - id: `on-demand-persisted-catch-up`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, + oracleReplayTest(it)( + `applies on-demand catch-up updates to hydrated persisted rows`, + async () => { + let subscriber!: (messages: Array>) => void + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const collectionMetadata = new Map(resumeState()) + const persistedRows = new Map([ + [1, { id: 1, name: `persisted`, stable: `stable-1` }], + ]) + const collection = createCollection( + persistedCollectionOptions< + OracleRow, + string | number, + never, + ElectricCollectionUtils + >({ + ...electricCollectionOptions({ + id: `on-demand-persisted-catch-up`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + syncMode: `on-demand`, + getKey: (row) => row.id, + startSync: true, + }), + persistence: { + adapter: createPersistedAdapter(collectionMetadata, persistedRows), }, - syncMode: `on-demand`, - getKey: (row) => row.id, - startSync: true, }), - persistence: { - adapter: createPersistedAdapter(collectionMetadata, persistedRows), - }, - }), - ) + ) - collection.startSyncImmediate() - await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) - await collection._sync.loadSubset({}) - subscriber([change(`update`, 1, `caught up`), upToDate]) - await vi.waitFor(() => expect(collection.status).toBe(`ready`)) + collection.startSyncImmediate() + await vi.waitFor(() => expect(subscriber).toBeTypeOf(`function`)) + await collection._sync.loadSubset({}) + subscriber([change(`update`, 1, `caught up`), upToDate]) + await vi.waitFor(() => expect(collection.status).toBe(`ready`)) - expect(collection.get(1)).toEqual( - expect.objectContaining({ name: `caught up`, stable: `stable-1` }), - ) - expect(collectionMetadata.get(`electric:resume`)).toEqual( - expect.objectContaining({ kind: `resume`, offset: `20_0` }), - ) - await collection.cleanup() - }) + expect(collection.get(1)).toEqual( + expect.objectContaining({ name: `caught up`, stable: `stable-1` }), + ) + expect(collectionMetadata.get(`electric:resume`)).toEqual( + expect.objectContaining({ kind: `resume`, offset: `20_0` }), + ) + await collection.cleanup() + }, + ) - it.each([ + oracleReplayTest(it).each([ { name: `malformed`, seed: new Map([ @@ -7030,31 +7085,34 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { ) await trace.collection.cleanup() }) - - it(`does not mix an explicit resume option with persisted metadata`, async () => { - const metadata = createMetadata(resumeState()) - const trace = createOracleCollection( - `explicit-resume`, - `on-demand`, - metadata.api, - { handle: `explicit-handle` }, - ) - - expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ - offset: `now`, - handle: `explicit-handle`, - }) - trace.subscriber([upToDate]) - expect(trace.collection.status).toBe(`ready`) - await trace.collection.cleanup() - }) + + oracleReplayTest(it)( + `does not mix an explicit resume option with persisted metadata`, + async () => { + const metadata = createMetadata(resumeState()) + const trace = createOracleCollection( + `explicit-resume`, + `on-demand`, + metadata.api, + { handle: `explicit-handle` }, + ) + + expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ + offset: `now`, + handle: `explicit-handle`, + }) + trace.subscriber([upToDate]) + expect(trace.collection.status).toBe(`ready`) + await trace.collection.cleanup() + }, + ) // An explicit source cursor owns this run's resume decision. A malformed // durable record is still invalid for a later implicit run, but it cannot // authorize this run to write a reset before any source evidence arrives. // The metadata Map observes that side effect at startup, before up-to-date // can legitimately stage a fresh resume record. - it.each([ + oracleReplayTest(it).each([ { name: `offset`, options: { offset: `10_0` as Offset } }, { name: `handle`, options: { handle: `explicit-handle` } }, ])( @@ -7076,54 +7134,57 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it(`lets an equal-timestamp reset dominate a stale hydrated resume`, async () => { - let subscriber!: (messages: Array>) => void - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const options = electricCollectionOptions({ - id: `equal-timestamp-reset`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table` }, - }, - getKey: (row) => row.id, - startSync: true, - }) - options.sync.importSyncMeta?.({ - version: 1, - resume: { - kind: `resume`, - requiresTagState: false, - offset: `10_0`, - handle: `stale-handle`, - shapeId, - updatedAt: 1, - }, - seenTxids: [], - }) - const originalSync = options.sync - const metadata = createMetadata( - new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), - ) - const collection = createCollection({ - ...options, - sync: { - ...originalSync, - sync: (params: Parameters[0]) => - originalSync.sync({ ...params, metadata: metadata.api }), - }, - }) + oracleReplayTest(it)( + `lets an equal-timestamp reset dominate a stale hydrated resume`, + async () => { + let subscriber!: (messages: Array>) => void + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const options = electricCollectionOptions({ + id: `equal-timestamp-reset`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + startSync: true, + }) + options.sync.importSyncMeta?.({ + version: 1, + resume: { + kind: `resume`, + requiresTagState: false, + offset: `10_0`, + handle: `stale-handle`, + shapeId, + updatedAt: 1, + }, + seenTxids: [], + }) + const originalSync = options.sync + const metadata = createMetadata( + new Map([[`electric:resume`, { kind: `reset`, updatedAt: 1 }]]), + ) + const collection = createCollection({ + ...options, + sync: { + ...originalSync, + sync: (params: Parameters[0]) => + originalSync.sync({ ...params, metadata: metadata.api }), + }, + }) - expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ - offset: undefined, - handle: undefined, - }) - subscriber([change(`insert`, 1, `full snapshot`), upToDate]) - expect(collection.status).toBe(`ready`) - await collection.cleanup() - }) + expect(vi.mocked(ShapeStream).mock.calls.at(-1)?.[0]).toMatchObject({ + offset: undefined, + handle: undefined, + }) + subscriber([change(`insert`, 1, `full snapshot`), upToDate]) + expect(collection.status).toBe(`ready`) + await collection.cleanup() + }, + ) electricHistoryProperty( `electric.resume-metadata-merge`, @@ -7362,94 +7423,106 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { ], ) - it(`retains the selected tag requirement when timestamps establish an order`, () => { - const merge = electricCollectionOptions({ - shapeOptions: { url: `http://test-url`, params: { table: `test_table` } }, - getKey: (row) => row.id, - }).sync.mergeSyncMeta! - for (const olderTag of [undefined, false, true]) { - for (const newerTag of [undefined, false, true]) { - const state = (updatedAt: number, tag: boolean | undefined) => ({ - version: 1, - seenTxids: [updatedAt], - resume: { - kind: `resume`, - offset: `10_0`, - handle: `shape-1`, - shapeId, - updatedAt, - ...(tag === undefined ? {} : { requiresTagState: tag }), - }, - }) - const older = state(1, olderTag) - const newer = state(2, newerTag) - const expected = { ...newer, seenTxids: [1, 2] } - expect(merge(older, newer)).toEqual(expected) - expect(merge(newer, older)).toEqual(expected) + oracleReplayTest(it)( + `retains the selected tag requirement when timestamps establish an order`, + () => { + const merge = electricCollectionOptions({ + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table` }, + }, + getKey: (row) => row.id, + }).sync.mergeSyncMeta! + for (const olderTag of [undefined, false, true]) { + for (const newerTag of [undefined, false, true]) { + const state = (updatedAt: number, tag: boolean | undefined) => ({ + version: 1, + seenTxids: [updatedAt], + resume: { + kind: `resume`, + offset: `10_0`, + handle: `shape-1`, + shapeId, + updatedAt, + ...(tag === undefined ? {} : { requiresTagState: tag }), + }, + }) + const older = state(1, olderTag) + const newer = state(2, newerTag) + const expected = { ...newer, seenTxids: [1, 2] } + expect(merge(older, newer)).toEqual(expected) + expect(merge(newer, older)).toEqual(expected) + } } - } - }) + }, + ) - it(`rejects an unseen partial update from an explicit eager resume`, async () => { - const metadata = createMetadata(resumeState()) - const trace = createOracleCollection( - `explicit-eager-resume`, - `eager`, - metadata.api, - { offset: `10_0` as Offset, handle: `explicit-handle` }, - ) + oracleReplayTest(it)( + `rejects an unseen partial update from an explicit eager resume`, + async () => { + const metadata = createMetadata(resumeState()) + const trace = createOracleCollection( + `explicit-eager-resume`, + `eager`, + metadata.api, + { offset: `10_0` as Offset, handle: `explicit-handle` }, + ) - trace.subscriber([change(`update`, 1, `partial`), upToDate]) + trace.subscriber([change(`update`, 1, `partial`), upToDate]) - expect(trace.collection.status).toBe(`error`) - expect(trace.collection.has(1)).toBe(false) - expect(metadata.state.get(`electric:resume`)).toEqual( - expect.objectContaining({ kind: `reset` }), - ) - await trace.collection.cleanup() - }) + expect(trace.collection.status).toBe(`error`) + expect(trace.collection.has(1)).toBe(false) + expect(metadata.state.get(`electric:resume`)).toEqual( + expect.objectContaining({ kind: `reset` }), + ) + await trace.collection.cleanup() + }, + ) - it(`accepts complete replica updates from an explicit eager resume`, async () => { - let subscriber!: (messages: Array>) => void - mockSubscribe.mockImplementationOnce((callback) => { - subscriber = callback - return vi.fn() - }) - const collection = createCollection( - electricCollectionOptions({ - id: `explicit-full-replica-resume`, - shapeOptions: { - url: `http://test-url`, - params: { table: `test_table`, replica: `full` }, - offset: `10_0` as Offset, - handle: `explicit-handle`, - }, - syncMode: `eager`, - getKey: (row) => row.id, - startSync: true, - }), - ) + oracleReplayTest(it)( + `accepts complete replica updates from an explicit eager resume`, + async () => { + let subscriber!: (messages: Array>) => void + mockSubscribe.mockImplementationOnce((callback) => { + subscriber = callback + return vi.fn() + }) + const collection = createCollection( + electricCollectionOptions({ + id: `explicit-full-replica-resume`, + shapeOptions: { + url: `http://test-url`, + params: { table: `test_table`, replica: `full` }, + offset: `10_0` as Offset, + handle: `explicit-handle`, + }, + syncMode: `eager`, + getKey: (row) => row.id, + startSync: true, + }), + ) - subscriber([ - { - key: `1`, - value: { id: 1, name: `complete update`, stable: `stable-1` }, - headers: { operation: `update` }, - }, - upToDate, - ]) + subscriber([ + { + key: `1`, + value: { id: 1, name: `complete update`, stable: `stable-1` }, + headers: { operation: `update` }, + }, + upToDate, + ]) - expect(collection.status).toBe(`ready`) - expect(collection.get(1)).toEqual( - expect.objectContaining({ - name: `complete update`, - stable: `stable-1`, - }), - ) - await collection.cleanup() - }) + expect(collection.status).toBe(`ready`) + expect(collection.get(1)).toEqual( + expect.objectContaining({ + name: `complete update`, + stable: `stable-1`, + }), + ) + await collection.cleanup() + }, + ) - it.each([10, 100])( + oracleReplayTest(it).each([10, 100])( `subset acquisition avoids scanning the applied baseline with %s rows`, async (size) => { const trace = createOracleCollection( @@ -7488,161 +7561,167 @@ describeUnlessQueuedPresenceReplay(`Electric adapter laws`, () => { }, ) - it(`ignores an unseen on-demand update without blocking readiness`, async () => { - const metadata = createMetadata(resumeState()) - const trace = createOracleCollection( - `on-demand-unseen-update`, - `on-demand`, - metadata.api, - ) + oracleReplayTest(it)( + `ignores an unseen on-demand update without blocking readiness`, + async () => { + const metadata = createMetadata(resumeState()) + const trace = createOracleCollection( + `on-demand-unseen-update`, + `on-demand`, + metadata.api, + ) - trace.subscriber([change(`update`, 1, `partial`), upToDate]) + trace.subscriber([change(`update`, 1, `partial`), upToDate]) - expect(trace.collection.status).toBe(`ready`) - expect(trace.collection.has(1)).toBe(false) - expect(metadata.state.get(`electric:resume`)).toEqual( - expect.objectContaining({ kind: `resume`, offset: `20_0` }), - ) - await trace.collection.cleanup() - }) + expect(trace.collection.status).toBe(`ready`) + expect(trace.collection.has(1)).toBe(false) + expect(metadata.state.get(`electric:resume`)).toEqual( + expect.objectContaining({ kind: `resume`, offset: `20_0` }), + ) + await trace.collection.cleanup() + }, + ) - it(`records match and txid evidence for an ignored on-demand update`, async () => { - const trace = createOracleCollection( - `on-demand-ignored-update-evidence`, - `on-demand`, - createMetadata(resumeState()).api, - ) - trace.subscriber([upToDate]) + oracleReplayTest(it)( + `records match and txid evidence for an ignored on-demand update`, + async () => { + const trace = createOracleCollection( + `on-demand-ignored-update-evidence`, + `on-demand`, + createMetadata(resumeState()).api, + ) + trace.subscriber([upToDate]) - const pendingMatch = trace.collection.utils.awaitMatch( - (message) => `value` in message && message.value.id === 808, - 100, - ) - const pendingTxid = trace.collection.utils.awaitTxId(808, 100) - const update = change(`update`, 808, `ignored`) - update.headers.txids = [808] + const pendingMatch = trace.collection.utils.awaitMatch( + (message) => `value` in message && message.value.id === 808, + 100, + ) + const pendingTxid = trace.collection.utils.awaitTxId(808, 100) + const update = change(`update`, 808, `ignored`) + update.headers.txids = [808] - trace.subscriber([update, upToDate]) + trace.subscriber([update, upToDate]) - await Promise.all([ - expect(pendingMatch).resolves.toBe(true), - expect(pendingTxid).resolves.toBe(true), - ]) - expect(trace.collection.has(808)).toBe(false) - await trace.collection.cleanup() - }) + await Promise.all([ + expect(pendingMatch).resolves.toBe(true), + expect(pendingTxid).resolves.toBe(true), + ]) + expect(trace.collection.has(808)).toBe(false) + await trace.collection.cleanup() + }, + ) - it(`reconstructs persistence-interleaving campaigns and rejects ablated, out-of-range, and foreign histories`, () => { - const campaigns = fc.sample(persistenceInterleavingCampaignArbitrary, { - seed: 18_530_601, - numRuns: 20, - }) - for (const campaign of campaigns) { - expect(reconstructPersistenceInterleavingCampaign(campaign)).toEqual( - campaign, - ) - expect(() => - reconstructPersistenceInterleavingCampaign({ - names: campaign.names, - }), - ).toThrow() - expect(() => - reconstructPersistenceInterleavingCampaign({ - histories: campaign.histories, - }), - ).toThrow() + oracleReplayTest(it)( + `reconstructs persistence-interleaving campaigns and rejects ablated, out-of-range, and foreign histories`, + () => { + const campaigns = fc.sample(persistenceInterleavingCampaignArbitrary, { + seed: 18_530_601, + numRuns: 20, + }) + for (const campaign of campaigns) { + expect(reconstructPersistenceInterleavingCampaign(campaign)).toEqual( + campaign, + ) + expect(() => + reconstructPersistenceInterleavingCampaign({ + names: campaign.names, + }), + ).toThrow() + expect(() => + reconstructPersistenceInterleavingCampaign({ + histories: campaign.histories, + }), + ).toThrow() + expect(() => + reconstructPersistenceInterleavingCampaign({ + ...campaign, + histories: [`open-transaction-hydration`, `foreign`], + }), + ).toThrow() + expect(() => + reconstructPersistenceInterleavingCampaign({ + ...campaign, + names: [``, campaign.names[1]], + }), + ).toThrow() + } + }, + ) + + oracleReplayTest(it)( + `rejects named wrong answers for persistence interleavings`, + () => { + const names: [string, string] = [`before`, `after`] expect(() => - reconstructPersistenceInterleavingCampaign({ - ...campaign, - histories: [`open-transaction-hydration`, `foreign`], - }), + expectPersistenceInterleavingObservation( + { + kind: `open-transaction-hydration`, + hydrationStartedBeforeCommit: true, + hydrationOutcomeBeforeRelease: `AbortError`, + cachedRowBeforeRelease: true, + hydrationOutcome: `fulfilled`, + publicRowsAtHydrationSettlement: [], + deliveryErrors: [`InvalidPersistedCollectionConfigError`], + publicRows: [], + durableRows: [], + status: `ready`, + publicError: undefined, + }, + names, + ), ).toThrow() expect(() => - reconstructPersistenceInterleavingCampaign({ - ...campaign, - names: [``, campaign.names[1]], - }), + expectPersistenceInterleavingObservation( + { + kind: `subset-supersedes-absence`, + publicRows: [], + durableRows: [], + status: `ready`, + publicError: undefined, + }, + names, + ), ).toThrow() - } - }) - - it(`rejects named wrong answers for persistence interleavings`, () => { - const names: [string, string] = [`before`, `after`] - expect(() => - expectPersistenceInterleavingObservation( - { - kind: `open-transaction-hydration`, - hydrationStartedBeforeCommit: true, - hydrationOutcomeBeforeRelease: `AbortError`, - cachedRowBeforeRelease: true, - hydrationOutcome: `fulfilled`, - publicRowsAtHydrationSettlement: [], - deliveryErrors: [`InvalidPersistedCollectionConfigError`], - publicRows: [], - durableRows: [], - status: `ready`, - publicError: undefined, - }, - names, - ), - ).toThrow() - expect(() => - expectPersistenceInterleavingObservation( - { - kind: `subset-supersedes-absence`, - publicRows: [], - durableRows: [], - status: `ready`, - publicError: undefined, - }, - names, - ), - ).toThrow() - }) + }, + ) - fcTest.prop([persistenceInterleavingCampaignArbitrary], { + oracleReplayTest(fcTest).prop([persistenceInterleavingCampaignArbitrary], { seed: 18_530_601, numRuns: oracleRuns(4), })( `preserves persisted Electric interleavings (fixed)`, runPersistenceInterleavingCampaign, ) - - fcTest.prop( - [persistenceInterleavingCampaignArbitrary], - oraclePropertyOptions(4, persistenceInterleavingProperty), - )( - `preserves persisted Electric interleavings (random)`, - runPersistenceInterleavingCampaign, - ) }) -if ( - requestedPersistenceInterleavingProperty === persistenceInterleavingProperty -) { - describe(`persisted Electric interleaving replay`, () => { +// Each campaign's random lane has its own suite. An ordinary run runs it +// beside the adapter laws. A path replay of its property runs only it. +oracleReplayTest(describe, persistenceInterleavingProperty)( + `persisted Electric interleaving campaign`, + () => { beforeEach(resetElectricOracleMocks) fcTest.prop( [persistenceInterleavingCampaignArbitrary], oraclePropertyOptions(4, persistenceInterleavingProperty), )( - `preserves persisted Electric interleavings (replay)`, + `preserves persisted Electric interleavings (random)`, runPersistenceInterleavingCampaign, ) - }) -} + }, +) -if (requestedQueuedPresenceProperty === queuedPresenceProperty) { - describe(`Electric queued-presence replay`, () => { +oracleReplayTest(describe, queuedPresenceProperty)( + `Electric queued-presence campaign`, + () => { beforeEach(resetElectricOracleMocks) fcTest.prop( queuedPresenceArbitraries, - queuedPresencePropertyOptions(`replay`), + queuedPresencePropertyOptions(`random`), )( - `generated queued Electric histories preserve staged presence and reset fences (replay)`, + `generated queued Electric histories preserve staged presence and reset fences (random)`, runQueuedPresenceCampaign, ) - }) -} + }, +) diff --git a/packages/electric-db-collection/tests/electric-sdk-delivery-oracle.property.test.ts b/packages/electric-db-collection/tests/electric-sdk-delivery-oracle.property.test.ts index 832b2b8a0d..c60c541519 100644 --- a/packages/electric-db-collection/tests/electric-sdk-delivery-oracle.property.test.ts +++ b/packages/electric-db-collection/tests/electric-sdk-delivery-oracle.property.test.ts @@ -16,7 +16,7 @@ import { import { electricCollectionOptions } from '../src/electric' import { oraclePropertyOptions, - readOracleRunConfig, + oracleReplayTest, } from '../../db/tests/oracle-config' import { managedTagPersistence, @@ -94,8 +94,10 @@ import type { Message } from '@electric-sql/client' // TANSTACK_DB_ORACLE_SEED=20260917 TANSTACK_DB_ORACLE_PATH=0 \ // TANSTACK_DB_ORACLE_PROPERTY=electric.sdk-snapshot-delivery \ // pnpm --dir packages/electric-db-collection exec vitest run tests/electric-sdk-delivery-oracle.property.test.ts -t 'replay' -const { replayPath, replayProperty } = readOracleRunConfig() -const fixedCase = replayPath === undefined ? it : it.skip +// Ordinary tests stand aside during any path replay, which runs one property. +const fixedCase = oracleReplayTest(it) +// TANSTACK_DB_ELECTRIC_SDK_DELIVERY_FAULT is a fault-injection switch, not a +// replay variable. It stays readable during replays. const replayFault = process.env.TANSTACK_DB_ELECTRIC_SDK_DELIVERY_FAULT if ( replayFault !== undefined && @@ -120,13 +122,10 @@ function sdkCampaigns( throw violation.report(error) } } - if (replayPath !== undefined) { - if (replayProperty === propertyId) - it(`${name} (replay)`, () => runCampaign()) - return - } - it(`${name} (fixed)`, () => runCampaign(fixedSeed)) - it(`${name} (random)`, () => runCampaign()) + // A path replay of the property runs only the random campaign, from its + // path. The fixed campaign stands aside during every path replay. + oracleReplayTest(it)(`${name} (fixed)`, () => runCampaign(fixedSeed)) + oracleReplayTest(it, propertyId)(`${name} (random)`, () => runCampaign()) } function violationCheckpoint(error: unknown): string { diff --git a/packages/electric-db-collection/tests/pg-serializer-oracle.property.test.ts b/packages/electric-db-collection/tests/pg-serializer-oracle.property.test.ts index 322e254768..c75e38fb9e 100644 --- a/packages/electric-db-collection/tests/pg-serializer-oracle.property.test.ts +++ b/packages/electric-db-collection/tests/pg-serializer-oracle.property.test.ts @@ -1,78 +1,38 @@ import { describe, expect, it } from 'vitest' import { fc, test as fcTest } from '@fast-check/vitest' import { serialize } from '../src/pg-serializer' - -// Replay one named law directly with PG_SERIALIZER_ORACLE_PROPERTY=, -// PG_SERIALIZER_ORACLE_SEED=, and -// PG_SERIALIZER_ORACLE_PATH=. -// Example: pnpm exec vitest run tests/pg-serializer-oracle.property.test.ts -t 'mixed finite arrays retain ordered typed contents' -const replayProperty = process.env.PG_SERIALIZER_ORACLE_PROPERTY -const replaySeedText = process.env.PG_SERIALIZER_ORACLE_SEED -const replayPath = process.env.PG_SERIALIZER_ORACLE_PATH -const runsText = process.env.PG_SERIALIZER_ORACLE_RUNS ?? `100` -const runs = Number(runsText) +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' + +// Replay one named law with TANSTACK_DB_ORACLE_SEED=, +// TANSTACK_DB_ORACLE_PATH=, and +// TANSTACK_DB_ORACLE_PROPERTY=. Example: +// TANSTACK_DB_ORACLE_SEED=1 TANSTACK_DB_ORACLE_PATH=0 \ +// TANSTACK_DB_ORACLE_PROPERTY=pg-serializer.mixed-array-contents \ +// pnpm exec vitest run tests/pg-serializer-oracle.property.test.ts const registeredProperties = new Set() -const example = replayProperty === undefined ? it : it.skip - -if (!Number.isSafeInteger(runs) || runs < 1) { - throw new Error(`PG_SERIALIZER_ORACLE_RUNS must be a positive integer`) -} -if ( - replayProperty === undefined && - (replaySeedText !== undefined || replayPath !== undefined) -) { - throw new Error(`PostgreSQL serializer replay requires a property name`) -} -if ( - replayProperty !== undefined && - (replaySeedText === undefined || replayPath === undefined) -) { - throw new Error(`PostgreSQL serializer replay requires seed and path`) -} -const replaySeed = - replaySeedText === undefined ? undefined : Number(replaySeedText) -if ( - replaySeedText !== undefined && - (replaySeedText.trim() === `` || !Number.isSafeInteger(replaySeed)) -) { - throw new Error(`PG_SERIALIZER_ORACLE_SEED must be an integer`) -} -if (replayPath !== undefined && !/^\d+(?::\d+)*$/.test(replayPath)) { - throw new Error( - `PG_SERIALIZER_ORACLE_PATH must contain colon-separated nonnegative integers`, - ) -} +// Examples stand aside during any path replay, which runs one property. +const example = oracleReplayTest(it) -function serializerReplayOptions( - seed: number | undefined, - path: string | undefined, -) { - return { numRuns: runs, seed, path } -} - -function serializerProp([arbitrary]: [fc.Arbitrary]) { +function serializerProp(property: string, [arbitrary]: [fc.Arbitrary]) { return (name: string, check: (value: T) => void): void => { if (registeredProperties.has(name)) { throw new Error(`duplicate PostgreSQL serializer property: ${name}`) } registeredProperties.add(name) - if (replayProperty === undefined) { - fcTest.prop([arbitrary], { numRuns: runs, seed: 20260930 })( - `${name} (fixed)`, - check, - ) - fcTest.prop([arbitrary], { numRuns: runs })(`${name} (random)`, check) - return - } - if (replayProperty === name) { - fcTest.prop([arbitrary], serializerReplayOptions(replaySeed, replayPath))( - `${name} (replay)`, - check, - ) - } else { - // Vitest requires each describe block to register at least one test. - it.skip(`${name} (not requested)`, () => {}) - } + // A path replay of the property runs only the random campaign, from its + // path. The fixed campaign stands aside during every path replay. + oracleReplayTest(fcTest).prop([arbitrary], { + numRuns: oracleRuns(100), + seed: 20260930, + })(`${name} (fixed)`, check) + oracleReplayTest(fcTest, property).prop( + [arbitrary], + oraclePropertyOptions(100, property), + )(`${name} (random)`, check) } } @@ -189,20 +149,26 @@ function expectFiniteArrayContents( describe(`pg-serializer property-based tests`, () => { describe(`string serialization`, () => { - serializerProp([fc.string()])(`strings pass through unchanged`, (str) => { - expect(serialize(str)).toBe(str) - }) + serializerProp(`pg-serializer.string-passthrough`, [fc.string()])( + `strings pass through unchanged`, + (str) => { + expect(serialize(str)).toBe(str) + }, + ) - serializerProp([fc.string()])(`strings are idempotent`, (str) => { - // serialize(serialize(str)) should equal serialize(str) for strings - const once = serialize(str) - const twice = serialize(once) - expect(twice).toBe(once) - }) + serializerProp(`pg-serializer.string-idempotent`, [fc.string()])( + `strings are idempotent`, + (str) => { + // serialize(serialize(str)) should equal serialize(str) for strings + const once = serialize(str) + const twice = serialize(once) + expect(twice).toBe(once) + }, + ) }) describe(`number serialization`, () => { - serializerProp([fc.integer()])( + serializerProp(`pg-serializer.integer-round-trip`, [fc.integer()])( `integers round-trip through strict numeric parsing`, (n) => { const serialized = serialize(n) @@ -210,7 +176,7 @@ describe(`pg-serializer property-based tests`, () => { }, ) - serializerProp([ + serializerProp(`pg-serializer.double-round-trip`, [ fc .double({ noNaN: true, noDefaultInfinity: true }) .filter((n) => !Object.is(n, -0)), @@ -220,14 +186,17 @@ describe(`pg-serializer property-based tests`, () => { expect(parseFiniteNumber(serialized)).toBe(n) }) - serializerProp([fc.integer()])(`integers produce numeric strings`, (n) => { - const serialized = serialize(n) - expect(serialized).toMatch(/^-?\d+$/) - }) + serializerProp(`pg-serializer.integer-text`, [fc.integer()])( + `integers produce numeric strings`, + (n) => { + const serialized = serialize(n) + expect(serialized).toMatch(/^-?\d+$/) + }, + ) }) describe(`bigint serialization`, () => { - serializerProp([fc.bigInt()])( + serializerProp(`pg-serializer.bigint-round-trip`, [fc.bigInt()])( `bigints round-trip through BigInt parsing`, (n) => { const serialized = serialize(n) @@ -235,14 +204,17 @@ describe(`pg-serializer property-based tests`, () => { }, ) - serializerProp([fc.bigInt()])(`bigints produce integer strings`, (n) => { - const serialized = serialize(n) - expect(serialized).toMatch(/^-?\d+$/) - }) + serializerProp(`pg-serializer.bigint-text`, [fc.bigInt()])( + `bigints produce integer strings`, + (n) => { + const serialized = serialize(n) + expect(serialized).toMatch(/^-?\d+$/) + }, + ) }) describe(`boolean serialization`, () => { - serializerProp([fc.boolean()])( + serializerProp(`pg-serializer.boolean-tokens`, [fc.boolean()])( `booleans serialize to 'true' or 'false'`, (b) => { const serialized = serialize(b) @@ -250,7 +222,7 @@ describe(`pg-serializer property-based tests`, () => { }, ) - serializerProp([fc.boolean()])( + serializerProp(`pg-serializer.boolean-round-trip`, [fc.boolean()])( `booleans round-trip through string comparison`, (b) => { const serialized = serialize(b) @@ -260,12 +232,11 @@ describe(`pg-serializer property-based tests`, () => { }) describe(`null/undefined serialization`, () => { - serializerProp([fc.constantFrom(null, undefined)])( - `null and undefined both serialize to empty string`, - (val) => { - expect(serialize(val)).toBe(``) - }, - ) + serializerProp(`pg-serializer.nullish-empty`, [ + fc.constantFrom(null, undefined), + ])(`null and undefined both serialize to empty string`, (val) => { + expect(serialize(val)).toBe(``) + }) }) describe(`date serialization`, () => { @@ -283,7 +254,7 @@ describe(`pg-serializer property-based tests`, () => { expect(serialize(date)).toBe(text) }) - serializerProp([arbitraryBoundedDate])( + serializerProp(`pg-serializer.date-iso`, [arbitraryBoundedDate])( `dates produce valid ISO strings`, (date) => { const serialized = serialize(date) @@ -293,7 +264,7 @@ describe(`pg-serializer property-based tests`, () => { }, ) - serializerProp([arbitraryBoundedDate])( + serializerProp(`pg-serializer.date-round-trip`, [arbitraryBoundedDate])( `dates round-trip through Date parsing`, (date) => { const serialized = serialize(date) @@ -309,32 +280,29 @@ describe(`pg-serializer property-based tests`, () => { expectFiniteArrayContents(input, serialize(input)) }) - serializerProp([fc.array(fc.integer(), { maxLength: 10 })])( - `integer arrays produce Postgres array format`, - (arr) => { - const serialized = serialize(arr) - expect(serialized).toMatch(/^\{.*\}$/) - }, - ) + serializerProp(`pg-serializer.integer-array-format`, [ + fc.array(fc.integer(), { maxLength: 10 }), + ])(`integer arrays produce Postgres array format`, (arr) => { + const serialized = serialize(arr) + expect(serialized).toMatch(/^\{.*\}$/) + }) - serializerProp([fc.array(fc.integer(), { maxLength: 10 })])( - `integer arrays can be parsed back`, - (arr) => { - const serialized = serialize(arr) - expectFiniteArrayContents(arr, serialized) - }, - ) + serializerProp(`pg-serializer.integer-array-contents`, [ + fc.array(fc.integer(), { maxLength: 10 }), + ])(`integer arrays can be parsed back`, (arr) => { + const serialized = serialize(arr) + expectFiniteArrayContents(arr, serialized) + }) - serializerProp([fc.array(fc.boolean(), { maxLength: 10 })])( - `boolean arrays serialize correctly`, - (arr) => { - const serialized = serialize(arr) - expect(serialized).toMatch(/^\{.*\}$/) - expectFiniteArrayContents(arr, serialized) - }, - ) + serializerProp(`pg-serializer.boolean-array-contents`, [ + fc.array(fc.boolean(), { maxLength: 10 }), + ])(`boolean arrays serialize correctly`, (arr) => { + const serialized = serialize(arr) + expect(serialized).toMatch(/^\{.*\}$/) + expectFiniteArrayContents(arr, serialized) + }) - serializerProp([ + serializerProp(`pg-serializer.nullish-array-contents`, [ fc.array(fc.constantFrom(null, undefined), { maxLength: 5 }), ])(`arrays with null/undefined serialize to NULL`, (arr) => { const serialized = serialize(arr) @@ -343,16 +311,15 @@ describe(`pg-serializer property-based tests`, () => { }) describe(`string array escaping`, () => { - serializerProp([fc.array(fc.string(), { maxLength: 5 })])( - `string arrays are properly quoted`, - (arr) => { - const serialized = serialize(arr) - expect(serialized).toMatch(/^\{.*\}$/) - expectFiniteArrayContents(arr, serialized) - }, - ) + serializerProp(`pg-serializer.string-array-contents`, [ + fc.array(fc.string(), { maxLength: 5 }), + ])(`string arrays are properly quoted`, (arr) => { + const serialized = serialize(arr) + expect(serialized).toMatch(/^\{.*\}$/) + expectFiniteArrayContents(arr, serialized) + }) - serializerProp([ + serializerProp(`pg-serializer.string-escaping`, [ fc .tuple(fc.string(), fc.constantFrom(`"`, `\\`), fc.string()) .map((parts) => parts.join(``)), @@ -370,7 +337,7 @@ describe(`pg-serializer property-based tests`, () => { }) describe(`consistency properties`, () => { - serializerProp([ + serializerProp(`pg-serializer.scalar-determinism`, [ fc.oneof( fc.string(), fc.integer(), @@ -384,14 +351,13 @@ describe(`pg-serializer property-based tests`, () => { expect(first).toBe(second) }) - serializerProp([fc.array(fc.integer(), { maxLength: 10 })])( - `array serialization is deterministic`, - (arr) => { - const first = serialize(arr) - const second = serialize(arr) - expect(first).toBe(second) - }, - ) + serializerProp(`pg-serializer.array-determinism`, [ + fc.array(fc.integer(), { maxLength: 10 }), + ])(`array serialization is deterministic`, (arr) => { + const first = serialize(arr) + const second = serialize(arr) + expect(first).toBe(second) + }) }) describe(`finite scalar and array checker calibration`, () => { @@ -526,10 +492,11 @@ describe(`pg-serializer property-based tests`, () => { } expect(failure.errorInstance).toMatchObject({ name: `AssertionError` }) expect(failure.counterexample[0]).toHaveLength(1) - const replay = fc.check( - property, - serializerReplayOptions(failure.seed, failure.counterexamplePath), - ) + const replay = fc.check(property, { + numRuns: oracleRuns(100), + seed: failure.seed, + path: failure.counterexamplePath, + }) expect(replay.failed).toBe(true) expect(replay.counterexample).toEqual(failure.counterexample) expect(replay.errorInstance).toMatchObject({ name: `AssertionError` }) @@ -553,7 +520,7 @@ describe(`pg-serializer property-based tests`, () => { expect(() => parseFiniteArray(output)).toThrow() }) - serializerProp([ + serializerProp(`pg-serializer.mixed-array-contents`, [ fc.array( fc.oneof( fc.string(), diff --git a/packages/node-db-sqlite-persistence/tests/expression-index-oracle.test.ts b/packages/node-db-sqlite-persistence/tests/expression-index-oracle.test.ts index 8b4639cf58..9ba77b9fe7 100644 --- a/packages/node-db-sqlite-persistence/tests/expression-index-oracle.test.ts +++ b/packages/node-db-sqlite-persistence/tests/expression-index-oracle.test.ts @@ -45,10 +45,12 @@ * * Reach, challenge, replay, and cleanup: the expression-index matrix and both * generated BigInt range checks run in fixed-seed and seedless-random campaigns. - * Supplying both TANSTACK_DB_WS5A_SEED and TANSTACK_DB_WS5A_PATH selects only - * the exact replay campaign; TANSTACK_DB_WS5A_PROPERTY selects a BigInt check - * instead of the default expression-index matrix. Either replay value alone - * rejects. Per-axis grammar ablations and same-path SQL/compiler faults must + * A path replay sets TANSTACK_DB_ORACLE_PATH, TANSTACK_DB_ORACLE_SEED, and + * TANSTACK_DB_ORACLE_PROPERTY to one of sqlite-persistence.expression-index, + * sqlite-persistence.bigint-roundtrip, or sqlite-persistence.bigint-rejection. + * That property's random campaign replays only its recorded history; the fixed + * campaigns and the other properties stand aside. A seed alone reseeds the + * random campaigns. Per-axis grammar ablations and same-path SQL/compiler faults must * fail the expression-index property. The * retained fixed campaign reconstructs the known valid matrix. The bounds * above state its range; exact grammar checks reject missing, duplicate, and @@ -70,6 +72,12 @@ import { decodePersistedStorageKey, persistedCollectionOptions, } from '@tanstack/db-sqlite-persistence-core' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, + readOracleRunConfig, +} from '../../db/tests/oracle-config.js' import { Temporal, temporalFamilies, @@ -133,84 +141,35 @@ const legalPathArbitrary = fc ) .map(([root, tail]) => [root, ...tail]) -type OracleReplayConfiguration = { - seed: number - path: string - numRuns: 1 -} +const EXPRESSION_INDEX_PROPERTY = `sqlite-persistence.expression-index` +const BIGINT_ROUNDTRIP_PROPERTY = `sqlite-persistence.bigint-roundtrip` +const BIGINT_REJECTION_PROPERTY = `sqlite-persistence.bigint-rejection` type GeneratedPropertyCampaign = { - name: `fixed` | `random` | `replay` - seed?: number - path?: string - numRuns: number + name: `fixed` | `random` + parameters: ReturnType } -function oracleReplayConfiguration( - environment: NodeJS.ProcessEnv = process.env, -): OracleReplayConfiguration | undefined { - const seedText = environment.TANSTACK_DB_WS5A_SEED - const path = environment.TANSTACK_DB_WS5A_PATH - if (seedText !== undefined && !/^-?\d+$/.test(seedText)) { - throw new Error(`TANSTACK_DB_WS5A_SEED must be an integer`) - } - if (seedText !== undefined && !Number.isSafeInteger(Number(seedText))) { - throw new Error(`TANSTACK_DB_WS5A_SEED must be an integer`) - } - if (path !== undefined && !/^\d+(?::\d+)*$/.test(path)) { - throw new Error( - `TANSTACK_DB_WS5A_PATH must contain colon-separated nonnegative integers`, - ) - } - if (seedText === undefined && path === undefined) return undefined - if (seedText === undefined) { - throw new Error( - `TANSTACK_DB_WS5A_PATH requires TANSTACK_DB_WS5A_SEED for exact replay`, - ) - } - if (path === undefined) { - throw new Error( - `TANSTACK_DB_WS5A_SEED requires TANSTACK_DB_WS5A_PATH for exact replay`, - ) - } +// The fixed campaign pins its seed and stands aside on every path replay. The +// random campaign runs in ordinary and seed-only runs, and as the replay of +// its property, where oraclePropertyOptions returns the recorded path. +const fixedGeneratedCampaign: GeneratedPropertyCampaign = { + name: `fixed`, + parameters: { + seed: DEFAULT_ORACLE_SEED, + numRuns: oracleRuns(DEFAULT_ORACLE_RUNS), + }, +} +function randomGeneratedCampaign(property: string): GeneratedPropertyCampaign { return { - seed: Number(seedText), - path, - numRuns: 1, + name: `random`, + parameters: oraclePropertyOptions(DEFAULT_ORACLE_RUNS, property), } } -const requestedReplay = oracleReplayConfiguration() -const requestedReplayProperty = - process.env.TANSTACK_DB_WS5A_PROPERTY ?? `expression-index` -if ( - ![`expression-index`, `bigint-roundtrip`, `bigint-rejection`].includes( - requestedReplayProperty, - ) -) { - throw new Error(`TANSTACK_DB_WS5A_PROPERTY must name a generated property`) -} - -const generatedPropertyCampaigns: Array = - requestedReplay === undefined - ? [ - { - name: `fixed`, - seed: DEFAULT_ORACLE_SEED, - numRuns: DEFAULT_ORACLE_RUNS, - }, - { name: `random`, numRuns: DEFAULT_ORACLE_RUNS }, - ] - : [{ name: `replay`, ...requestedReplay }] - function generatedCampaignParameters(campaign: GeneratedPropertyCampaign) { - return { - numRuns: campaign.numRuns, - verbose: 2 as const, - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - ...(campaign.path === undefined ? {} : { path: campaign.path }), - } + return { verbose: 2 as const, ...campaign.parameters } } function createNestedRow( @@ -1335,44 +1294,61 @@ async function assertExpressionIndexHistory( } describe(`SQLite expression-index oracle`, () => { - const it = requestedReplay === undefined ? vitestIt : vitestIt.skip + const it = oracleReplayTest(vitestIt) it.each([ { label: `empty seed`, - environment: { TANSTACK_DB_WS5A_SEED: `` }, - message: `TANSTACK_DB_WS5A_SEED must be an integer`, + environment: { TANSTACK_DB_ORACLE_SEED: `` }, + message: `TANSTACK_DB_ORACLE_SEED must be an integer`, }, { - label: `seed without path`, - environment: { TANSTACK_DB_WS5A_SEED: `1659005` }, - message: `TANSTACK_DB_WS5A_SEED requires TANSTACK_DB_WS5A_PATH for exact replay`, + label: `path without seed`, + environment: { TANSTACK_DB_ORACLE_PATH: `0` }, + message: `TANSTACK_DB_ORACLE_PATH requires TANSTACK_DB_ORACLE_SEED`, }, { - label: `path without seed`, - environment: { TANSTACK_DB_WS5A_PATH: `0` }, - message: `TANSTACK_DB_WS5A_PATH requires TANSTACK_DB_WS5A_SEED for exact replay`, + label: `property without path`, + environment: { + TANSTACK_DB_ORACLE_SEED: `1659005`, + TANSTACK_DB_ORACLE_PROPERTY: `sqlite-persistence.expression-index`, + }, + message: `TANSTACK_DB_ORACLE_PROPERTY requires TANSTACK_DB_ORACLE_PATH`, }, { label: `invalid shrink path`, environment: { - TANSTACK_DB_WS5A_SEED: `1659005`, - TANSTACK_DB_WS5A_PATH: `0:`, + TANSTACK_DB_ORACLE_SEED: `1659005`, + TANSTACK_DB_ORACLE_PATH: `0:`, + TANSTACK_DB_ORACLE_PROPERTY: `sqlite-persistence.expression-index`, }, - message: `TANSTACK_DB_WS5A_PATH must contain colon-separated nonnegative integers`, + message: `TANSTACK_DB_ORACLE_PATH must contain colon-separated nonnegative integers`, }, ])(`rejects a $label replay configuration`, ({ environment, message }) => { - expect(() => oracleReplayConfiguration(environment)).toThrow(message) + expect(() => readOracleRunConfig(environment)).toThrow(message) }) - it(`accepts only a checked seed and shrink-path replay pair`, () => { + it(`accepts a seed alone and a checked seed and shrink-path replay pair`, () => { + expect(readOracleRunConfig({ TANSTACK_DB_ORACLE_SEED: `1659005` })).toEqual( + { + multiplier: 1, + replaySeed: 1659005, + replayPath: undefined, + replayProperty: undefined, + }, + ) expect( - oracleReplayConfiguration({ - TANSTACK_DB_WS5A_SEED: `-1659005`, - TANSTACK_DB_WS5A_PATH: `0:1:2`, + readOracleRunConfig({ + TANSTACK_DB_ORACLE_SEED: `-1659005`, + TANSTACK_DB_ORACLE_PATH: `0:1:2`, + TANSTACK_DB_ORACLE_PROPERTY: `sqlite-persistence.expression-index`, }), - ).toEqual({ seed: -1659005, path: `0:1:2`, numRuns: 1 }) - expect(oracleReplayConfiguration({})).toBeUndefined() + ).toEqual({ + multiplier: 1, + replaySeed: -1659005, + replayPath: `0:1:2`, + replayProperty: `sqlite-persistence.expression-index`, + }) }) it.each([ @@ -2912,112 +2888,114 @@ describe(`SQLite expression-index oracle`, () => { }, [() => driver.close()]) }) - const generatedBigIntRoundTripTest = - requestedReplay === undefined || - requestedReplayProperty === `bigint-roundtrip` - ? vitestIt - : vitestIt.skip + async function roundTripGeneratedBigInt(campaign: GeneratedPropertyCampaign) { + await fc.assert( + fc.asyncProperty( + fc.bigInt({ min: SQLITE_BIGINT_MIN, max: SQLITE_BIGINT_MAX }), + async (value) => { + const driver = new BetterSqlite3SQLiteDriver({ + filename: `:memory:`, + }) + const adapter = createSQLiteCorePersistenceAdapter({ driver }) + const collectionId = `generated-bigint-in-range` - generatedBigIntRoundTripTest.each(generatedPropertyCampaigns)( - `round-trips generated signed-64-bit BigInts through the real adapter in the $name campaign`, - async (campaign) => { - await fc.assert( - fc.asyncProperty( - fc.bigInt({ min: SQLITE_BIGINT_MIN, max: SQLITE_BIGINT_MAX }), - async (value) => { - const driver = new BetterSqlite3SQLiteDriver({ - filename: `:memory:`, - }) - const adapter = createSQLiteCorePersistenceAdapter({ driver }) - const collectionId = `generated-bigint-in-range` - - await withFailurePreservingCleanup(async () => { - await adapter.applyCommittedTx( - { kind: `eager`, collectionId: collectionId }, - { - txId: `seed-${value}`, - term: 1, - seq: 1, - rowVersion: 1, - mutations: [ - { - type: `insert`, - key: `match`, - value: { nested: { count: value } }, - }, - ], - }, - ) + await withFailurePreservingCleanup(async () => { + await adapter.applyCommittedTx( + { kind: `eager`, collectionId: collectionId }, + { + txId: `seed-${value}`, + term: 1, + seq: 1, + rowVersion: 1, + mutations: [ + { + type: `insert`, + key: `match`, + value: { nested: { count: value } }, + }, + ], + }, + ) - const rows = await adapter.loadSubset( - { kind: `eager`, collectionId: collectionId }, - { - where: new IR.Func(`eq`, [ - new IR.PropRef([`nested`, `count`]), - new IR.Value(value), - ]), - }, - ) - expect(rows.map((row) => row.key)).toEqual([`match`]) - expect(rows[0]?.value).toEqual({ nested: { count: value } }) - }, [() => driver.close()]) - }, - ), - generatedCampaignParameters(campaign), - ) - }, + const rows = await adapter.loadSubset( + { kind: `eager`, collectionId: collectionId }, + { + where: new IR.Func(`eq`, [ + new IR.PropRef([`nested`, `count`]), + new IR.Value(value), + ]), + }, + ) + expect(rows.map((row) => row.key)).toEqual([`match`]) + expect(rows[0]?.value).toEqual({ nested: { count: value } }) + }, [() => driver.close()]) + }, + ), + generatedCampaignParameters(campaign), + ) + } + oracleReplayTest(vitestIt).each([fixedGeneratedCampaign])( + `round-trips generated signed-64-bit BigInts through the real adapter in the $name campaign`, + roundTripGeneratedBigInt, + ) + oracleReplayTest(vitestIt, BIGINT_ROUNDTRIP_PROPERTY).each([ + randomGeneratedCampaign(BIGINT_ROUNDTRIP_PROPERTY), + ])( + `round-trips generated signed-64-bit BigInts through the real adapter in the $name campaign`, + roundTripGeneratedBigInt, ) - const generatedBigIntRejectionTest = - requestedReplay === undefined || - requestedReplayProperty === `bigint-rejection` - ? vitestIt - : vitestIt.skip - - generatedBigIntRejectionTest.each(generatedPropertyCampaigns)( - `rejects generated BigInts immediately outside the signed range in the $name campaign`, - async (campaign) => { - const outOfRangeBigIntArbitrary = fc.oneof( - fc - .bigInt({ min: 1n, max: 1_000n }) - .map((distance) => SQLITE_BIGINT_MIN - distance), - fc - .bigInt({ min: 1n, max: 1_000n }) - .map((distance) => SQLITE_BIGINT_MAX + distance), - ) + async function rejectGeneratedBigInt(campaign: GeneratedPropertyCampaign) { + const outOfRangeBigIntArbitrary = fc.oneof( + fc + .bigInt({ min: 1n, max: 1_000n }) + .map((distance) => SQLITE_BIGINT_MIN - distance), + fc + .bigInt({ min: 1n, max: 1_000n }) + .map((distance) => SQLITE_BIGINT_MAX + distance), + ) - await fc.assert( - fc.asyncProperty(outOfRangeBigIntArbitrary, async (value) => { - const driver = new BetterSqlite3SQLiteDriver({ filename: `:memory:` }) - const adapter = createSQLiteCorePersistenceAdapter({ driver }) + await fc.assert( + fc.asyncProperty(outOfRangeBigIntArbitrary, async (value) => { + const driver = new BetterSqlite3SQLiteDriver({ filename: `:memory:` }) + const adapter = createSQLiteCorePersistenceAdapter({ driver }) - await withFailurePreservingCleanup(async () => { - await expect( - adapter.applyCommittedTx( - { - kind: `eager`, - collectionId: `generated-bigint-out-of-range`, - }, - { - txId: `reject-${value}`, - term: 1, - seq: 1, - rowVersion: 1, - mutations: [ - { - type: `insert`, - key: `rejected`, - value: { nested: { count: value } }, - }, - ], - }, - ), - ).rejects.toThrow(bigintRangeError(value)) - }, [() => driver.close()]) - }), - generatedCampaignParameters(campaign), - ) - }, + await withFailurePreservingCleanup(async () => { + await expect( + adapter.applyCommittedTx( + { + kind: `eager`, + collectionId: `generated-bigint-out-of-range`, + }, + { + txId: `reject-${value}`, + term: 1, + seq: 1, + rowVersion: 1, + mutations: [ + { + type: `insert`, + key: `rejected`, + value: { nested: { count: value } }, + }, + ], + }, + ), + ).rejects.toThrow(bigintRangeError(value)) + }, [() => driver.close()]) + }), + generatedCampaignParameters(campaign), + ) + } + oracleReplayTest(vitestIt).each([fixedGeneratedCampaign])( + `rejects generated BigInts immediately outside the signed range in the $name campaign`, + rejectGeneratedBigInt, + ) + oracleReplayTest(vitestIt, BIGINT_REJECTION_PROPERTY).each([ + randomGeneratedCampaign(BIGINT_REJECTION_PROPERTY), + ])( + `rejects generated BigInts immediately outside the signed range in the $name campaign`, + rejectGeneratedBigInt, ) it(`does not confuse an alias-qualified field ref with a nested JSON path`, async () => { @@ -3252,33 +3230,35 @@ describe(`SQLite expression-index oracle`, () => { ) }) - const generatedExpressionIndexTest = - requestedReplay === undefined || - requestedReplayProperty === `expression-index` - ? vitestIt - : vitestIt.skip - - generatedExpressionIndexTest.each(generatedPropertyCampaigns)( + async function reachGeneratedExpressionIndex( + campaign: GeneratedPropertyCampaign, + ) { + const details = await runGeneratedScenarioProperty(campaign) + const parameters = generatedCampaignParameters(campaign) + + expect(details.numRuns).toBe(campaign.parameters.numRuns) + expect(`seed` in parameters).toBe(campaign.parameters.seed !== undefined) + expect(`path` in parameters).toBe(campaign.parameters.path !== undefined) + if (campaign.parameters.seed !== undefined) { + expect(details.seed).toBe(campaign.parameters.seed) + } + if (campaign.parameters.path !== undefined) { + expect(details.runConfiguration.path).toBe(campaign.parameters.path) + } + expect(details.seed).toEqual(expect.any(Number)) + } + oracleReplayTest(vitestIt).each([fixedGeneratedCampaign])( `reaches every declared generated expression-index scenario in the $name campaign`, - async (campaign) => { - const details = await runGeneratedScenarioProperty(campaign) - const parameters = generatedCampaignParameters(campaign) - - expect(details.numRuns).toBe(campaign.numRuns) - expect(`seed` in parameters).toBe(campaign.seed !== undefined) - expect(`path` in parameters).toBe(campaign.path !== undefined) - if (campaign.seed !== undefined) { - expect(details.seed).toBe(campaign.seed) - } - if (campaign.path !== undefined) { - expect(details.runConfiguration.path).toBe(campaign.path) - } - expect(details.seed).toEqual(expect.any(Number)) - }, + reachGeneratedExpressionIndex, + ) + oracleReplayTest(vitestIt, EXPRESSION_INDEX_PROPERTY).each([ + randomGeneratedCampaign(EXPRESSION_INDEX_PROPERTY), + ])( + `reaches every declared generated expression-index scenario in the $name campaign`, + reachGeneratedExpressionIndex, ) - const generatedControlTest = - requestedReplay === undefined ? vitestIt : vitestIt.skip + const generatedControlTest = oracleReplayTest(vitestIt) generatedControlTest.each(GENERATED_SCENARIO_KINDS)( `rejects the generated grammar when the %s axis is ablated`, @@ -3349,22 +3329,21 @@ describe(`SQLite expression-index oracle`, () => { expect(details.error ?? ``, evidence).not.toContain(`did not reach`) expect(details.error, evidence).toContain(`"kind": "${kind}"`) - const replayPath = details.counterexamplePath - if (replayPath === null) { + if (details.counterexamplePath === null) { throw new Error(`${kind} wrong-answer control did not shrink to a path`) } const replay = await fc.check( generatedScenarioProperty({ faultKind: kind }), { seed: details.seed, - path: replayPath, + path: details.counterexamplePath, numRuns: 1, verbose: 2, }, ) const replayEvidence = JSON.stringify({ seed: replay.seed, - requestedPath: replayPath, + requestedPath: details.counterexamplePath, replayPath: replay.counterexamplePath, error: replay.error, }) @@ -3378,7 +3357,7 @@ describe(`SQLite expression-index oracle`, () => { // Native values add a bounded rank/text dimension to this SQL/plan owner. // The common model is independent; this driver observes real SQL before cleanup. describe(`Temporal expression-index refinement`, () => { - const it = requestedReplay === undefined ? vitestIt : vitestIt.skip + const it = oracleReplayTest(vitestIt) it.each(temporalFamilies)( `uses native keys and identity for $kind / $name`, async (family) => { diff --git a/packages/offline-transactions/tests/KeyScheduler-oracle.property.test.ts b/packages/offline-transactions/tests/KeyScheduler-oracle.property.test.ts index 5fd1e581af..2507650f6b 100644 --- a/packages/offline-transactions/tests/KeyScheduler-oracle.property.test.ts +++ b/packages/offline-transactions/tests/KeyScheduler-oracle.property.test.ts @@ -1,9 +1,15 @@ import { fc } from '@fast-check/vitest' import { afterEach, describe, expect, it, vi } from 'vitest' import { KeyScheduler } from '../src/executor/KeyScheduler' -import { readOfflineOracleConfig } from './oracle-config' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' import type { OfflineTransaction } from '../src/types' +const FIFO_LEDGER_PROPERTY = `offline.key-scheduler-fifo-ledger` + type Command = | { type: `schedule` @@ -57,14 +63,7 @@ type Snapshot = { const BASE_TIME = Date.parse(`2026-01-01T00:00:00.000Z`) const FIXED_SEED = 1815 -const { - runs: RUNS, - seed: SEED, - path: PATH, -} = readOfflineOracleConfig({ - prefix: `TANSTACK_DB_OFFLINE_ORACLE`, - defaultRuns: 150, -}) +const RUNS = oracleRuns(150) /** * # Which offline transaction may run next? @@ -103,8 +102,9 @@ const { * histories cross retry-deadline order. Five observation mutants prove * the comparison rejects bypass, double issue, stale payload, stale clear, and * failed selective retirement. The generated property runs at fixed seed 1815 - * and at a random seed with the same budget. Set TANSTACK_DB_OFFLINE_ORACLE_SEED - * and TANSTACK_DB_OFFLINE_ORACLE_PATH to replay a failure directly. + * and at a random seed with the same budget. Set TANSTACK_DB_ORACLE_SEED, + * TANSTACK_DB_ORACLE_PATH, and TANSTACK_DB_ORACLE_PROPERTY to replay a failure + * directly. * * Limits: persistence, caller promise settlement, leadership, and real timers * have separate owners. The model does not promise fairness beyond FIFO order. @@ -517,32 +517,32 @@ function runHistory( describe(`KeyScheduler generated lifecycle`, () => { afterEach(() => vi.useRealTimers()) - function runGeneratedHistory(seed?: number, path?: string): void { - fc.assert( - fc.property(legalHistory, (history) => { - runHistory(history) - }), - { - ...(seed === undefined ? {} : { seed }), - numRuns: RUNS, - ...(path === undefined ? {} : { path }), - }, - ) - } - - if (SEED === undefined) { - it(`matches the FIFO retry ledger at the fixed seed`, () => { - runGeneratedHistory(FIXED_SEED) - }) + // The fixed campaign stands aside during any path replay. The random + // campaign is the replay of FIFO_LEDGER_PROPERTY: it reseeds under a seed + // alone and runs the recorded path under a path replay of that property. + oracleReplayTest(it)( + `matches the FIFO retry ledger at the fixed seed`, + () => { + fc.assert( + fc.property(legalHistory, (history) => { + runHistory(history) + }), + { seed: FIXED_SEED, numRuns: RUNS }, + ) + }, + ) - it(`matches the FIFO retry ledger at a random seed`, () => { - runGeneratedHistory() - }) - } else { - it(`replays the FIFO retry ledger at the requested seed and path`, () => { - runGeneratedHistory(SEED, PATH) - }) - } + oracleReplayTest(it, FIFO_LEDGER_PROPERTY)( + `matches the FIFO retry ledger at a random or replayed seed`, + () => { + fc.assert( + fc.property(legalHistory, (history) => { + runHistory(history) + }), + oraclePropertyOptions(150, FIFO_LEDGER_PROPERTY), + ) + }, + ) it(`reconstructs active removal with a waiting sibling`, () => { const token = (selector: number, slot = 0): CommandToken => ({ diff --git a/packages/offline-transactions/tests/fifo-retry-oracle.property.test.ts b/packages/offline-transactions/tests/fifo-retry-oracle.property.test.ts index 982f2b8173..c9b8b57cc6 100644 --- a/packages/offline-transactions/tests/fifo-retry-oracle.property.test.ts +++ b/packages/offline-transactions/tests/fifo-retry-oracle.property.test.ts @@ -3,8 +3,9 @@ import { expect, it, vi } from 'vitest' import { KeyScheduler } from '../src/executor/KeyScheduler' import { TransactionExecutor } from '../src/executor/TransactionExecutor' import { OutboxManager } from '../src/outbox/OutboxManager' +import { oracleReplayTest } from '../../db/tests/oracle-config' import { FakeStorageAdapter } from './harness' -import { readOfflineOracleConfig } from './oracle-config' +import { oracleLaneOptions, readOfflineOracleConfig } from './oracle-config' import type { OfflineTransaction } from '../src/types' /** @@ -48,18 +49,12 @@ type ExpectedState = { outboxCount: number } -const { - runs: replayRuns, - seed: replaySeed, - path: replayPath, -} = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, +const FIFO_RETRY_PROPERTY = `offline.fifo-retry` +const fifoRetryOracle = readOfflineOracleConfig({ + property: FIFO_RETRY_PROPERTY, defaultRuns: 30, }) -const campaignSeeds = - replayPath === undefined ? [20260916, undefined] : [undefined] - function expectedStateAt( elapsed: number, delay: number, @@ -77,149 +72,144 @@ function expectState(actual: ExpectedState, expected: ExpectedState): void { expect(actual.outboxCount).toBe(expected.outboxCount) } -it.each(campaignSeeds)( - `waits for the FIFO head and executes eligible work at its deadline (seed %s)`, - async (fixedSeed) => { - const seed = fixedSeed ?? replaySeed - await fc.assert( - fc.asyncProperty( - fc.integer({ min: 2, max: 5 }), - fc.integer({ min: 2, max: 10000 }), - async (count, delay) => { - let clearExecutor: (() => void) | undefined - let primaryFailure: unknown - let failed = false - try { - vi.useFakeTimers() - const scheduler = new KeyScheduler() - const outbox = new OutboxManager(new FakeStorageAdapter(), {}) - const calls: Array = [] +// The fixed lane stands aside during any path replay. The random lane runs in +// ordinary and seed-only runs, and it is the replay of FIFO_RETRY_PROPERTY. +async function checkFifoRetry(seed: number | undefined): Promise { + await fc.assert( + fc.asyncProperty( + fc.integer({ min: 2, max: 5 }), + fc.integer({ min: 2, max: 10000 }), + async (count, delay) => { + let clearExecutor: (() => void) | undefined + let primaryFailure: unknown + let failed = false + try { + vi.useFakeTimers() + const scheduler = new KeyScheduler() + const outbox = new OutboxManager(new FakeStorageAdapter(), {}) + const calls: Array = [] - // Record each transaction when the production executor runs it. - const executor = new TransactionExecutor( - scheduler, - outbox, - { - collections: {}, - mutationFns: { - persist: ({ transaction }) => { - calls.push(transaction.id) - return Promise.resolve() - }, + // Record each transaction when the production executor runs it. + const executor = new TransactionExecutor( + scheduler, + outbox, + { + collections: {}, + mutationFns: { + persist: ({ transaction }) => { + calls.push(transaction.id) + return Promise.resolve() }, }, - { - isOfflineEnabled: true, - resolveTransaction: () => {}, - rejectTransaction: () => {}, - registerRestorationTransaction: () => {}, - isOnline: () => true, - }, - ) - clearExecutor = () => executor.clear() - const transactions: Array = Array.from( - { length: count }, - (_unused, index) => ({ - id: `tx-${index}`, - mutationFnName: `persist`, - mutations: [], - keys: [], - idempotencyKey: `key-${index}`, - createdAt: new Date(index), - nextAttemptAt: Date.now() + (index === 0 ? delay : 0), - retryCount: 0, - version: 1, - }), - ) - const transactionIds = transactions.map( - (transaction) => transaction.id, - ) + }, + { + isOfflineEnabled: true, + resolveTransaction: () => {}, + rejectTransaction: () => {}, + registerRestorationTransaction: () => {}, + isOnline: () => true, + }, + ) + clearExecutor = () => executor.clear() + const transactions: Array = Array.from( + { length: count }, + (_unused, index) => ({ + id: `tx-${index}`, + mutationFnName: `persist`, + mutations: [], + keys: [], + idempotencyKey: `key-${index}`, + createdAt: new Date(index), + nextAttemptAt: Date.now() + (index === 0 ? delay : 0), + retryCount: 0, + version: 1, + }), + ) + const transactionIds = transactions.map( + (transaction) => transaction.id, + ) - for (const transaction of transactions) { - await outbox.add(transaction) - scheduler.schedule(transaction) - } - await executor.executeAll() + for (const transaction of transactions) { + await outbox.add(transaction) + scheduler.schedule(transaction) + } + await executor.executeAll() - // A ready tail waits for the delayed head. The virtual-clock cuts - // distinguish early execution from failure to wake when eligible. - const beforeDeadline = expectedStateAt(0, delay, transactionIds) - const observedBeforeDeadline = { - calls: [...calls], - outboxCount: await outbox.count(), - } - expectState(observedBeforeDeadline, beforeDeadline) - // A scheduler that lets a ready tail bypass the delayed head - // reaches this checkpoint with an extra call. Calibrate the same - // comparison against that plausible wrong observation. - expect(() => - expectState( - { ...observedBeforeDeadline, calls: [transactionIds[1]!] }, - beforeDeadline, - ), - ).toThrow() - await vi.advanceTimersByTimeAsync(delay - 1) - const nearDeadline = expectedStateAt( - delay - 1, - delay, - transactionIds, - ) + // A ready tail waits for the delayed head. The virtual-clock cuts + // distinguish early execution from failure to wake when eligible. + const beforeDeadline = expectedStateAt(0, delay, transactionIds) + const observedBeforeDeadline = { + calls: [...calls], + outboxCount: await outbox.count(), + } + expectState(observedBeforeDeadline, beforeDeadline) + // A scheduler that lets a ready tail bypass the delayed head + // reaches this checkpoint with an extra call. Calibrate the same + // comparison against that plausible wrong observation. + expect(() => expectState( - { calls: [...calls], outboxCount: await outbox.count() }, - nearDeadline, - ) - await vi.advanceTimersByTimeAsync(1) - const atDeadline = expectedStateAt(delay, delay, transactionIds) - const observedAtDeadline = { - calls: [...calls], - outboxCount: await outbox.count(), - } - expectState(observedAtDeadline, atDeadline) - // A scheduler that never wakes at eligibility leaves the whole - // queue untouched at this controlled-clock checkpoint. - expect(() => - expectState( - { calls: [], outboxCount: transactionIds.length }, - atDeadline, - ), - ).toThrow() - } catch (error) { - primaryFailure = error - failed = true + { ...observedBeforeDeadline, calls: [transactionIds[1]!] }, + beforeDeadline, + ), + ).toThrow() + await vi.advanceTimersByTimeAsync(delay - 1) + const nearDeadline = expectedStateAt(delay - 1, delay, transactionIds) + expectState( + { calls: [...calls], outboxCount: await outbox.count() }, + nearDeadline, + ) + await vi.advanceTimersByTimeAsync(1) + const atDeadline = expectedStateAt(delay, delay, transactionIds) + const observedAtDeadline = { + calls: [...calls], + outboxCount: await outbox.count(), } + expectState(observedAtDeadline, atDeadline) + // A scheduler that never wakes at eligibility leaves the whole + // queue untouched at this controlled-clock checkpoint. + expect(() => + expectState( + { calls: [], outboxCount: transactionIds.length }, + atDeadline, + ), + ).toThrow() + } catch (error) { + primaryFailure = error + failed = true + } - const cleanupFailures: Array = [] - const cleanups: Array<[string, (() => void) | undefined]> = [ - [`executor.clear`, clearExecutor], - [`real timer restore`, () => vi.useRealTimers()], - ] - for (const [name, cleanup] of cleanups) { - if (!cleanup) continue - try { - cleanup() - } catch (error) { - cleanupFailures.push( - new Error(`${name} failed`, { cause: error }), - ) - } - } - if (cleanupFailures.length > 0) { - throw new AggregateError( - cleanupFailures, - `FIFO retry oracle cleanup failed`, - { cause: failed ? primaryFailure : undefined }, - ) + const cleanupFailures: Array = [] + const cleanups: Array<[string, (() => void) | undefined]> = [ + [`executor.clear`, clearExecutor], + [`real timer restore`, () => vi.useRealTimers()], + ] + for (const [name, cleanup] of cleanups) { + if (!cleanup) continue + try { + cleanup() + } catch (error) { + cleanupFailures.push(new Error(`${name} failed`, { cause: error })) } - if (failed) throw primaryFailure - }, - ), - { - numRuns: replayRuns, - ...(seed === undefined ? {} : { seed }), - ...(fixedSeed === undefined && replayPath !== undefined - ? { path: replayPath } - : {}), + } + if (cleanupFailures.length > 0) { + throw new AggregateError( + cleanupFailures, + `FIFO retry oracle cleanup failed`, + { cause: failed ? primaryFailure : undefined }, + ) + } + if (failed) throw primaryFailure }, - ) - }, + ), + oracleLaneOptions(fifoRetryOracle, seed), + ) +} + +oracleReplayTest(it).each([20260916])( + `waits for the FIFO head and executes eligible work at its deadline (seed %s)`, + checkFifoRetry, +) +oracleReplayTest(it, FIFO_RETRY_PROPERTY).each([undefined])( + `waits for the FIFO head and executes eligible work at its deadline (seed %s)`, + checkFifoRetry, ) diff --git a/packages/offline-transactions/tests/interruption-settlement-oracle.property.test.ts b/packages/offline-transactions/tests/interruption-settlement-oracle.property.test.ts index b4ffe587e0..6322911209 100644 --- a/packages/offline-transactions/tests/interruption-settlement-oracle.property.test.ts +++ b/packages/offline-transactions/tests/interruption-settlement-oracle.property.test.ts @@ -1,13 +1,16 @@ import fc from 'fast-check' import { expect, it, vi } from 'vitest' import { NonRetriableError } from '../src/types' +import { + oraclePropertyOptions, + oracleReplayTest, +} from '../../db/tests/oracle-config' import { FakeStorageAdapter, createTestOfflineEnvironment } from './harness' import { atOracleCheckpoint, captureUnhandledRejections, cleanupOfflineOracle, } from './oracle-lifecycle' -import { readOfflineOracleConfig } from './oracle-config' import type { OnlineDetector } from '../src/types' import type { TestItem } from './harness' @@ -79,10 +82,9 @@ type History = { turn: number } -const config = readOfflineOracleConfig({ - prefix: `TANSTACK_DB_OFFLINE_ORACLE`, - defaultRuns: 60, -}) +// A path replay of this property runs only the generated campaign, from its +// recorded seed and path. +const interruptionProperty = `offline.interruption-settlement` const history: fc.Arbitrary = fc.record({ outcomes: fc.array(fc.constantFrom(`fulfil`, `reject`), { @@ -410,24 +412,24 @@ it.each(fixedLandings)( 60_000, ) -it(`keeps settlement and ownership for an interruption at any microtask turn`, async () => { - // A permanent provider failure logs once by design; this law ignores logs. - const warn = vi.spyOn(console, `warn`).mockImplementation(() => {}) - try { - await fc.assert( - fc.asyncProperty(history, async (generated) => { - // Settlement and ownership must hold for every generated history, - // regardless of which landing it reaches. Coverage is verified by the - // fixed tests above; this campaign proves the core law. - await checkInterruption(generated) - }), - { - numRuns: config.runs, - ...(config.seed === undefined ? {} : { seed: config.seed }), - ...(config.path === undefined ? {} : { path: config.path }), - }, - ) - } finally { - warn.mockRestore() - } -}, 120_000) +oracleReplayTest(it, interruptionProperty)( + `keeps settlement and ownership for an interruption at any microtask turn`, + async () => { + // A permanent provider failure logs once by design; this law ignores logs. + const warn = vi.spyOn(console, `warn`).mockImplementation(() => {}) + try { + await fc.assert( + fc.asyncProperty(history, async (generated) => { + // Settlement and ownership must hold for every generated history, + // regardless of which landing it reaches. Coverage is verified by the + // fixed tests above; this campaign proves the core law. + await checkInterruption(generated) + }), + oraclePropertyOptions(60, interruptionProperty), + ) + } finally { + warn.mockRestore() + } + }, + 120_000, +) diff --git a/packages/offline-transactions/tests/leadership-replay-oracle.property.test.ts b/packages/offline-transactions/tests/leadership-replay-oracle.property.test.ts index 6fb6473459..50195ae882 100644 --- a/packages/offline-transactions/tests/leadership-replay-oracle.property.test.ts +++ b/packages/offline-transactions/tests/leadership-replay-oracle.property.test.ts @@ -6,9 +6,10 @@ import { OfflineExecutor } from '../src/OfflineExecutor' import { TransactionExecutor } from '../src/executor/TransactionExecutor' import { KeyScheduler } from '../src/executor/KeyScheduler' import { NonRetriableError } from '../src/types' +import { oracleReplayTest } from '../../db/tests/oracle-config' import { FakeStorageAdapter, createTestOfflineEnvironment } from './harness' import { atOracleCheckpoint, cleanupOfflineOracle } from './oracle-lifecycle' -import { readOfflineOracleConfig } from './oracle-config' +import { oracleLaneOptions, readOfflineOracleConfig } from './oracle-config' import type { OfflineTransaction, OnlineDetector } from '../src/types' import type { TestItem } from './harness' @@ -120,12 +121,14 @@ function foldReplayLedger(events: ReadonlyArray): { } } -// These campaigns share the package replay variables. Target this file and one -// test name when replaying a shrink path, because paths are property-specific. -// For example, from the repository root: -// OFFLINE_ORACLE_SEED=20260923 OFFLINE_ORACLE_PATH=0:0 \ +// Each campaign below is its own replay property, and a path belongs to one of +// them. To replay a shrink path, set TANSTACK_DB_ORACLE_SEED, +// TANSTACK_DB_ORACLE_PATH, and TANSTACK_DB_ORACLE_PROPERTY to that campaign's +// `offline.leadership.*` property. For example, from the repository root: +// TANSTACK_DB_ORACLE_SEED=20260923 TANSTACK_DB_ORACLE_PATH=0:0 \ +// TANSTACK_DB_ORACLE_PROPERTY=offline.leadership.stale-read \ // pnpm --dir packages/offline-transactions exec vitest run \ -// tests/leadership-replay-oracle.property.test.ts -t 'filters every durably removed transaction' +// tests/leadership-replay-oracle.property.test.ts // // Grammar controls: each fixed lane enumerates the named causal boundary, and // the seedless lane samples adjacent values with the same property/checkpoint. @@ -145,53 +148,37 @@ function foldReplayLedger(events: ReadonlyArray): { // Delivery before capture and acknowledgment before provider completion are // causally invalid histories, so neither is generated. Zero stored work and // all-removed stale reads lie outside these stored-work/retained-peer claims. +const SERIAL_WORK_PROPERTY = `offline.leadership.serial-work` +const REPORT_PROPERTY = `offline.leadership.report` +const DELAYED_READ_PROPERTY = `offline.leadership.delayed-read` +const STARTUP_PROPERTY = `offline.leadership.startup` +const STALE_READ_PROPERTY = `offline.leadership.stale-read` +const STORAGE_FAILURE_PROPERTY = `offline.leadership.storage-failure` const serialWorkOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: SERIAL_WORK_PROPERTY, defaultRuns: 20, }) const leadershipReportOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: REPORT_PROPERTY, defaultRuns: 40, }) const delayedReadOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: DELAYED_READ_PROPERTY, defaultRuns: 30, }) const startupOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: STARTUP_PROPERTY, defaultRuns: 10, }) const staleReadOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: STALE_READ_PROPERTY, defaultRuns: 20, }) const storageFailureOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: STORAGE_FAILURE_PROPERTY, defaultRuns: 10, }) -type OracleConfig = ReturnType - -// A normal run has paired fixed and seedless campaigns. A requested seed/path -// selects one direct replay; the caller targets its test name as well. -function oracleSeeds( - fixedSeed: number, - config: OracleConfig, -): Array { - return config.seed === undefined ? [fixedSeed, undefined] : [config.seed] -} - -function oracleOptions( - config: OracleConfig, - seed: number | undefined, -): { numRuns: number; seed?: number; path?: string } { - return { - numRuns: config.runs, - ...(seed === undefined ? {} : { seed }), - ...(config.path === undefined ? {} : { path: config.path }), - } -} - it(`removes an earlier deletion-pending row before replaying an equal-time peer`, async () => { // The README promises creation-order FIFO and deletion before success. This // fixed offline executor restart history uses storage insertion order to @@ -1180,33 +1167,169 @@ it(`keeps permanently failed work owned until durable deletion settles`, async ( } }) -it.each( - [`construction`, `leadership`, `outbox read`, `retry hook`].flatMap( - (boundary) => - oracleSeeds(20260918, startupOracle).map((seed) => ({ boundary, seed })), - ), -)( - `does not revive a disposed executor after $boundary (seed $seed)`, - async ({ boundary, seed }) => { - await fc.assert( - fc.asyncProperty(fc.integer({ min: 1, max: 4 }), async (count) => { - const pending = gate() - const entered = gate() - let reading = false - class Storage extends FakeStorageAdapter { - override async keys() { - if (reading && boundary === `outbox read`) { - entered.resolve() - await pending.promise - } - return super.keys() +const startupCases = (seed: number | undefined) => + [`construction`, `leadership`, `outbox read`, `retry hook`].map( + (boundary) => ({ + boundary, + seed, + }), + ) + +// The fixed lane stands aside during any path replay. The random lane runs in +// ordinary and seed-only runs, and it is the replay of STARTUP_PROPERTY. +async function checkStartup({ + boundary, + seed, +}: ReturnType[number]): Promise { + await fc.assert( + fc.asyncProperty(fc.integer({ min: 1, max: 4 }), async (count) => { + const pending = gate() + const entered = gate() + let reading = false + class Storage extends FakeStorageAdapter { + override async keys() { + if (reading && boundary === `outbox read`) { + entered.resolve() + await pending.promise } + return super.keys() } - const storage = new Storage() - const outbox = new OutboxManager(storage, {}) - for (let index = 0; index < count; index++) - await outbox.add({ - id: `startup-${index}`, + } + const storage = new Storage() + const outbox = new OutboxManager(storage, {}) + for (let index = 0; index < count; index++) + await outbox.add({ + id: `startup-${index}`, + mutationFnName: `syncData`, + mutations: [], + keys: [], + idempotencyKey: `key-${index}`, + createdAt: new Date(index), + retryCount: 0, + nextAttemptAt: 0, + version: 1, + }) + const snapshot = storage.snapshot() + const callbacks = new Set<(leader: boolean) => void>() + let leader = false + let requests = 0 + const restore = vi.spyOn( + OfflineExecutor.prototype, + `registerRestorationTransaction`, + ) + reading = true + const env = createTestOfflineEnvironment({ + storage, + config: { + beforeRetry: (transactions) => { + if (boundary === `retry hook`) env.executor.dispose() + return transactions + }, + leaderElection: { + requestLeadership: async () => { + requests++ + if (boundary === `leadership`) { + entered.resolve() + await pending.promise + } + leader = true + return true + }, + releaseLeadership: () => { + leader = false + }, + isLeader: () => leader, + onLeadershipChange: (callback) => { + callbacks.add(callback) + return () => { + callbacks.delete(callback) + } + }, + }, + }, + }) + let hasPrimaryFailure = false + try { + if (boundary === `leadership` || boundary === `outbox read`) + await atOracleCheckpoint( + entered.promise, + `startup reached ${boundary}`, + ) + if (boundary !== `retry hook`) env.executor.dispose() + pending.resolve() + await atOracleCheckpoint( + env.executor.waitForInit(), + `disposed startup completed`, + ) + await turn() + expect(env.mutationCalls).toEqual([]) + expect(env.executor.isOfflineEnabled).toBe(false) + expect(callbacks.size).toBe(0) + expect(env.executor.getPendingCount()).toBe(0) + expect(leader).toBe(false) + expect(restore).not.toHaveBeenCalled() + expect(storage.snapshot()).toEqual(snapshot) + if (boundary === `construction`) expect(requests).toBe(0) + } catch (error) { + hasPrimaryFailure = true + throw error + } finally { + pending.resolve() + restore.mockRestore() + await cleanupOfflineOracle( + [ + () => env.executor.dispose(), + () => env.executor.waitForInit(), + () => env.collection.cleanup(), + ], + hasPrimaryFailure, + ) + } + }), + oracleLaneOptions(startupOracle, seed), + ) +} + +oracleReplayTest(it).each(startupCases(20260918))( + `does not revive a disposed executor after $boundary (seed $seed)`, + checkStartup, +) +oracleReplayTest(it, STARTUP_PROPERTY).each(startupCases(undefined))( + `does not revive a disposed executor after $boundary (seed $seed)`, + checkStartup, +) + +const serialWorkCases = (seed: number | undefined) => + [`loss`, `dispose`].flatMap((stop) => + [`provider`, `acknowledgment`, `retry`].map((boundary) => ({ + stop, + boundary, + seed, + })), + ) + +// The fixed lane stands aside during any path replay. The random lane runs in +// ordinary and seed-only runs, and it is the replay of SERIAL_WORK_PROPERTY. +async function checkSerialWork({ + stop, + boundary, + seed, +}: ReturnType[number]): Promise { + await fc.assert( + fc.asyncProperty( + fc.integer({ min: 2, max: 5 }), + fc.string({ maxLength: 20 }), + async (count, payload) => { + vi.useFakeTimers() + const executions = vi.spyOn(TransactionExecutor.prototype, `executeAll`) + const entered = gate() + const provider = gate() + const acknowledgment = gate() + const acknowledging = gate() + const transactions: Array = Array.from( + { length: count }, + (_, index) => ({ + id: `stored-${index}`, mutationFnName: `syncData`, mutations: [], keys: [], @@ -1215,37 +1338,230 @@ it.each( retryCount: 0, nextAttemptAt: 0, version: 1, - }) - const snapshot = storage.snapshot() - const callbacks = new Set<(leader: boolean) => void>() - let leader = false - let requests = 0 - const restore = vi.spyOn( - OfflineExecutor.prototype, - `registerRestorationTransaction`, + metadata: { payload }, + }), ) - reading = true + class Storage extends FakeStorageAdapter { + override async delete(key: string) { + if (boundary === `acknowledgment`) { + acknowledging.resolve() + await acknowledgment.promise + } + return super.delete(key) + } + } + const storage = new Storage() + const outbox = new OutboxManager(storage, {}) + for (const transaction of transactions) await outbox.add(transaction) + const calls: Array<{ id: string; key: string; metadata: unknown }> = [] const env = createTestOfflineEnvironment({ storage, - config: { - beforeRetry: (transactions) => { - if (boundary === `retry hook`) env.executor.dispose() - return transactions + config: { jitter: false }, + mutationFn: async ({ transaction, idempotencyKey, attempt }) => { + calls.push({ + id: transaction.id, + key: idempotencyKey, + metadata: transaction.metadata, + }) + entered.resolve() + await provider.promise + if ( + boundary === `retry` && + attempt === 1 && + transaction.id === `stored-0` + ) + throw new Error(`transient provider failure`) + }, + }) + const completions = transactions.map(({ id }) => { + const observed: { state: `pending` | `fulfilled` | `rejected` } = { + state: `pending`, + } + void env.executor.waitForTransactionCompletion(id).then( + () => { + observed.state = `fulfilled` }, + () => { + observed.state = `rejected` + }, + ) + return observed + }) + let replacement: + ReturnType | undefined + const expectedCall = ({ + id, + idempotencyKey, + metadata, + }: OfflineTransaction) => ({ id, key: idempotencyKey, metadata }) + let hasPrimaryFailure = false + try { + await env.executor.waitForInit() + await entered.promise + expect(calls).toEqual(transactions.slice(0, 1).map(expectedCall)) + if (boundary !== `provider`) { + provider.resolve() + if (boundary === `acknowledgment`) await acknowledging.promise + await vi.advanceTimersByTimeAsync(0) + } + if (stop === `loss`) env.leader.setLeader(false) + else env.executor.dispose() + provider.resolve() + acknowledgment.resolve() + const executionCount = executions.mock.calls.length + await vi.advanceTimersByTimeAsync(2000) + expect(executions).toHaveBeenCalledTimes(executionCount) + env.executor.getOnlineDetector().notifyOnline() + await vi.advanceTimersByTimeAsync(0) + + // Only the issued call may finish. Queue length, timers or final rows + // alone cannot tell whether a former owner made extra remote requests. + expect(calls).toEqual(transactions.slice(0, 1).map(expectedCall)) + expect(env.executor.isOfflineEnabled).toBe(false) + expect(completions.map(({ state }) => state)).toEqual( + transactions.map((_, index) => + boundary !== `retry` && index === 0 ? `fulfilled` : `pending`, + ), + ) + const retained = + boundary === `retry` ? transactions : transactions.slice(1) + expect( + (await outbox.getAll()).map(({ id, idempotencyKey, metadata }) => ({ + id, + idempotencyKey, + metadata, + })), + ).toEqual( + retained.map(({ id, idempotencyKey, metadata }) => ({ + id, + idempotencyKey, + metadata, + })), + ) + + if (stop === `loss`) env.leader.setLeader(true) + else { + replacement = createTestOfflineEnvironment({ + storage, + mutationFn: ({ transaction, idempotencyKey }) => { + calls.push({ + id: transaction.id, + key: idempotencyKey, + metadata: transaction.metadata, + }) + return Promise.resolve() + }, + }) + await replacement.executor.waitForInit() + // Disposed executors cannot be revived by a later elector report. + env.leader.setLeader(true) + } + await vi.advanceTimersByTimeAsync(0) + expect(calls).toEqual([ + ...transactions.slice(0, 1).map(expectedCall), + ...retained.map(expectedCall), + ]) + expect(await outbox.getAll()).toEqual([]) + if (stop === `loss`) + expect( + completions.every(({ state }) => state === `fulfilled`), + ).toBe(true) + } catch (error) { + hasPrimaryFailure = true + throw error + } finally { + provider.resolve() + acknowledgment.resolve() + executions.mockRestore() + vi.useRealTimers() + await cleanupOfflineOracle( + [ + () => env.executor.dispose(), + () => replacement?.executor.dispose(), + () => env.collection.cleanup(), + () => replacement?.collection.cleanup(), + ], + hasPrimaryFailure, + ) + } + }, + ), + oracleLaneOptions(serialWorkOracle, seed), + ) +} + +oracleReplayTest(it).each(serialWorkCases(20260917))( + `retains serial work after $stop at $boundary (seed $seed)`, + checkSerialWork, +) +oracleReplayTest(it, SERIAL_WORK_PROPERTY).each(serialWorkCases(undefined))( + `retains serial work after $stop at $boundary (seed $seed)`, + checkSerialWork, +) + +// Repeated reports and reacquisition must not readmit work already in flight. +// Use the real serializer and executor: an invalid storage key would let a +// zero-execution trace falsely satisfy an at-most-once assertion. +const reportCases = (seed: number | undefined) => + [`provider`, `acknowledgment`].map((boundary) => ({ boundary, seed })) + +// The fixed lane stands aside during any path replay. The random lane runs in +// ordinary and seed-only runs, and it is the replay of REPORT_PROPERTY. +async function checkReport({ + boundary, + seed, +}: ReturnType[number]): Promise { + await fc.assert( + fc.asyncProperty( + fc.record({ + pendingReports: fc.integer({ min: 0, max: 5 }), + settledReports: fc.integer({ min: 0, max: 5 }), + separateTurns: fc.boolean(), + regainLeadership: fc.boolean(), + value: fc.string({ maxLength: 20 }), + }), + async ({ + pendingReports, + settledReports, + separateTurns, + regainLeadership, + value, + }) => { + const request = gate() + const entered = gate() + const release = gate() + const acknowledging = gate() + const acknowledgment = gate() + class Storage extends FakeStorageAdapter { + override async delete(key: string) { + acknowledging.resolve() + await acknowledgment.promise + return super.delete(key) + } + } + const storage = new Storage() + const callbacks = new Set<(leader: boolean) => void>() + const calls: Array<{ + id: string + key: string + row: Record + }> = [] + const row = { + id: `row`, + value, + completed: false, + updatedAt: new Date(1700000000123), + } + const env = createTestOfflineEnvironment({ + storage, + config: { leaderElection: { requestLeadership: async () => { - requests++ - if (boundary === `leadership`) { - entered.resolve() - await pending.promise - } - leader = true + await request.promise return true }, - releaseLeadership: () => { - leader = false - }, - isLeader: () => leader, + releaseLeadership: () => {}, + isLeader: () => true, onLeadershipChange: (callback) => { callbacks.add(callback) return () => { @@ -1254,439 +1570,136 @@ it.each( }, }, }, + mutationFn: async (params) => { + const mutations = params.transaction.mutations + calls.push({ + id: params.transaction.id, + key: params.idempotencyKey, + row: structuredClone(mutations[0].modified), + }) + entered.resolve() + await release.promise + env.applyMutations(mutations) + }, }) + const completion = env.executor + .waitForTransactionCompletion(`valid`) + .then( + () => `fulfilled`, + (error: unknown) => `rejected:${String(error)}`, + ) + const transaction = createTransaction({ + autoCommit: false, + mutationFn: async () => {}, + }) + const rolledBack = transaction.isPersisted.promise.catch( + () => undefined, + ) + const expected = [{ id: `valid`, key: `once`, row }] + const report = async (count: number) => { + expect(callbacks.size).toBe(1) + for (let i = 0; i < count; i++) { + if (regainLeadership) + for (const callback of callbacks) callback(false) + for (const callback of callbacks) callback(true) + if (separateTurns) await turn() + } + await turn() + } let hasPrimaryFailure = false try { - if (boundary === `leadership` || boundary === `outbox read`) - await atOracleCheckpoint( - entered.promise, - `startup reached ${boundary}`, - ) - if (boundary !== `retry hook`) env.executor.dispose() - pending.resolve() + transaction.mutate(() => env.collection.insert(row)) + await new OutboxManager(storage, { + [env.collection.id]: env.collection, + }).add({ + id: `valid`, + mutationFnName: env.mutationFnName, + mutations: transaction.mutations, + keys: transaction.mutations.map((mutation) => mutation.globalKey), + idempotencyKey: `once`, + createdAt: new Date(1700000000000), + retryCount: 0, + nextAttemptAt: 0, + metadata: {}, + version: 1, + }) + transaction.rollback() + await rolledBack + request.resolve() + await atOracleCheckpoint(entered.promise, `replay provider entered`) await atOracleCheckpoint( env.executor.waitForInit(), - `disposed startup completed`, + `replay initialized`, ) - await turn() - expect(env.mutationCalls).toEqual([]) - expect(env.executor.isOfflineEnabled).toBe(false) - expect(callbacks.size).toBe(0) - expect(env.executor.getPendingCount()).toBe(0) - expect(leader).toBe(false) - expect(restore).not.toHaveBeenCalled() - expect(storage.snapshot()).toEqual(snapshot) - if (boundary === `construction`) expect(requests).toBe(0) + expect(calls).toEqual(expected) + if (boundary === `acknowledgment`) { + release.resolve() + await atOracleCheckpoint( + acknowledging.promise, + `provider completed; durable deletion held`, + ) + } + await report(pendingReports) + expect(calls).toEqual(expected) + release.resolve() + acknowledgment.resolve() + expect(await atOracleCheckpoint(completion, `replay completed`)).toBe( + `fulfilled`, + ) + await report(settledReports) + expect(calls).toEqual(expected) + expect(await env.executor.peekOutbox()).toEqual([]) + expect(env.serverState.get(`row`)).toEqual(row) + // Collection reads also expose virtual row metadata. + expect(env.collection.get(`row`)).toMatchObject(row) } catch (error) { hasPrimaryFailure = true throw error } finally { - pending.resolve() - restore.mockRestore() + request.resolve() + release.resolve() + acknowledgment.resolve() await cleanupOfflineOracle( [ - () => env.executor.dispose(), + () => { + transaction.rollback() + }, + () => rolledBack, () => env.executor.waitForInit(), + () => completion, + () => turn(), + () => env.executor.dispose(), () => env.collection.cleanup(), ], hasPrimaryFailure, ) } - }), - oracleOptions(startupOracle, seed), - ) - }, -) - -it.each( - [`loss`, `dispose`].flatMap((stop) => - [`provider`, `acknowledgment`, `retry`].flatMap((boundary) => - oracleSeeds(20260917, serialWorkOracle).map((seed) => ({ - stop, - boundary, - seed, - })), + }, ), - ), -)( - `retains serial work after $stop at $boundary (seed $seed)`, - async ({ stop, boundary, seed }) => { - await fc.assert( - fc.asyncProperty( - fc.integer({ min: 2, max: 5 }), - fc.string({ maxLength: 20 }), - async (count, payload) => { - vi.useFakeTimers() - const executions = vi.spyOn( - TransactionExecutor.prototype, - `executeAll`, - ) - const entered = gate() - const provider = gate() - const acknowledgment = gate() - const acknowledging = gate() - const transactions: Array = Array.from( - { length: count }, - (_, index) => ({ - id: `stored-${index}`, - mutationFnName: `syncData`, - mutations: [], - keys: [], - idempotencyKey: `key-${index}`, - createdAt: new Date(index), - retryCount: 0, - nextAttemptAt: 0, - version: 1, - metadata: { payload }, - }), - ) - class Storage extends FakeStorageAdapter { - override async delete(key: string) { - if (boundary === `acknowledgment`) { - acknowledging.resolve() - await acknowledgment.promise - } - return super.delete(key) - } - } - const storage = new Storage() - const outbox = new OutboxManager(storage, {}) - for (const transaction of transactions) await outbox.add(transaction) - const calls: Array<{ id: string; key: string; metadata: unknown }> = - [] - const env = createTestOfflineEnvironment({ - storage, - config: { jitter: false }, - mutationFn: async ({ transaction, idempotencyKey, attempt }) => { - calls.push({ - id: transaction.id, - key: idempotencyKey, - metadata: transaction.metadata, - }) - entered.resolve() - await provider.promise - if ( - boundary === `retry` && - attempt === 1 && - transaction.id === `stored-0` - ) - throw new Error(`transient provider failure`) - }, - }) - const completions = transactions.map(({ id }) => { - const observed: { state: `pending` | `fulfilled` | `rejected` } = { - state: `pending`, - } - void env.executor.waitForTransactionCompletion(id).then( - () => { - observed.state = `fulfilled` - }, - () => { - observed.state = `rejected` - }, - ) - return observed - }) - let replacement: - ReturnType | undefined - const expectedCall = ({ - id, - idempotencyKey, - metadata, - }: OfflineTransaction) => ({ id, key: idempotencyKey, metadata }) - let hasPrimaryFailure = false - try { - await env.executor.waitForInit() - await entered.promise - expect(calls).toEqual(transactions.slice(0, 1).map(expectedCall)) - if (boundary !== `provider`) { - provider.resolve() - if (boundary === `acknowledgment`) await acknowledging.promise - await vi.advanceTimersByTimeAsync(0) - } - if (stop === `loss`) env.leader.setLeader(false) - else env.executor.dispose() - provider.resolve() - acknowledgment.resolve() - const executionCount = executions.mock.calls.length - await vi.advanceTimersByTimeAsync(2000) - expect(executions).toHaveBeenCalledTimes(executionCount) - env.executor.getOnlineDetector().notifyOnline() - await vi.advanceTimersByTimeAsync(0) - - // Only the issued call may finish. Queue length, timers or final rows - // alone cannot tell whether a former owner made extra remote requests. - expect(calls).toEqual(transactions.slice(0, 1).map(expectedCall)) - expect(env.executor.isOfflineEnabled).toBe(false) - expect(completions.map(({ state }) => state)).toEqual( - transactions.map((_, index) => - boundary !== `retry` && index === 0 ? `fulfilled` : `pending`, - ), - ) - const retained = - boundary === `retry` ? transactions : transactions.slice(1) - expect( - (await outbox.getAll()).map( - ({ id, idempotencyKey, metadata }) => ({ - id, - idempotencyKey, - metadata, - }), - ), - ).toEqual( - retained.map(({ id, idempotencyKey, metadata }) => ({ - id, - idempotencyKey, - metadata, - })), - ) + { + ...oracleLaneOptions(leadershipReportOracle, seed), + examples: [ + [ + { + pendingReports: 1, + settledReports: 1, + separateTurns: true, + regainLeadership: true, + value: `payload`, + }, + ], + ], + }, + ) +} - if (stop === `loss`) env.leader.setLeader(true) - else { - replacement = createTestOfflineEnvironment({ - storage, - mutationFn: ({ transaction, idempotencyKey }) => { - calls.push({ - id: transaction.id, - key: idempotencyKey, - metadata: transaction.metadata, - }) - return Promise.resolve() - }, - }) - await replacement.executor.waitForInit() - // Disposed executors cannot be revived by a later elector report. - env.leader.setLeader(true) - } - await vi.advanceTimersByTimeAsync(0) - expect(calls).toEqual([ - ...transactions.slice(0, 1).map(expectedCall), - ...retained.map(expectedCall), - ]) - expect(await outbox.getAll()).toEqual([]) - if (stop === `loss`) - expect( - completions.every(({ state }) => state === `fulfilled`), - ).toBe(true) - } catch (error) { - hasPrimaryFailure = true - throw error - } finally { - provider.resolve() - acknowledgment.resolve() - executions.mockRestore() - vi.useRealTimers() - await cleanupOfflineOracle( - [ - () => env.executor.dispose(), - () => replacement?.executor.dispose(), - () => env.collection.cleanup(), - () => replacement?.collection.cleanup(), - ], - hasPrimaryFailure, - ) - } - }, - ), - oracleOptions(serialWorkOracle, seed), - ) - }, +oracleReplayTest(it).each(reportCases(20260912))( + `preserves replay under repeated reports and leadership regain at $boundary (seed $seed)`, + checkReport, ) - -// Repeated reports and reacquisition must not readmit work already in flight. -// Use the real serializer and executor: an invalid storage key would let a -// zero-execution trace falsely satisfy an at-most-once assertion. -it.each( - [`provider`, `acknowledgment`].flatMap((boundary) => - oracleSeeds(20260912, leadershipReportOracle).map((seed) => ({ - boundary, - seed, - })), - ), -)( +oracleReplayTest(it, REPORT_PROPERTY).each(reportCases(undefined))( `preserves replay under repeated reports and leadership regain at $boundary (seed $seed)`, - async ({ boundary, seed }) => { - await fc.assert( - fc.asyncProperty( - fc.record({ - pendingReports: fc.integer({ min: 0, max: 5 }), - settledReports: fc.integer({ min: 0, max: 5 }), - separateTurns: fc.boolean(), - regainLeadership: fc.boolean(), - value: fc.string({ maxLength: 20 }), - }), - async ({ - pendingReports, - settledReports, - separateTurns, - regainLeadership, - value, - }) => { - const request = gate() - const entered = gate() - const release = gate() - const acknowledging = gate() - const acknowledgment = gate() - class Storage extends FakeStorageAdapter { - override async delete(key: string) { - acknowledging.resolve() - await acknowledgment.promise - return super.delete(key) - } - } - const storage = new Storage() - const callbacks = new Set<(leader: boolean) => void>() - const calls: Array<{ - id: string - key: string - row: Record - }> = [] - const row = { - id: `row`, - value, - completed: false, - updatedAt: new Date(1700000000123), - } - const env = createTestOfflineEnvironment({ - storage, - config: { - leaderElection: { - requestLeadership: async () => { - await request.promise - return true - }, - releaseLeadership: () => {}, - isLeader: () => true, - onLeadershipChange: (callback) => { - callbacks.add(callback) - return () => { - callbacks.delete(callback) - } - }, - }, - }, - mutationFn: async (params) => { - const mutations = params.transaction.mutations - calls.push({ - id: params.transaction.id, - key: params.idempotencyKey, - row: structuredClone(mutations[0].modified), - }) - entered.resolve() - await release.promise - env.applyMutations(mutations) - }, - }) - const completion = env.executor - .waitForTransactionCompletion(`valid`) - .then( - () => `fulfilled`, - (error: unknown) => `rejected:${String(error)}`, - ) - const transaction = createTransaction({ - autoCommit: false, - mutationFn: async () => {}, - }) - const rolledBack = transaction.isPersisted.promise.catch( - () => undefined, - ) - const expected = [{ id: `valid`, key: `once`, row }] - const report = async (count: number) => { - expect(callbacks.size).toBe(1) - for (let i = 0; i < count; i++) { - if (regainLeadership) - for (const callback of callbacks) callback(false) - for (const callback of callbacks) callback(true) - if (separateTurns) await turn() - } - await turn() - } - let hasPrimaryFailure = false - try { - transaction.mutate(() => env.collection.insert(row)) - await new OutboxManager(storage, { - [env.collection.id]: env.collection, - }).add({ - id: `valid`, - mutationFnName: env.mutationFnName, - mutations: transaction.mutations, - keys: transaction.mutations.map((mutation) => mutation.globalKey), - idempotencyKey: `once`, - createdAt: new Date(1700000000000), - retryCount: 0, - nextAttemptAt: 0, - metadata: {}, - version: 1, - }) - transaction.rollback() - await rolledBack - request.resolve() - await atOracleCheckpoint(entered.promise, `replay provider entered`) - await atOracleCheckpoint( - env.executor.waitForInit(), - `replay initialized`, - ) - expect(calls).toEqual(expected) - if (boundary === `acknowledgment`) { - release.resolve() - await atOracleCheckpoint( - acknowledging.promise, - `provider completed; durable deletion held`, - ) - } - await report(pendingReports) - expect(calls).toEqual(expected) - release.resolve() - acknowledgment.resolve() - expect( - await atOracleCheckpoint(completion, `replay completed`), - ).toBe(`fulfilled`) - await report(settledReports) - expect(calls).toEqual(expected) - expect(await env.executor.peekOutbox()).toEqual([]) - expect(env.serverState.get(`row`)).toEqual(row) - // Collection reads also expose virtual row metadata. - expect(env.collection.get(`row`)).toMatchObject(row) - } catch (error) { - hasPrimaryFailure = true - throw error - } finally { - request.resolve() - release.resolve() - acknowledgment.resolve() - await cleanupOfflineOracle( - [ - () => { - transaction.rollback() - }, - () => rolledBack, - () => env.executor.waitForInit(), - () => completion, - () => turn(), - () => env.executor.dispose(), - () => env.collection.cleanup(), - ], - hasPrimaryFailure, - ) - } - }, - ), - { - ...oracleOptions(leadershipReportOracle, seed), - examples: [ - [ - { - pendingReports: 1, - settledReports: 1, - separateTurns: true, - regainLeadership: true, - value: `payload`, - }, - ], - ], - }, - ) - }, + checkReport, ) it(`does not readmit a permanently rejected row from a stale outbox read`, async () => { @@ -1756,88 +1769,96 @@ it(`does not readmit a permanently rejected row from a stale outbox read`, async } }) -it.each(oracleSeeds(20260923, staleReadOracle))( - `filters every durably removed transaction from a generated stale replay read (seed %s)`, - async (seed) => { - await fc.assert( - fc.asyncProperty( - fc.record({ - count: fc.integer({ min: 2, max: 6 }), - removedPrefix: fc.integer({ min: 1, max: 5 }), - }), - async ({ count, removedPrefix }) => { - const captured = gate() - const delivery = gate() - let hold = true - let capturedSnapshot: Array = [] - class Storage extends FakeStorageAdapter { - override async keys() { - const keys = await super.keys() - capturedSnapshot = [...keys] - return keys - } - override async get(key: string) { - const value = await super.get(key) - if (hold && key === `tx:stored-0`) { - captured.resolve() - await delivery.promise - } - return value +// The fixed lane stands aside during any path replay. The random lane runs in +// ordinary and seed-only runs, and it is the replay of STALE_READ_PROPERTY. +async function checkStaleRead(seed: number | undefined): Promise { + await fc.assert( + fc.asyncProperty( + fc.record({ + count: fc.integer({ min: 2, max: 6 }), + removedPrefix: fc.integer({ min: 1, max: 5 }), + }), + async ({ count, removedPrefix }) => { + const captured = gate() + const delivery = gate() + let hold = true + let capturedSnapshot: Array = [] + class Storage extends FakeStorageAdapter { + override async keys() { + const keys = await super.keys() + capturedSnapshot = [...keys] + return keys + } + override async get(key: string) { + const value = await super.get(key) + if (hold && key === `tx:stored-0`) { + captured.resolve() + await delivery.promise } + return value } - const storage = new Storage() - const outbox = new OutboxManager(storage, {}) - const transactions = Array.from({ length: count }, (_, index) => - storedTransaction(`stored-${index}`), + } + const storage = new Storage() + const outbox = new OutboxManager(storage, {}) + const transactions = Array.from({ length: count }, (_, index) => + storedTransaction(`stored-${index}`), + ) + const ledger: Array = [] + for (const transaction of transactions) { + await outbox.add(transaction) + ledger.push({ type: `stored`, id: transaction.id }) + } + const reading = outbox.getAll() + let hasPrimaryFailure = false + try { + await atOracleCheckpoint(captured.promise, `stale replay captured`) + ledger.push({ + type: `read-captured`, + ids: capturedSnapshot.map((key) => key.slice(`tx:`.length)), + }) + const removed = transactions.slice( + 0, + Math.min(count - 1, removedPrefix), ) - const ledger: Array = [] - for (const transaction of transactions) { - await outbox.add(transaction) - ledger.push({ type: `stored`, id: transaction.id }) + await outbox.removeMany(removed.map(({ id }) => id)) + for (const transaction of removed) { + ledger.push({ type: `durably-removed`, id: transaction.id }) } - const reading = outbox.getAll() - let hasPrimaryFailure = false - try { - await atOracleCheckpoint(captured.promise, `stale replay captured`) - ledger.push({ - type: `read-captured`, - ids: capturedSnapshot.map((key) => key.slice(`tx:`.length)), - }) - const removed = transactions.slice( - 0, - Math.min(count - 1, removedPrefix), - ) - await outbox.removeMany(removed.map(({ id }) => id)) - for (const transaction of removed) { - ledger.push({ type: `durably-removed`, id: transaction.id }) - } - hold = false - delivery.resolve() - ledger.push({ type: `read-delivered` }) - - const expected = foldReplayLedger(ledger) - const delivered = await atOracleCheckpoint( - reading, - `stale replay delivered`, - ) - expect(delivered.map(({ id }) => id)).toEqual(expected.unfinished) - expect(expected.staleCapture).not.toEqual(expected.unfinished) - } catch (error) { - hasPrimaryFailure = true - throw error - } finally { - hold = false - delivery.resolve() - await cleanupOfflineOracle([() => reading], hasPrimaryFailure) - } - }, - ), - { - ...oracleOptions(staleReadOracle, seed), - examples: [[{ count: 3, removedPrefix: 1 }]], + hold = false + delivery.resolve() + ledger.push({ type: `read-delivered` }) + + const expected = foldReplayLedger(ledger) + const delivered = await atOracleCheckpoint( + reading, + `stale replay delivered`, + ) + expect(delivered.map(({ id }) => id)).toEqual(expected.unfinished) + expect(expected.staleCapture).not.toEqual(expected.unfinished) + } catch (error) { + hasPrimaryFailure = true + throw error + } finally { + hold = false + delivery.resolve() + await cleanupOfflineOracle([() => reading], hasPrimaryFailure) + } }, - ) - }, + ), + { + ...oracleLaneOptions(staleReadOracle, seed), + examples: [[{ count: 3, removedPrefix: 1 }]], + }, + ) +} + +oracleReplayTest(it).each([20260923])( + `filters every durably removed transaction from a generated stale replay read (seed %s)`, + checkStaleRead, +) +oracleReplayTest(it, STALE_READ_PROPERTY).each([undefined])( + `filters every durably removed transaction from a generated stale replay read (seed %s)`, + checkStaleRead, ) it(`append-only replay judgment rejects stale admission after durable removal`, () => { @@ -1973,241 +1994,249 @@ it.each([1, 4])( // A replay scan can capture A, then block on another storage read until A's // successful deletion. Pending-only dedupe no longer remembers A at admission. // Peers admitted only through this scan must still execute exactly once. -it.each(oracleSeeds(20260919, delayedReadOracle))( - `admits only unfinished work from delayed replay reads (seed %s)`, - async (seed) => { - await fc.assert( - fc.asyncProperty( - fc.record({ - peers: fc.integer({ min: 1, max: 4 }), - scans: fc.integer({ min: 1, max: 3 }), - afterAcknowledgment: fc.boolean(), - payload: fc.string({ maxLength: 12 }), - }), - async ({ peers, scans, afterAcknowledgment, payload }) => { - const entered = gate() - const provider = gate() - const captured = gate() - const delivery = gate() - let hold = false - let captures = 0 - class Storage extends FakeStorageAdapter { - override async get(key: string) { - const value = await super.get(key) - if (hold && key === `tx:peer-0`) { - if (++captures === scans) captured.resolve() - await delivery.promise - } - return value - } - } - const storage = new Storage() - const outbox = new OutboxManager(storage, {}) - const transactions: Array = [ - `active`, - ...Array.from({ length: peers }, (_, index) => `peer-${index}`), - ].map((id, index) => ({ - id, - mutationFnName: `syncData`, - mutations: [], - keys: [], - idempotencyKey: `${id}/once`, - createdAt: new Date(index), - retryCount: 0, - nextAttemptAt: 0, - version: 1, - metadata: { payload }, - })) - await outbox.add(transactions[0]!) - const loads = vi.spyOn( - TransactionExecutor.prototype, - `loadPendingTransactions`, - ) - const executions = vi.spyOn( - TransactionExecutor.prototype, - `executeAll`, - ) - const env = createTestOfflineEnvironment({ - storage, - mutationFn: async ({ transaction }) => { - if (transaction.id === `active`) { - entered.resolve() - await provider.promise - } - }, - }) - const outcomes = transactions.map(({ id }) => - env.executor.waitForTransactionCompletion(id), - ) - // Observe rejection now, even if a preceding reach assertion fails. - for (const outcome of outcomes) void outcome.catch(() => {}) - let hasPrimaryFailure = false - try { - await atOracleCheckpoint( - env.executor.waitForInit(), - `initial replay`, - ) - await atOracleCheckpoint(entered.promise, `provider entered`) - for (const transaction of transactions.slice(1)) - await outbox.add(transaction) - hold = true - const reads: Array> = [] - for (let index = 0; index < scans; index++) { - env.leader.setLeader(false) - env.leader.setLeader(true) - reads.push(loads.mock.results.at(-1)!.value) - } - await atOracleCheckpoint( - captured.promise, - `all reads captured active row`, - ) - expect(captures).toBe(scans) - if (afterAcknowledgment) { - provider.resolve() - await atOracleCheckpoint(outcomes[0]!, `active row acknowledged`) - expect(await outbox.get(`active`)).toBeNull() - expect( - env.mutationCalls.map((call) => call.transaction.id), - ).toEqual([`active`]) - } - hold = false - delivery.resolve() - await atOracleCheckpoint( - Promise.all(reads), - `replay reads delivered`, - ) - provider.resolve() - await atOracleCheckpoint( - Promise.all(outcomes), - `all callers completed`, - ) - await atOracleCheckpoint( - executions.mock.results.at(-1)!.value, - `replay drained`, - ) - expect( - env.mutationCalls.map(({ transaction, idempotencyKey }) => ({ - id: transaction.id, - key: idempotencyKey, - metadata: transaction.metadata, - })), - ).toEqual( - transactions.map(({ id, idempotencyKey, metadata }) => ({ - id, - key: idempotencyKey, - metadata, - })), - ) - expect(storage.snapshot()).toEqual({}) - expect(env.executor.getPendingCount()).toBe(0) - } catch (error) { - hasPrimaryFailure = true - throw error - } finally { - hold = false - provider.resolve() - delivery.resolve() - await cleanupOfflineOracle( - [ - () => env.executor.dispose(), - () => env.collection.cleanup(), - () => { - loads.mockRestore() - executions.mockRestore() - }, - ], - hasPrimaryFailure, - ) - } - }, - ), - { - ...oracleOptions(delayedReadOracle, seed), - examples: [ - [ - { - peers: 1, - scans: 1, - afterAcknowledgment: true, - payload: `witness`, - }, - ], - [ - { - peers: 1, - scans: 1, - afterAcknowledgment: false, - payload: `control`, - }, - ], - ], - }, - ) - }, -) - -it.each( - [`keys`, `get`].flatMap((operation) => - oracleSeeds(20260916, storageFailureOracle).map((seed) => ({ - operation, - seed, - })), - ), -)( - `rejects initialization when storage $operation fails without losing records (seed $seed)`, - async ({ operation, seed }) => { - await fc.assert( - fc.asyncProperty(fc.integer({ min: 1, max: 3 }), async (count) => { - const error = new Error(`stored work unavailable`) +// The fixed lane stands aside during any path replay. The random lane runs in +// ordinary and seed-only runs, and it is the replay of DELAYED_READ_PROPERTY. +async function checkDelayedRead(seed: number | undefined): Promise { + await fc.assert( + fc.asyncProperty( + fc.record({ + peers: fc.integer({ min: 1, max: 4 }), + scans: fc.integer({ min: 1, max: 3 }), + afterAcknowledgment: fc.boolean(), + payload: fc.string({ maxLength: 12 }), + }), + async ({ peers, scans, afterAcknowledgment, payload }) => { + const entered = gate() + const provider = gate() + const captured = gate() + const delivery = gate() + let hold = false + let captures = 0 class Storage extends FakeStorageAdapter { - override async keys() { - if (operation === `keys`) throw error - return super.keys() - } override async get(key: string) { - if (operation === `get`) throw error - return super.get(key) + const value = await super.get(key) + if (hold && key === `tx:peer-0`) { + if (++captures === scans) captured.resolve() + await delivery.promise + } + return value } } const storage = new Storage() const outbox = new OutboxManager(storage, {}) - for (let index = 0; index < count; index++) - await outbox.add({ - id: `stored-${index}`, - mutationFnName: `syncData`, - mutations: [], - keys: [], - idempotencyKey: `key-${index}`, - createdAt: new Date(index), - retryCount: 0, - nextAttemptAt: 0, - version: 1, - }) - const before = storage.snapshot() - const env = createTestOfflineEnvironment({ storage }) + const transactions: Array = [ + `active`, + ...Array.from({ length: peers }, (_, index) => `peer-${index}`), + ].map((id, index) => ({ + id, + mutationFnName: `syncData`, + mutations: [], + keys: [], + idempotencyKey: `${id}/once`, + createdAt: new Date(index), + retryCount: 0, + nextAttemptAt: 0, + version: 1, + metadata: { payload }, + })) + await outbox.add(transactions[0]!) + const loads = vi.spyOn( + TransactionExecutor.prototype, + `loadPendingTransactions`, + ) + const executions = vi.spyOn(TransactionExecutor.prototype, `executeAll`) + const env = createTestOfflineEnvironment({ + storage, + mutationFn: async ({ transaction }) => { + if (transaction.id === `active`) { + entered.resolve() + await provider.promise + } + }, + }) + const outcomes = transactions.map(({ id }) => + env.executor.waitForTransactionCompletion(id), + ) + // Observe rejection now, even if a preceding reach assertion fails. + for (const outcome of outcomes) void outcome.catch(() => {}) let hasPrimaryFailure = false try { - await expect( - atOracleCheckpoint( - env.executor.waitForInit(), - `failed storage initialization`, - ), - ).rejects.toBe(error) - expect(env.mutationCalls).toEqual([]) - expect(storage.snapshot()).toEqual(before) - } catch (failure) { + await atOracleCheckpoint(env.executor.waitForInit(), `initial replay`) + await atOracleCheckpoint(entered.promise, `provider entered`) + for (const transaction of transactions.slice(1)) + await outbox.add(transaction) + hold = true + const reads: Array> = [] + for (let index = 0; index < scans; index++) { + env.leader.setLeader(false) + env.leader.setLeader(true) + reads.push(loads.mock.results.at(-1)!.value) + } + await atOracleCheckpoint( + captured.promise, + `all reads captured active row`, + ) + expect(captures).toBe(scans) + if (afterAcknowledgment) { + provider.resolve() + await atOracleCheckpoint(outcomes[0]!, `active row acknowledged`) + expect(await outbox.get(`active`)).toBeNull() + expect( + env.mutationCalls.map((call) => call.transaction.id), + ).toEqual([`active`]) + } + hold = false + delivery.resolve() + await atOracleCheckpoint(Promise.all(reads), `replay reads delivered`) + provider.resolve() + await atOracleCheckpoint( + Promise.all(outcomes), + `all callers completed`, + ) + await atOracleCheckpoint( + executions.mock.results.at(-1)!.value, + `replay drained`, + ) + expect( + env.mutationCalls.map(({ transaction, idempotencyKey }) => ({ + id: transaction.id, + key: idempotencyKey, + metadata: transaction.metadata, + })), + ).toEqual( + transactions.map(({ id, idempotencyKey, metadata }) => ({ + id, + key: idempotencyKey, + metadata, + })), + ) + expect(storage.snapshot()).toEqual({}) + expect(env.executor.getPendingCount()).toBe(0) + } catch (error) { hasPrimaryFailure = true - throw failure + throw error } finally { + hold = false + provider.resolve() + delivery.resolve() await cleanupOfflineOracle( - [() => env.executor.dispose(), () => env.collection.cleanup()], + [ + () => env.executor.dispose(), + () => env.collection.cleanup(), + () => { + loads.mockRestore() + executions.mockRestore() + }, + ], hasPrimaryFailure, ) } - }), - oracleOptions(storageFailureOracle, seed), - ) - }, + }, + ), + { + ...oracleLaneOptions(delayedReadOracle, seed), + examples: [ + [ + { + peers: 1, + scans: 1, + afterAcknowledgment: true, + payload: `witness`, + }, + ], + [ + { + peers: 1, + scans: 1, + afterAcknowledgment: false, + payload: `control`, + }, + ], + ], + }, + ) +} + +oracleReplayTest(it).each([20260919])( + `admits only unfinished work from delayed replay reads (seed %s)`, + checkDelayedRead, +) +oracleReplayTest(it, DELAYED_READ_PROPERTY).each([undefined])( + `admits only unfinished work from delayed replay reads (seed %s)`, + checkDelayedRead, +) + +const storageFailureCases = (seed: number | undefined) => + [`keys`, `get`].map((operation) => ({ operation, seed })) + +// The fixed lane stands aside during any path replay. The random lane runs in +// ordinary and seed-only runs, and it is the replay of STORAGE_FAILURE_PROPERTY. +async function checkStorageFailure({ + operation, + seed, +}: ReturnType[number]): Promise { + await fc.assert( + fc.asyncProperty(fc.integer({ min: 1, max: 3 }), async (count) => { + const error = new Error(`stored work unavailable`) + class Storage extends FakeStorageAdapter { + override async keys() { + if (operation === `keys`) throw error + return super.keys() + } + override async get(key: string) { + if (operation === `get`) throw error + return super.get(key) + } + } + const storage = new Storage() + const outbox = new OutboxManager(storage, {}) + for (let index = 0; index < count; index++) + await outbox.add({ + id: `stored-${index}`, + mutationFnName: `syncData`, + mutations: [], + keys: [], + idempotencyKey: `key-${index}`, + createdAt: new Date(index), + retryCount: 0, + nextAttemptAt: 0, + version: 1, + }) + const before = storage.snapshot() + const env = createTestOfflineEnvironment({ storage }) + let hasPrimaryFailure = false + try { + await expect( + atOracleCheckpoint( + env.executor.waitForInit(), + `failed storage initialization`, + ), + ).rejects.toBe(error) + expect(env.mutationCalls).toEqual([]) + expect(storage.snapshot()).toEqual(before) + } catch (failure) { + hasPrimaryFailure = true + throw failure + } finally { + await cleanupOfflineOracle( + [() => env.executor.dispose(), () => env.collection.cleanup()], + hasPrimaryFailure, + ) + } + }), + oracleLaneOptions(storageFailureOracle, seed), + ) +} + +oracleReplayTest(it).each(storageFailureCases(20260916))( + `rejects initialization when storage $operation fails without losing records (seed $seed)`, + checkStorageFailure, +) +oracleReplayTest(it, STORAGE_FAILURE_PROPERTY).each( + storageFailureCases(undefined), +)( + `rejects initialization when storage $operation fails without losing records (seed $seed)`, + checkStorageFailure, ) it(`keeps startup replay available when discarded-work cleanup fails`, async () => { diff --git a/packages/offline-transactions/tests/oracle-config.ts b/packages/offline-transactions/tests/oracle-config.ts index 8384b6da64..47ff04632a 100644 --- a/packages/offline-transactions/tests/oracle-config.ts +++ b/packages/offline-transactions/tests/oracle-config.ts @@ -1,44 +1,50 @@ -type OracleEnvironment = Record +import { oraclePropertyOptions, oracleRuns } from '../../db/tests/oracle-config' type OfflineOracleConfigOptions = { - prefix: `OFFLINE_ORACLE` | `TANSTACK_DB_OFFLINE_ORACLE` + /** A property registered in packages/db/tests/oracle-config.ts. */ + property: string + /** The random campaign's budget before TANSTACK_DB_ORACLE_RUNS_MULTIPLIER. */ defaultRuns: number - defaultSeed?: number - environment?: OracleEnvironment + /** Fixed examples that fast-check counts against the same run budget. */ + reservedRuns?: number } +/** + * The budgets of one offline oracle property. `runs` sizes its fixed-seed + * lane. `random` holds the shared property options for its random lane: a seed + * alone reseeds it, and a path replay of `property` runs its recorded history + * once. + */ export function readOfflineOracleConfig({ - prefix, + property, defaultRuns, - defaultSeed, - environment = process.env, + reservedRuns = 0, }: OfflineOracleConfigOptions): { runs: number - seed: number | undefined - path: string | undefined + random: ReturnType } { - const runsText = environment[`${prefix}_RUNS`] ?? String(defaultRuns) - const runs = Number(runsText) - if (runsText.trim() === `` || !Number.isSafeInteger(runs) || runs < 1) { - throw new Error(`Invalid ${prefix}_RUNS`) - } - - const seedText = environment[`${prefix}_SEED`] - const seed = seedText === undefined ? defaultSeed : Number(seedText) - if ( - seedText !== undefined && - (seedText.trim() === `` || !Number.isSafeInteger(seed)) - ) { - throw new Error(`Invalid ${prefix}_SEED`) - } - - const path = environment[`${prefix}_PATH`] - if ( - path !== undefined && - (seed === undefined || !/^\d+(?::\d+)*$/.test(path)) - ) { - throw new Error(`${prefix}_PATH requires a seed and numeric shrink path`) + // Both lanes share one budget: the scaled base plus the reserved fixed + // examples. A path replay keeps its single run. + const runs = oracleRuns(defaultRuns) + reservedRuns + const random = oraclePropertyOptions(defaultRuns, property) + return { + runs, + random: random.path === undefined ? { ...random, numRuns: runs } : random, } +} - return { runs, seed, path } +/** + * A lane that passes fixed `examples` keeps them during a path replay. + * fast-check counts the examples in the recorded path, so a replay without + * them would select a different value. + * + * The fast-check options for one lane. An `undefined` seed selects the random + * lane. A number selects the fixed lane, which keeps its ordinary budget. + */ +export function oracleLaneOptions( + config: ReturnType, + seed: number | undefined, +): + ReturnType | { numRuns: number; seed: number } { + return seed === undefined ? config.random : { numRuns: config.runs, seed } } diff --git a/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts b/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts index 08e12dc5ed..8a7a50c694 100644 --- a/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts +++ b/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts @@ -6,9 +6,10 @@ import { MissingTemporalConstructorError, TransactionSerializer, } from '../src/outbox/TransactionSerializer' +import { oracleReplayTest } from '../../db/tests/oracle-config' import { cleanupOfflineOracle } from './oracle-lifecycle' import { FakeStorageAdapter } from './harness' -import { readOfflineOracleConfig } from './oracle-config' +import { oracleLaneOptions, readOfflineOracleConfig } from './oracle-config' import type { OfflineTransaction } from '../src/types' import type { PendingMutation } from '@tanstack/db' @@ -1363,18 +1364,24 @@ const pinned: Array = [ // fails its ordinary object with an own __proto__ key at the wire checkpoint. // This package's test root is separate from core's named replay portfolio. // Keep a local replay entry point rather than importing files outside rootDir. -const { - runs: numRuns, - seed: replaySeed, - path: replayPath, -} = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, +// Each random campaign is its own replay property, because a recorded path +// belongs to one generator. Its fixed lane and its random lane share one +// generator, budget, and assertion set. +const MUTATION_WIRE_PROPERTY = `offline.serializer.mutation-wire` +const DATE_MARKER_PROPERTY = `offline.serializer.date-marker` +const VERSION_TWO_PROPERTY = `offline.serializer.version-two` +const mutationWireOracle = readOfflineOracleConfig({ + property: MUTATION_WIRE_PROPERTY, + defaultRuns: 100, +}) +const dateMarkerOracle = readOfflineOracleConfig({ + property: DATE_MARKER_PROPERTY, + defaultRuns: 100, +}) +const versionTwoOracle = readOfflineOracleConfig({ + property: VERSION_TWO_PROPERTY, defaultRuns: 100, }) -// An explicit seed selects direct replay. Normal runs keep both the fixed and -// unseeded campaigns with the same generator, budget, and assertions. -const campaigns = (fixedSeed: number): Array => - replaySeed === undefined ? [fixedSeed, undefined] : [undefined] const mutationEdits = fc.array( fc.record({ @@ -1394,63 +1401,67 @@ const mutationMeaningProperty = (fault: Fault = `none`) => checkRoundtrip(edits, time, fault), ) -it.each(campaigns(20260914))( +const checkMutationWire = async (seed: number | undefined): Promise => { + await fc.assert(mutationMeaningProperty(), { + ...oracleLaneOptions(mutationWireOracle, seed), + examples: [[pinned, 1704067200000]], + }) +} + +oracleReplayTest(it).each([20260914])( `preserves mutation wire meaning across restart (seed %s)`, - async (seed) => { - await fc.assert(mutationMeaningProperty(), { - numRuns, - seed: seed ?? replaySeed, - ...(seed === undefined && replayPath !== undefined - ? { path: replayPath } - : {}), - examples: [[pinned, 1704067200000]], + checkMutationWire, +) +oracleReplayTest(it, MUTATION_WIRE_PROPERTY).each([undefined])( + `preserves mutation wire meaning across restart (seed %s)`, + checkMutationWire, +) + +oracleReplayTest(it, MUTATION_WIRE_PROPERTY)( + `replays a shrunk mutation-wire failure at the same checkpoint`, + async () => { + // The fault drops a mutation field after the real encoder runs. It leaves + // zero-edit histories green, so fast-check must find and shrink a generated + // history that reaches the wire assertion. A seed alone reseeds this + // capture, and a path replay of MUTATION_WIRE_PROPERTY runs the recorded + // path once. Captured replay: TANSTACK_DB_ORACLE_SEED=20260914 + // TANSTACK_DB_ORACLE_PATH=0:1:0:0:0:1:1:1:1:1 + // TANSTACK_DB_ORACLE_PROPERTY=offline.serializer.mutation-wire + const property = mutationMeaningProperty(`omit-changes`) + const replay = mutationWireOracle.random + const captured = await fc.check(property, { + ...replay, + seed: replay.seed ?? 20260914, }) + expect(captured.failed).toBe(true) + expect(captured.errorInstance).toMatchObject({ name: `AssertionError` }) + expect(captured.error).toContain(`mutation wire checkpoint`) + if (replay.seed !== undefined) return + + expect(captured.numShrinks).toBeGreaterThan(0) + expect(captured.counterexamplePath).toMatch(/^\d+(?::\d+)*$/) + const capturedPath = captured.counterexamplePath + if (capturedPath === null) throw new Error(`Missing captured shrink path`) + const original = await fc.check(property, { + seed: captured.seed, + numRuns: mutationWireOracle.runs, + endOnFailure: true, + }) + expect(original.failed).toBe(true) + expect(original.errorInstance).toMatchObject({ name: `AssertionError` }) + expect(original.error).toContain(`mutation wire checkpoint`) + const replayed = await fc.check(property, { + seed: captured.seed, + path: capturedPath, + numRuns: mutationWireOracle.runs, + }) + expect(replayed.failed).toBe(true) + expect(replayed.counterexample).toEqual(captured.counterexample) + expect(replayed.errorInstance).toMatchObject({ name: `AssertionError` }) + expect(replayed.error).toContain(`mutation wire checkpoint`) }, ) -it(`replays a shrunk mutation-wire failure at the same checkpoint`, async () => { - // The fault drops a mutation field after the real encoder runs. It leaves - // zero-edit histories green, so fast-check must find and shrink a generated - // history that reaches the wire assertion. The environment variables permit - // a command to run the recorded seed/path without repeating the capture. - // Captured replay: OFFLINE_ORACLE_SEED=20260914 - // OFFLINE_ORACLE_PATH=0:1:0:0:0:1:1:1:1:1 pnpm exec vitest run - // tests/transaction-serializer-oracle.property.test.ts --configLoader runner - // -t 'replays a shrunk mutation-wire failure' - const property = mutationMeaningProperty(`omit-changes`) - const captured = await fc.check(property, { - seed: replaySeed ?? 20260914, - ...(replayPath === undefined ? {} : { path: replayPath }), - numRuns, - }) - expect(captured.failed).toBe(true) - expect(captured.errorInstance).toMatchObject({ name: `AssertionError` }) - expect(captured.error).toContain(`mutation wire checkpoint`) - if (replaySeed !== undefined) return - - expect(captured.numShrinks).toBeGreaterThan(0) - expect(captured.counterexamplePath).toMatch(/^\d+(?::\d+)*$/) - const capturedPath = captured.counterexamplePath - if (capturedPath === null) throw new Error(`Missing captured shrink path`) - const original = await fc.check(property, { - seed: captured.seed, - numRuns, - endOnFailure: true, - }) - expect(original.failed).toBe(true) - expect(original.errorInstance).toMatchObject({ name: `AssertionError` }) - expect(original.error).toContain(`mutation wire checkpoint`) - const replayed = await fc.check(property, { - seed: captured.seed, - path: capturedPath, - numRuns, - }) - expect(replayed.failed).toBe(true) - expect(replayed.counterexample).toEqual(captured.counterexample) - expect(replayed.errorInstance).toMatchObject({ name: `AssertionError` }) - expect(replayed.error).toContain(`mutation wire checkpoint`) -}) - it(`retains the wire mismatch when cleanup also fails`, async () => { let failure: unknown try { @@ -1475,81 +1486,84 @@ it(`retains the wire mismatch when cleanup also fails`, async () => { ]) }) -it.each(campaigns(20260915))( - `reads unversioned Date-marker records across restart (seed %s)`, - async (seed) => { - await fc.assert( - fc.asyncProperty( - fc.array( - fc.record({ - kind: fc.constantFrom(`insert`, `update`, `delete`), - slot: fc.integer({ min: 0, max: 1 }), - before: leaf, - after: leaf, - }), - { minLength: 1, maxLength: pinned.length }, - ), - async (edits) => { - // Old records had no object escape. Use only unambiguous legacy trees, - // but cross nested arrays, ordinary objects, Dates and date-like strings. - const nest = (pair: Pair): Pair => ({ - runtime: { nested: [pair.runtime] }, - wire: { nested: [pair.wire] }, - }) - await checkRoundtrip( - edits.map((edit) => ({ - ...edit, - before: nest(edit.before), - after: nest(edit.after), - })), - 0, - `none`, - `encoder`, - true, - ) - }, +const checkDateMarker = async (seed: number | undefined): Promise => { + await fc.assert( + fc.asyncProperty( + fc.array( + fc.record({ + kind: fc.constantFrom(`insert`, `update`, `delete`), + slot: fc.integer({ min: 0, max: 1 }), + before: leaf, + after: leaf, + }), + { minLength: 1, maxLength: pinned.length }, ), - { - seed: seed ?? replaySeed, - numRuns, - ...(seed === undefined && replayPath !== undefined - ? { path: replayPath } - : {}), - examples: [ - [[{ kind: `update`, slot: 0, before: datePair(0), after: twin }]], - ], + async (edits) => { + // Old records had no object escape. Use only unambiguous legacy trees, + // but cross nested arrays, ordinary objects, Dates and date-like strings. + const nest = (pair: Pair): Pair => ({ + runtime: { nested: [pair.runtime] }, + wire: { nested: [pair.wire] }, + }) + await checkRoundtrip( + edits.map((edit) => ({ + ...edit, + before: nest(edit.before), + after: nest(edit.after), + })), + 0, + `none`, + `encoder`, + true, + ) }, - ) - }, + ), + { + ...oracleLaneOptions(dateMarkerOracle, seed), + examples: [ + [[{ kind: `update`, slot: 0, before: datePair(0), after: twin }]], + ], + }, + ) +} + +oracleReplayTest(it).each([20260915])( + `reads unversioned Date-marker records across restart (seed %s)`, + checkDateMarker, +) +oracleReplayTest(it, DATE_MARKER_PROPERTY).each([undefined])( + `reads unversioned Date-marker records across restart (seed %s)`, + checkDateMarker, ) -it.each(campaigns(20260916))( - `reads version-two escaped values across restart (seed %s)`, - async (seed) => { - await fc.assert( - fc.asyncProperty( - fc.array( - fc.record({ - kind: fc.constantFrom(`insert`, `update`, `delete`), - slot: fc.integer({ min: 0, max: 1 }), - before: tree(2), - after: tree(2), - }), - { minLength: 1, maxLength: pinned.length }, - ), - async (edits) => - checkRoundtrip(edits, 0, `none`, `encoder`, false, true), +const checkVersionTwo = async (seed: number | undefined): Promise => { + await fc.assert( + fc.asyncProperty( + fc.array( + fc.record({ + kind: fc.constantFrom(`insert`, `update`, `delete`), + slot: fc.integer({ min: 0, max: 1 }), + before: tree(2), + after: tree(2), + }), + { minLength: 1, maxLength: pinned.length }, ), - { - seed: seed ?? replaySeed, - numRuns, - ...(seed === undefined && replayPath !== undefined - ? { path: replayPath } - : {}), - examples: [[pinned]], - }, - ) - }, + async (edits) => checkRoundtrip(edits, 0, `none`, `encoder`, false, true), + ), + { + ...oracleLaneOptions(versionTwoOracle, seed), + examples: [[pinned]], + }, + ) +} + +oracleReplayTest(it).each([20260916])( + `reads version-two escaped values across restart (seed %s)`, + checkVersionTwo, +) +oracleReplayTest(it, VERSION_TWO_PROPERTY).each([undefined])( + `reads version-two escaped values across restart (seed %s)`, + checkVersionTwo, ) it.each( diff --git a/packages/offline-transactions/tests/transaction-settlement-oracle.property.test.ts b/packages/offline-transactions/tests/transaction-settlement-oracle.property.test.ts index 6f78560330..dbb6f3bc39 100644 --- a/packages/offline-transactions/tests/transaction-settlement-oracle.property.test.ts +++ b/packages/offline-transactions/tests/transaction-settlement-oracle.property.test.ts @@ -7,13 +7,17 @@ import { OutboxManager } from '../src/outbox/OutboxManager' import { KeyScheduler } from '../src/executor/KeyScheduler' import { TransactionExecutor } from '../src/executor/TransactionExecutor' import { DefaultRetryPolicy } from '../src/retry/RetryPolicy' +import { + oracleReplayRequested, + oracleReplayTest, +} from '../../db/tests/oracle-config' import { FakeStorageAdapter, createTestOfflineEnvironment } from './harness' import { atOracleCheckpoint, captureUnhandledRejections, cleanupOfflineOracle, } from './oracle-lifecycle' -import { readOfflineOracleConfig } from './oracle-config' +import { oracleLaneOptions, readOfflineOracleConfig } from './oracle-config' import type { TestItem } from './harness' import type { Transaction } from '@tanstack/db' import type { @@ -147,40 +151,30 @@ import type { * Here, an offline executor restart constructs a fresh executor over the retained * outbox. Outbox replay resumes unfinished durable work, including terminal-phase * deletion; neither operation is a Collection sync restart or truncate replay. - * To replay one oracle shrink directly, set OFFLINE_ORACLE_SEED and - * OFFLINE_ORACLE_PATH, then select this file and the failing test name. + * To replay one oracle shrink directly, set TANSTACK_DB_ORACLE_SEED, + * TANSTACK_DB_ORACLE_PATH, and TANSTACK_DB_ORACLE_PROPERTY to the failing + * campaign's property (`offline.settlement.*`). The other campaigns in this + * file stand aside. */ +// Each campaign is its own replay property, because a recorded path belongs to +// one generator. A path replay of a campaign's property runs only that campaign. +const SETTLEMENT_PROPERTY = `offline.settlement.transaction` +const RETRY_RECORD_PROPERTY = `offline.settlement.retry-record` +const ADMISSION_PROPERTY = `offline.settlement.admission` +const RETRY_DECISION_PROPERTY = `offline.settlement.retry-decision` const settlementOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: SETTLEMENT_PROPERTY, defaultRuns: 40, }) const retryRecordOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: RETRY_RECORD_PROPERTY, defaultRuns: 20, }) const admissionOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, + property: ADMISSION_PROPERTY, defaultRuns: 10, }) -const retryDecisionOracle = readOfflineOracleConfig({ - prefix: `OFFLINE_ORACLE`, - defaultRuns: 12, -}) - -type OracleConfig = ReturnType - -function oracleSeeds(fixedSeed: number, config: OracleConfig) { - return config.seed === undefined ? [fixedSeed, undefined] : [config.seed] -} - -function oracleOptions(config: OracleConfig, seed: number | undefined) { - return { - numRuns: config.runs, - ...(seed === undefined ? {} : { seed }), - ...(config.path === undefined ? {} : { path: config.path }), - } -} function gate() { let resolve!: () => void @@ -241,593 +235,591 @@ const settlementTransaction = fc tx.commit === `action` ? { ...tx, waitBefore: false } : tx, ) -it.each(oracleSeeds(20260913, settlementOracle))( - `settles each transaction independently while preserving FIFO (seed %s)`, - async (seed) => { - await fc.assert( - fc.asyncProperty( - // One record per transaction, so every commit path and an unread - // receipt are as likely at every position. - fc.record({ - sharedKeys: fc.boolean(), - width: fc.integer({ min: 1, max: 3 }), - transactions: fc.array(settlementTransaction, { - minLength: 2, - maxLength: 5, - }), +const checkSettlement = async (seed: number | undefined): Promise => { + await fc.assert( + fc.asyncProperty( + // One record per transaction, so every commit path and an unread + // receipt are as likely at every position. + fc.record({ + sharedKeys: fc.boolean(), + width: fc.integer({ min: 1, max: 3 }), + transactions: fc.array(settlementTransaction, { + minLength: 2, + maxLength: 5, }), - async ({ sharedKeys, width, transactions }) => { - const succeeds = transactions.map((tx) => tx.succeeds) - // Every path but a manual commit commits after the mutate callback. - const autoCommits = transactions.map((tx) => tx.commit !== `manual`) - // An auto-commit's receipt may go unread, as a caller may. - const unread = transactions.map( - (tx, index) => autoCommits[index]! && !tx.readReceipt, + }), + async ({ sharedKeys, width, transactions }) => { + const succeeds = transactions.map((tx) => tx.succeeds) + // Every path but a manual commit commits after the mutate callback. + const autoCommits = transactions.map((tx) => tx.commit !== `manual`) + // An auto-commit's receipt may go unread, as a caller may. + const unread = transactions.map( + (tx, index) => autoCommits[index]! && !tx.readReceipt, + ) + const failures = succeeds.map( + (_value, index) => new NonRetriableError(`failed-${index}`), + ) + const entered = succeeds.map(() => gate()) + const release = succeeds.map(() => gate()) + const statuses: Record = {} + const observed: Array> = [] + const observe = (name: string, promise: Promise) => { + statuses[name] = `pending` + observed.push( + promise.then( + () => { + statuses[name] = `fulfilled` + }, + (error: unknown) => { + statuses[name] = error + }, + ), ) - const failures = succeeds.map( - (_value, index) => new NonRetriableError(`failed-${index}`), + } + const calls: Array<{ + id: string + rows: Array> + }> = [] + const expectedRows = succeeds.map((_, index) => + Array.from({ length: width }, (_unused, column) => ({ + id: `${sharedKeys ? 0 : index}:${column}`, + value: `value-${index}`, + completed: false, + updatedAt: new Date(1700000000000 + index), + })), + ) + const env = createTestOfflineEnvironment({ + mutationFn: async (params) => { + const index = calls.length + const mutations = params.transaction.mutations + calls.push({ + id: params.transaction.id, + rows: mutations.map((mutation) => + structuredClone(mutation.modified), + ), + }) + if (!entered[index]) + throw new NonRetriableError(`Unexpected extra execution`) + entered[index].resolve() + await release[index]!.promise + if (!succeeds[index]) throw failures[index] + env.applyMutations(mutations) + }, + }) + const ids: Array = [] + // Empty and rolled-back transactions, which settle without the provider. + const settledIds: Array = [] + const made: Array = [] + const expectedServer = new Map() + const assertState = async (completed: number) => { + expect(calls).toEqual( + ids + .slice(0, Math.min(completed + 1, ids.length)) + .map((id, index) => ({ id, rows: expectedRows[index] })), ) - const entered = succeeds.map(() => gate()) - const release = succeeds.map(() => gate()) - const statuses: Record = {} - const observed: Array> = [] - const observe = (name: string, promise: Promise) => { - statuses[name] = `pending` - observed.push( - promise.then( - () => { - statuses[name] = `fulfilled` - }, - (error: unknown) => { - statuses[name] = error - }, + const actualStatuses = Object.fromEntries( + Object.entries(statuses).map(([name, status]) => [ + name, + status instanceof Error ? status.message : status, + ]), + ) + const expectedStatuses = Object.fromEntries([ + ...emptyReceipts.map((name) => [name, `fulfilled`]), + // A plain rollback rejects isPersisted with no reason (core + // contract); the waiter names the rollback with an Error. + ...rolledBack.flatMap(({ wait, receipt }) => [ + [receipt, undefined], + [wait, `Offline transaction was rolled back`], + ]), + ...ids.flatMap((_id, index) => { + const status = + index < completed + ? succeeds[index] + ? `fulfilled` + : failures[index]!.message + : `pending` + return [ + ...(unread[index] ? [] : [[`commit-${index}`, status]]), + [`wait-${index}`, status], + ] + }), + ]) + expect(actualStatuses).toEqual(expectedStatuses) + if (completed === 1) { + expect(() => + expect({ ...actualStatuses, 'wait-1': `fulfilled` }).toEqual( + expectedStatuses, ), - ) + ).toThrowError(expect.objectContaining({ name: `AssertionError` })) } - const calls: Array<{ - id: string - rows: Array> - }> = [] - const expectedRows = succeeds.map((_, index) => - Array.from({ length: width }, (_unused, column) => ({ - id: `${sharedKeys ? 0 : index}:${column}`, - value: `value-${index}`, - completed: false, - updatedAt: new Date(1700000000000 + index), - })), - ) - const env = createTestOfflineEnvironment({ - mutationFn: async (params) => { - const index = calls.length - const mutations = params.transaction.mutations - calls.push({ - id: params.transaction.id, - rows: mutations.map((mutation) => - structuredClone(mutation.modified), - ), - }) - if (!entered[index]) - throw new NonRetriableError(`Unexpected extra execution`) - entered[index].resolve() - await release[index]!.promise - if (!succeeds[index]) throw failures[index] - env.applyMutations(mutations) - }, - }) - const ids: Array = [] - // Empty and rolled-back transactions, which settle without the provider. - const settledIds: Array = [] - const made: Array = [] - const expectedServer = new Map() - const assertState = async (completed: number) => { - expect(calls).toEqual( - ids - .slice(0, Math.min(completed + 1, ids.length)) - .map((id, index) => ({ id, rows: expectedRows[index] })), - ) - const actualStatuses = Object.fromEntries( - Object.entries(statuses).map(([name, status]) => [ - name, - status instanceof Error ? status.message : status, - ]), - ) - const expectedStatuses = Object.fromEntries([ - ...emptyReceipts.map((name) => [name, `fulfilled`]), - // A plain rollback rejects isPersisted with no reason (core - // contract); the waiter names the rollback with an Error. - ...rolledBack.flatMap(({ wait, receipt }) => [ - [receipt, undefined], - [wait, `Offline transaction was rolled back`], - ]), - ...ids.flatMap((_id, index) => { - const status = - index < completed - ? succeeds[index] - ? `fulfilled` - : failures[index]!.message - : `pending` - return [ - ...(unread[index] ? [] : [[`commit-${index}`, status]]), - [`wait-${index}`, status], - ] - }), - ]) - expect(actualStatuses).toEqual(expectedStatuses) - if (completed === 1) { - expect(() => - expect({ ...actualStatuses, 'wait-1': `fulfilled` }).toEqual( - expectedStatuses, - ), - ).toThrowError( - expect.objectContaining({ name: `AssertionError` }), - ) - } - for (let index = 0; index < completed; index++) { - if (!succeeds[index]) { - if (!unread[index]) - expect(statuses[`commit-${index}`]).toBe(failures[index]) - expect(statuses[`wait-${index}`]).toBe(failures[index]) - } - } - expect(env.serverState).toEqual(expectedServer) - // Queued offline transactions already contributed optimistic state. - // A sibling's failed insert does not remove it. - const expectedLocal = new Map(expectedServer) - for (const rows of expectedRows.slice(completed)) - for (const row of rows) expectedLocal.set(row.id, row) - expectLocalRows(env.collection.toArray, expectedLocal.values()) - if (completed === 1) { - const wrong = localRows(env.collection.toArray) - if (wrong.length > 0) wrong[0]!.completed = !wrong[0]!.completed - else wrong.push(expectedRows[0]![0]!) - expect(() => - expectLocalRows(wrong, expectedLocal.values()), - ).toThrowError( - expect.objectContaining({ name: `AssertionError` }), - ) + for (let index = 0; index < completed; index++) { + if (!succeeds[index]) { + if (!unread[index]) + expect(statuses[`commit-${index}`]).toBe(failures[index]) + expect(statuses[`wait-${index}`]).toBe(failures[index]) } - expect( - (await env.executor.peekOutbox()).map((tx) => tx.id), - ).toEqual(ids.slice(completed)) } - // Receipts of empty transactions, which settle before any provider. - const emptyReceipts: Array = [] - // Receipts of transactions rolled back before they commit. - const rolledBack: Array<{ wait: string; receipt: string }> = [] - // Start a transaction on one commit path. An auto-commit has no - // commit() caller; its receipt is isPersisted, which settles like a - // manual commit's. - const start = ( - commit: SettlementCommit, - write: () => void, - beforeMutate?: (id: string) => void, - ) => { - if (commit === `action`) { - const transaction = env.executor.createOfflineAction({ - mutationFnName: env.mutationFnName, - onMutate: write, - })(undefined) - return { - transaction, - receipt: () => transaction.when(`settled`), - } - } - const tx = env.executor.createOfflineTransaction({ + expect(env.serverState).toEqual(expectedServer) + // Queued offline transactions already contributed optimistic state. + // A sibling's failed insert does not remove it. + const expectedLocal = new Map(expectedServer) + for (const rows of expectedRows.slice(completed)) + for (const row of rows) expectedLocal.set(row.id, row) + expectLocalRows(env.collection.toArray, expectedLocal.values()) + if (completed === 1) { + const wrong = localRows(env.collection.toArray) + if (wrong.length > 0) wrong[0]!.completed = !wrong[0]!.completed + else wrong.push(expectedRows[0]![0]!) + expect(() => + expectLocalRows(wrong, expectedLocal.values()), + ).toThrowError(expect.objectContaining({ name: `AssertionError` })) + } + expect((await env.executor.peekOutbox()).map((tx) => tx.id)).toEqual( + ids.slice(completed), + ) + } + // Receipts of empty transactions, which settle before any provider. + const emptyReceipts: Array = [] + // Receipts of transactions rolled back before they commit. + const rolledBack: Array<{ wait: string; receipt: string }> = [] + // Start a transaction on one commit path. An auto-commit has no + // commit() caller; its receipt is isPersisted, which settles like a + // manual commit's. + const start = ( + commit: SettlementCommit, + write: () => void, + beforeMutate?: (id: string) => void, + ) => { + if (commit === `action`) { + const transaction = env.executor.createOfflineAction({ mutationFnName: env.mutationFnName, - // A transaction auto-commits when the option is omitted. - ...(commit === `default` - ? {} - : { autoCommit: commit === `auto` }), - }) - beforeMutate?.(tx.id) - const transaction = tx.mutate(write) + onMutate: write, + })(undefined) return { transaction, - receipt: (): Promise => - commit === `manual` ? tx.commit() : transaction.when(`settled`), + receipt: () => transaction.when(`settled`), } } - let hasPrimaryFailure = false - // No failure, manual or auto-committed, escapes as an unhandled - // process rejection, and none is logged again: isPersisted reports - // it. - const { rejections: unhandled, cleanup: cleanupUnhandled } = - captureUnhandledRejections() - const logged = vi.spyOn(console, `error`).mockImplementation(() => {}) - // Settlement writes no debug output. - const debug = vi.spyOn(console, `log`).mockImplementation(() => {}) - try { - await env.waitForLeader() - for (let index = 0; index < succeeds.length; index++) { - const { commit, emptyBefore, rolledBackBefore } = - transactions[index]! - if (rolledBackBefore) { - const tx = env.executor.createOfflineTransaction({ - mutationFnName: env.mutationFnName, - autoCommit: false, - }) - const transaction = tx.mutate(() => - env.collection.insert({ - id: `rolled-${index}`, - value: `rolled`, - completed: false, - updatedAt: new Date(0), - }), - ) - const names = { - wait: `rolled-wait-${index}`, - receipt: `rolled-receipt-${index}`, - } - observe( - names.wait, - env.executor.waitForTransactionCompletion(transaction.id), - ) - observe(names.receipt, transaction.when(`settled`)) - rolledBack.push(names) - settledIds.push(transaction.id) - tx.rollback() - } - if (emptyBefore) { - const empty = start(commit, () => {}) - observe( - `empty-wait-${index}`, - env.executor.waitForTransactionCompletion( - empty.transaction.id, - ), - ) - emptyReceipts.push(`empty-wait-${index}`) - settledIds.push(empty.transaction.id) - if (!unread[index]) { - observe(`empty-commit-${index}`, empty.receipt()) - emptyReceipts.push(`empty-commit-${index}`) - } - } - const write = () => { - for (const row of expectedRows[index]!) { - if (sharedKeys && index > 0) { - env.collection.update(row.id, (draft) => { - draft.value = row.value - draft.updatedAt = row.updatedAt - }) - } else env.collection.insert(row) - } + const tx = env.executor.createOfflineTransaction({ + mutationFnName: env.mutationFnName, + // A transaction auto-commits when the option is omitted. + ...(commit === `default` ? {} : { autoCommit: commit === `auto` }), + }) + beforeMutate?.(tx.id) + const transaction = tx.mutate(write) + return { + transaction, + receipt: (): Promise => + commit === `manual` ? tx.commit() : transaction.when(`settled`), + } + } + let hasPrimaryFailure = false + // No failure, manual or auto-committed, escapes as an unhandled + // process rejection, and none is logged again: isPersisted reports + // it. + const { rejections: unhandled, cleanup: cleanupUnhandled } = + captureUnhandledRejections() + const logged = vi.spyOn(console, `error`).mockImplementation(() => {}) + // Settlement writes no debug output. + const debug = vi.spyOn(console, `log`).mockImplementation(() => {}) + try { + await env.waitForLeader() + for (let index = 0; index < succeeds.length; index++) { + const { commit, emptyBefore, rolledBackBefore } = + transactions[index]! + if (rolledBackBefore) { + const tx = env.executor.createOfflineTransaction({ + mutationFnName: env.mutationFnName, + autoCommit: false, + }) + const transaction = tx.mutate(() => + env.collection.insert({ + id: `rolled-${index}`, + value: `rolled`, + completed: false, + updatedAt: new Date(0), + }), + ) + const names = { + wait: `rolled-wait-${index}`, + receipt: `rolled-receipt-${index}`, } - const waitFor = (id: string) => - observe( - `wait-${index}`, - env.executor.waitForTransactionCompletion(id), - ) - const { transaction, receipt } = start( - commit, - write, - transactions[index]!.waitBefore ? waitFor : undefined, + observe( + names.wait, + env.executor.waitForTransactionCompletion(transaction.id), ) - ids.push(transaction.id) - made.push(transaction) - if (!(`wait-${index}` in statuses)) waitFor(transaction.id) - if (!unread[index]) observe(`commit-${index}`, receipt()) - if (index === 0) - await atOracleCheckpoint( - entered[0]!.promise, - `first provider entered`, - ) + observe(names.receipt, transaction.when(`settled`)) + rolledBack.push(names) + settledIds.push(transaction.id) + tx.rollback() } - // Drain enqueue continuations with the first real provider still held. - await turn() - await assertState(0) - for (let index = 0; index < succeeds.length; index++) { - release[index]!.resolve() - if (index + 1 < succeeds.length) - await atOracleCheckpoint( - entered[index + 1]!.promise, - `provider ${index + 1} entered`, - ) - else - await atOracleCheckpoint( - Promise.all(observed), - `all transactions settled`, - ) - await turn() - if (succeeds[index]) - for (const row of expectedRows[index]!) - expectedServer.set(row.id, row) - await assertState(index + 1) + if (emptyBefore) { + const empty = start(commit, () => {}) + observe( + `empty-wait-${index}`, + env.executor.waitForTransactionCompletion(empty.transaction.id), + ) + emptyReceipts.push(`empty-wait-${index}`) + settledIds.push(empty.transaction.id) + if (!unread[index]) { + observe(`empty-commit-${index}`, empty.receipt()) + emptyReceipts.push(`empty-commit-${index}`) + } } - await turn() - expect(unhandled, `unhandled rejections`).toEqual([]) - // A waiter registered after its transaction settled has nothing to - // wait for, so it rejects instead of hanging. This covers every - // settlement path: the provider, an empty transaction, and a - // rollback before commit. An id this executor never saw can still - // arrive from another tab, so its waiter stays pending. - const lateOutcome = (id: string) => - env.executor.waitForTransactionCompletion(id).then( - () => `fulfilled`, - (error: unknown) => - error instanceof Error && - error.message.includes(`already settled`) - ? `rejected` - : `wrong rejection: ${String(error)}`, + const write = () => { + for (const row of expectedRows[index]!) { + if (sharedKeys && index > 0) { + env.collection.update(row.id, (draft) => { + draft.value = row.value + draft.updatedAt = row.updatedAt + }) + } else env.collection.insert(row) + } + } + const waitFor = (id: string) => + observe( + `wait-${index}`, + env.executor.waitForTransactionCompletion(id), + ) + const { transaction, receipt } = start( + commit, + write, + transactions[index]!.waitBefore ? waitFor : undefined, + ) + ids.push(transaction.id) + made.push(transaction) + if (!(`wait-${index}` in statuses)) waitFor(transaction.id) + if (!unread[index]) observe(`commit-${index}`, receipt()) + if (index === 0) + await atOracleCheckpoint( + entered[0]!.promise, + `first provider entered`, + ) + } + // Drain enqueue continuations with the first real provider still held. + await turn() + await assertState(0) + for (let index = 0; index < succeeds.length; index++) { + release[index]!.resolve() + if (index + 1 < succeeds.length) + await atOracleCheckpoint( + entered[index + 1]!.promise, + `provider ${index + 1} entered`, + ) + else + await atOracleCheckpoint( + Promise.all(observed), + `all transactions settled`, ) - const late = [...ids, ...settledIds].map((id) => ({ - id, - outcome: lateOutcome(id), - })) - const unknown = lateOutcome(`never-queued`) await turn() - const pending = turn().then(() => `pending`) - for (const { id, outcome } of late) - expect( - await Promise.race([outcome, pending]), - `late waiter for ${id}`, - ).toBe(`rejected`) - expect( - await Promise.race([unknown, pending]), - `waiter for an unknown id`, - ).toBe(`pending`) - // A receipt nobody read still settles with its own outcome; read - // it only now, after the unhandled-rejection checkpoint. - for (const [index, transaction] of made.entries()) { - if (!unread[index]) continue - if (succeeds[index]) - await expect(transaction.when(`settled`)).resolves.toBe( - transaction, - ) - else - await expect(transaction.when(`settled`)).rejects.toBe( - failures[index], - ) - } - // A failure of the mutation function is reported through the - // receipt only; nothing is logged as an error. - expect(logged.mock.calls, `errors logged`).toEqual([]) - expect(debug.mock.calls, `debug output`).toEqual([]) - } catch (error) { - hasPrimaryFailure = true - throw error - } finally { - cleanupUnhandled() - logged.mockRestore() - debug.mockRestore() - for (const item of release) item.resolve() - await cleanupOfflineOracle( - [ - () => Promise.all(observed), - () => env.executor.dispose(), - () => env.collection.cleanup(), - ], - hasPrimaryFailure, + if (succeeds[index]) + for (const row of expectedRows[index]!) + expectedServer.set(row.id, row) + await assertState(index + 1) + } + await turn() + expect(unhandled, `unhandled rejections`).toEqual([]) + // A waiter registered after its transaction settled has nothing to + // wait for, so it rejects instead of hanging. This covers every + // settlement path: the provider, an empty transaction, and a + // rollback before commit. An id this executor never saw can still + // arrive from another tab, so its waiter stays pending. + const lateOutcome = (id: string) => + env.executor.waitForTransactionCompletion(id).then( + () => `fulfilled`, + (error: unknown) => + error instanceof Error && + error.message.includes(`already settled`) + ? `rejected` + : `wrong rejection: ${String(error)}`, ) + const late = [...ids, ...settledIds].map((id) => ({ + id, + outcome: lateOutcome(id), + })) + const unknown = lateOutcome(`never-queued`) + await turn() + const pending = turn().then(() => `pending`) + for (const { id, outcome } of late) + expect( + await Promise.race([outcome, pending]), + `late waiter for ${id}`, + ).toBe(`rejected`) + expect( + await Promise.race([unknown, pending]), + `waiter for an unknown id`, + ).toBe(`pending`) + // A receipt nobody read still settles with its own outcome; read + // it only now, after the unhandled-rejection checkpoint. + for (const [index, transaction] of made.entries()) { + if (!unread[index]) continue + if (succeeds[index]) + await expect(transaction.when(`settled`)).resolves.toBe( + transaction, + ) + else + await expect(transaction.when(`settled`)).rejects.toBe( + failures[index], + ) } - }, - ), - { - ...oracleOptions(settlementOracle, seed), - examples: [ - [ - { - sharedKeys: true, - width: 1, - transactions: [ - { - succeeds: true, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: true, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - ], - }, - ], - [ - { - sharedKeys: false, - width: 2, - transactions: [ - { - succeeds: true, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: false, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: true, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - ], - }, - ], - [ - { - sharedKeys: true, - width: 2, - transactions: [ - { - succeeds: false, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: false, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: false, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - ], - }, - ], - [ - { - sharedKeys: true, - width: 1, - transactions: [ - { - succeeds: false, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: true, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: false, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - ], - }, - ], - [ - { - sharedKeys: true, - width: 1, - transactions: [ - { - succeeds: false, - commit: `auto`, - readReceipt: false, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: false, - }, - { - succeeds: true, - commit: `manual`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: true, - }, - { - succeeds: false, - commit: `auto`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: false, - }, - ], - }, - ], - // A failed auto-commit with the default option, and a failed - // action, settle through isPersisted only. - [ - { - sharedKeys: false, - width: 1, - transactions: [ - { - succeeds: false, - commit: `default`, - readReceipt: false, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: false, - }, - { - succeeds: false, - commit: `action`, - readReceipt: true, - emptyBefore: false, - rolledBackBefore: false, - waitBefore: false, - }, - ], - }, - ], - // Empty transactions on every commit path complete without the - // provider, before and between real ones. - [ - { - sharedKeys: false, - width: 1, - transactions: [ - { - succeeds: true, - commit: `manual`, - readReceipt: true, - emptyBefore: true, - rolledBackBefore: true, - waitBefore: false, - }, - { - succeeds: false, - commit: `auto`, - readReceipt: false, - emptyBefore: true, - rolledBackBefore: true, - waitBefore: false, - }, - { - succeeds: true, - commit: `default`, - readReceipt: true, - emptyBefore: true, - rolledBackBefore: true, - waitBefore: false, - }, - { - succeeds: true, - commit: `action`, - readReceipt: true, - emptyBefore: true, - rolledBackBefore: true, - waitBefore: false, - }, - ], - }, - ], - ], + // A failure of the mutation function is reported through the + // receipt only; nothing is logged as an error. + expect(logged.mock.calls, `errors logged`).toEqual([]) + expect(debug.mock.calls, `debug output`).toEqual([]) + } catch (error) { + hasPrimaryFailure = true + throw error + } finally { + cleanupUnhandled() + logged.mockRestore() + debug.mockRestore() + for (const item of release) item.resolve() + await cleanupOfflineOracle( + [ + () => Promise.all(observed), + () => env.executor.dispose(), + () => env.collection.cleanup(), + ], + hasPrimaryFailure, + ) + } }, - ) - }, + ), + { + ...oracleLaneOptions(settlementOracle, seed), + examples: [ + [ + { + sharedKeys: true, + width: 1, + transactions: [ + { + succeeds: true, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: true, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + ], + }, + ], + [ + { + sharedKeys: false, + width: 2, + transactions: [ + { + succeeds: true, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: false, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: true, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + ], + }, + ], + [ + { + sharedKeys: true, + width: 2, + transactions: [ + { + succeeds: false, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: false, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: false, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + ], + }, + ], + [ + { + sharedKeys: true, + width: 1, + transactions: [ + { + succeeds: false, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: true, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: false, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + ], + }, + ], + [ + { + sharedKeys: true, + width: 1, + transactions: [ + { + succeeds: false, + commit: `auto`, + readReceipt: false, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: false, + }, + { + succeeds: true, + commit: `manual`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: true, + }, + { + succeeds: false, + commit: `auto`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: false, + }, + ], + }, + ], + // A failed auto-commit with the default option, and a failed + // action, settle through isPersisted only. + [ + { + sharedKeys: false, + width: 1, + transactions: [ + { + succeeds: false, + commit: `default`, + readReceipt: false, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: false, + }, + { + succeeds: false, + commit: `action`, + readReceipt: true, + emptyBefore: false, + rolledBackBefore: false, + waitBefore: false, + }, + ], + }, + ], + // Empty transactions on every commit path complete without the + // provider, before and between real ones. + [ + { + sharedKeys: false, + width: 1, + transactions: [ + { + succeeds: true, + commit: `manual`, + readReceipt: true, + emptyBefore: true, + rolledBackBefore: true, + waitBefore: false, + }, + { + succeeds: false, + commit: `auto`, + readReceipt: false, + emptyBefore: true, + rolledBackBefore: true, + waitBefore: false, + }, + { + succeeds: true, + commit: `default`, + readReceipt: true, + emptyBefore: true, + rolledBackBefore: true, + waitBefore: false, + }, + { + succeeds: true, + commit: `action`, + readReceipt: true, + emptyBefore: true, + rolledBackBefore: true, + waitBefore: false, + }, + ], + }, + ], + ], + }, + ) +} + +oracleReplayTest(it).each([20260913])( + `settles each transaction independently while preserving FIFO (seed %s)`, + checkSettlement, +) +oracleReplayTest(it, SETTLEMENT_PROPERTY).each([undefined])( + `settles each transaction independently while preserving FIFO (seed %s)`, + checkSettlement, ) it.each([ @@ -1025,8 +1017,11 @@ const retryDecisionExamples: Array<[RetryDecisionCase, number]> = [ ] // fast-check counts examples against numRuns. Reserve a run for every fixed // cell, then use the configured budget for generated peer-count combinations. -const retryDecisionRuns = - retryDecisionExamples.length + retryDecisionOracle.runs +const retryDecisionOracle = readOfflineOracleConfig({ + property: RETRY_DECISION_PROPERTY, + defaultRuns: 12, + reservedRuns: retryDecisionExamples.length, +}) // This table is the contract model, not a copy of the executor. The permanent // error has no configurable decision. Otherwise an explicit hook answer wins; @@ -1098,268 +1093,271 @@ function retryDecisionProviderError(kind: RetryErrorKind): Error { // Duplicate IDs and storage failures belong to other settlement grammars here. // After each normal campaign, the reach check proves that fast-check executed // every fixed cell. A seed-and-path replay runs only its requested history. -it.each(oracleSeeds(20261008, retryDecisionOracle))( - `refines an optional retry decision at the durable FIFO checkpoint (seed %s)`, - async (seed) => { - const reachedCases = new Set() - await fc.assert( - fc.asyncProperty( - fc.constantFrom(...retryDecisionCases), - fc.integer({ min: 1, max: 3 }), - async (scenario, peerCount) => { - reachedCases.add(`${scenario.errorKind}:${scenario.hook}`) - const providerError = retryDecisionProviderError(scenario.errorKind) - const firstEntered = gate() - const releaseFirst = gate() - const firstPeerEntered = gate() - const releaseFirstPeer = gate() - const peersAdmitted = gate() - const decisionStored = gate() - let headId = `` - const peerIds: Array = [] - const admittedPeers = new Set() - const calls: Array = [] - const hookCalls: Array<{ error: Error; retryCount: number }> = [] - let headAttempts = 0 - let storedDecision: `retry` | `terminal` | undefined - class DecisionStorage extends FakeStorageAdapter { - override async set(key: string, value: string): Promise { - await super.set(key, value) - if (peerIds.some((id) => key === `tx:${id}`)) { - admittedPeers.add(key) - if (admittedPeers.size === peerCount) peersAdmitted.resolve() - } - if ( - key === `tx:${headId}` && - (JSON.parse(value) as { retryCount: number }).retryCount === 1 - ) { - storedDecision = `retry` - decisionStored.resolve() - } +async function checkRetryDecision(seed: number | undefined): Promise { + const reachedCases = new Set() + await fc.assert( + fc.asyncProperty( + fc.constantFrom(...retryDecisionCases), + fc.integer({ min: 1, max: 3 }), + async (scenario, peerCount) => { + reachedCases.add(`${scenario.errorKind}:${scenario.hook}`) + const providerError = retryDecisionProviderError(scenario.errorKind) + const firstEntered = gate() + const releaseFirst = gate() + const firstPeerEntered = gate() + const releaseFirstPeer = gate() + const peersAdmitted = gate() + const decisionStored = gate() + let headId = `` + const peerIds: Array = [] + const admittedPeers = new Set() + const calls: Array = [] + const hookCalls: Array<{ error: Error; retryCount: number }> = [] + let headAttempts = 0 + let storedDecision: `retry` | `terminal` | undefined + class DecisionStorage extends FakeStorageAdapter { + override async set(key: string, value: string): Promise { + await super.set(key, value) + if (peerIds.some((id) => key === `tx:${id}`)) { + admittedPeers.add(key) + if (admittedPeers.size === peerCount) peersAdmitted.resolve() } - - override async delete(key: string): Promise { - await super.delete(key) - if (key === `tx:${headId}`) { - storedDecision = `terminal` - decisionStored.resolve() - } + if ( + key === `tx:${headId}` && + (JSON.parse(value) as { retryCount: number }).retryCount === 1 + ) { + storedDecision = `retry` + decisionStored.resolve() } } - const shouldRetry: NonNullable = ( - error: Error, - retryCount: number, - ): boolean | undefined => { - hookCalls.push({ error, retryCount }) - if (scenario.hook === `defer`) return undefined - return scenario.hook === `retry` + + override async delete(key: string): Promise { + await super.delete(key) + if (key === `tx:${headId}`) { + storedDecision = `terminal` + decisionStored.resolve() + } } - const config = - scenario.hook === `absent` - ? { jitter: false } - : { jitter: false, shouldRetry } - const env = createTestOfflineEnvironment({ - storage: new DecisionStorage(), - config, - mutationFn: async (params) => { - const id = params.transaction.id - calls.push(id) - if (id === headId && headAttempts++ === 0) { - firstEntered.resolve() - await releaseFirst.promise - throw providerError - } - if (id === peerIds[0]) { - firstPeerEntered.resolve() - await releaseFirstPeer.promise - } - env.applyMutations(params.transaction.mutations) - }, + } + const shouldRetry: NonNullable = ( + error: Error, + retryCount: number, + ): boolean | undefined => { + hookCalls.push({ error, retryCount }) + if (scenario.hook === `defer`) return undefined + return scenario.hook === `retry` + } + const config = + scenario.hook === `absent` + ? { jitter: false } + : { jitter: false, shouldRetry } + const env = createTestOfflineEnvironment({ + storage: new DecisionStorage(), + config, + mutationFn: async (params) => { + const id = params.transaction.id + calls.push(id) + if (id === headId && headAttempts++ === 0) { + firstEntered.resolve() + await releaseFirst.promise + throw providerError + } + if (id === peerIds[0]) { + firstPeerEntered.resolve() + await releaseFirstPeer.promise + } + env.applyMutations(params.transaction.mutations) + }, + }) + const warning = vi.spyOn(console, `warn`).mockImplementation(() => {}) + let headCommitStatus: unknown = `pending` + let headWaitStatus: unknown = `pending` + const peerStatuses: Array = [] + const observed: Array> = [] + let hasPrimaryFailure = false + try { + await env.waitForLeader() + const head = env.executor.createOfflineTransaction({ + mutationFnName: env.mutationFnName, + autoCommit: false, }) - const warning = vi.spyOn(console, `warn`).mockImplementation(() => {}) - let headCommitStatus: unknown = `pending` - let headWaitStatus: unknown = `pending` - const peerStatuses: Array = [] - const observed: Array> = [] - let hasPrimaryFailure = false - try { - await env.waitForLeader() - const head = env.executor.createOfflineTransaction({ + headId = head.id + observed.push( + env.executor.waitForTransactionCompletion(headId).then( + () => { + headWaitStatus = `fulfilled` + }, + (error: unknown) => { + headWaitStatus = error + }, + ), + ) + head.mutate(() => + env.collection.insert({ + id: `head`, + value: `head`, + completed: false, + updatedAt: new Date(0), + }), + ) + observed.push( + head.commit().then( + () => { + headCommitStatus = `fulfilled` + }, + (error: unknown) => { + headCommitStatus = error + }, + ), + ) + await atOracleCheckpoint( + firstEntered.promise, + `head provider entered`, + ) + + for (let index = 0; index < peerCount; index++) { + const peer = env.executor.createOfflineTransaction({ mutationFnName: env.mutationFnName, autoCommit: false, }) - headId = head.id - observed.push( - env.executor.waitForTransactionCompletion(headId).then( - () => { - headWaitStatus = `fulfilled` - }, - (error: unknown) => { - headWaitStatus = error - }, - ), - ) - head.mutate(() => + peerIds.push(peer.id) + peer.mutate(() => env.collection.insert({ - id: `head`, - value: `head`, + id: `peer-${index}`, + value: `peer-${index}`, completed: false, - updatedAt: new Date(0), + updatedAt: new Date(index + 1), }), ) + peerStatuses[index] = `pending` observed.push( - head.commit().then( + peer.commit().then( () => { - headCommitStatus = `fulfilled` + peerStatuses[index] = `fulfilled` }, (error: unknown) => { - headCommitStatus = error + peerStatuses[index] = error }, ), ) - await atOracleCheckpoint( - firstEntered.promise, - `head provider entered`, - ) - - for (let index = 0; index < peerCount; index++) { - const peer = env.executor.createOfflineTransaction({ - mutationFnName: env.mutationFnName, - autoCommit: false, - }) - peerIds.push(peer.id) - peer.mutate(() => - env.collection.insert({ - id: `peer-${index}`, - value: `peer-${index}`, - completed: false, - updatedAt: new Date(index + 1), - }), - ) - peerStatuses[index] = `pending` - observed.push( - peer.commit().then( - () => { - peerStatuses[index] = `fulfilled` - }, - (error: unknown) => { - peerStatuses[index] = error - }, - ), - ) - } - await atOracleCheckpoint(peersAdmitted.promise, `peers admitted`) - - // The provider failure occurs with later work already durable. - // Observe only after the head's retry record or deletion settles. - const releasedAt = Date.now() - releaseFirst.resolve() - await atOracleCheckpoint( - decisionStored.promise, - `head decision stored`, - ) - const expected = expectedRetryDecision( - scenario, - headId, - peerIds, - providerError, - ) - expect(storedDecision).toBe(expected.retry ? `retry` : `terminal`) - if (!expected.retry) - await atOracleCheckpoint( - firstPeerEntered.promise, - `terminal decision admitted peer`, - ) - await turn() - // These public observations distinguish retaining the head from - // removing it, including per-ID settlement and optimistic state. - expect(calls).toEqual(expected.calls) - expect( - (await env.executor.peekOutbox()).map(({ id, retryCount }) => ({ - id, - retryCount, - })), - ).toEqual(expected.outbox) - expect(headCommitStatus).toBe(expected.headStatus) - expect(headWaitStatus).toBe(expected.headStatus) - expect(peerStatuses).toEqual(peerIds.map(() => `pending`)) - expect(env.collection.toArray.map(({ id }) => id).sort()).toEqual( - expected.localRows, - ) - expect(env.serverState.size).toBe(0) - expect(hookCalls).toEqual( - scenario.hook === `absent` || scenario.errorKind === `permanent` - ? [] - : [{ error: providerError, retryCount: 0 }], - ) - if (hookCalls.length > 0) - expect(hookCalls[0]!.error).toBe(providerError) - if (expected.retry) { - const [record] = await env.executor.peekOutbox() - expect(record?.lastError?.message).toBe(providerError.message) - expect(record!.nextAttemptAt).toBeGreaterThan(releasedAt) - } + } + await atOracleCheckpoint(peersAdmitted.promise, `peers admitted`) - releaseFirstPeer.resolve() - if (expected.retry) env.executor.getOnlineDetector().notifyOnline() + // The provider failure occurs with later work already durable. + // Observe only after the head's retry record or deletion settles. + const releasedAt = Date.now() + releaseFirst.resolve() + await atOracleCheckpoint( + decisionStored.promise, + `head decision stored`, + ) + const expected = expectedRetryDecision( + scenario, + headId, + peerIds, + providerError, + ) + expect(storedDecision).toBe(expected.retry ? `retry` : `terminal`) + if (!expected.retry) await atOracleCheckpoint( - Promise.all(observed), - `retry decision history settled`, - ) - expect(headCommitStatus).toBe( - expected.retry ? `fulfilled` : providerError, - ) - expect(headWaitStatus).toBe(headCommitStatus) - expect(peerStatuses).toEqual(peerIds.map(() => `fulfilled`)) - expect(calls).toEqual( - expected.retry - ? [headId, headId, ...peerIds] - : [headId, ...peerIds], - ) - expect(await env.executor.peekOutbox()).toEqual([]) - expect([...env.serverState.keys()].sort()).toEqual( - expected.localRows, - ) - } catch (error) { - hasPrimaryFailure = true - throw error - } finally { - releaseFirst.resolve() - releaseFirstPeer.resolve() - const cleanupError = new Error(`retry decision oracle cleanup`) - if ( - headId && - (headCommitStatus === `pending` || headWaitStatus === `pending`) - ) - env.executor.rejectTransaction(headId, cleanupError) - for (const [index, id] of peerIds.entries()) - if (peerStatuses[index] === `pending`) - env.executor.rejectTransaction(id, cleanupError) - await cleanupOfflineOracle( - [ - () => Promise.all(observed), - () => env.executor.dispose(), - () => env.collection.cleanup(), - () => warning.mockRestore(), - ], - hasPrimaryFailure, + firstPeerEntered.promise, + `terminal decision admitted peer`, ) + await turn() + // These public observations distinguish retaining the head from + // removing it, including per-ID settlement and optimistic state. + expect(calls).toEqual(expected.calls) + expect( + (await env.executor.peekOutbox()).map(({ id, retryCount }) => ({ + id, + retryCount, + })), + ).toEqual(expected.outbox) + expect(headCommitStatus).toBe(expected.headStatus) + expect(headWaitStatus).toBe(expected.headStatus) + expect(peerStatuses).toEqual(peerIds.map(() => `pending`)) + expect(env.collection.toArray.map(({ id }) => id).sort()).toEqual( + expected.localRows, + ) + expect(env.serverState.size).toBe(0) + expect(hookCalls).toEqual( + scenario.hook === `absent` || scenario.errorKind === `permanent` + ? [] + : [{ error: providerError, retryCount: 0 }], + ) + if (hookCalls.length > 0) + expect(hookCalls[0]!.error).toBe(providerError) + if (expected.retry) { + const [record] = await env.executor.peekOutbox() + expect(record?.lastError?.message).toBe(providerError.message) + expect(record!.nextAttemptAt).toBeGreaterThan(releasedAt) } - }, - ), - { - ...oracleOptions(retryDecisionOracle, seed), - numRuns: retryDecisionRuns, - examples: retryDecisionExamples, + + releaseFirstPeer.resolve() + if (expected.retry) env.executor.getOnlineDetector().notifyOnline() + await atOracleCheckpoint( + Promise.all(observed), + `retry decision history settled`, + ) + expect(headCommitStatus).toBe( + expected.retry ? `fulfilled` : providerError, + ) + expect(headWaitStatus).toBe(headCommitStatus) + expect(peerStatuses).toEqual(peerIds.map(() => `fulfilled`)) + expect(calls).toEqual( + expected.retry + ? [headId, headId, ...peerIds] + : [headId, ...peerIds], + ) + expect(await env.executor.peekOutbox()).toEqual([]) + expect([...env.serverState.keys()].sort()).toEqual(expected.localRows) + } catch (error) { + hasPrimaryFailure = true + throw error + } finally { + releaseFirst.resolve() + releaseFirstPeer.resolve() + const cleanupError = new Error(`retry decision oracle cleanup`) + if ( + headId && + (headCommitStatus === `pending` || headWaitStatus === `pending`) + ) + env.executor.rejectTransaction(headId, cleanupError) + for (const [index, id] of peerIds.entries()) + if (peerStatuses[index] === `pending`) + env.executor.rejectTransaction(id, cleanupError) + await cleanupOfflineOracle( + [ + () => Promise.all(observed), + () => env.executor.dispose(), + () => env.collection.cleanup(), + () => warning.mockRestore(), + ], + hasPrimaryFailure, + ) + } }, + ), + { + ...oracleLaneOptions(retryDecisionOracle, seed), + examples: retryDecisionExamples, + }, + ) + if (!oracleReplayRequested()) + expect([...reachedCases].sort()).toEqual( + retryDecisionCases + .map(({ errorKind, hook }) => `${errorKind}:${hook}`) + .sort(), ) - if (retryDecisionOracle.path === undefined) - expect([...reachedCases].sort()).toEqual( - retryDecisionCases - .map(({ errorKind, hook }) => `${errorKind}:${hook}`) - .sort(), - ) - }, +} + +oracleReplayTest(it).each([20261008])( + `refines an optional retry decision at the durable FIFO checkpoint (seed %s)`, + checkRetryDecision, +) +oracleReplayTest(it, RETRY_DECISION_PROPERTY).each([undefined])( + `refines an optional retry decision at the durable FIFO checkpoint (seed %s)`, + checkRetryDecision, ) // The no-hook run is the reference for existing retry timing. With the same @@ -2753,13 +2751,17 @@ async function checkRetryRecordFailure(seed: number | undefined) { } }), { - ...oracleOptions(retryRecordOracle, seed), + ...oracleLaneOptions(retryRecordOracle, seed), examples: [[2], [5]], }, ) } -it.each(oracleSeeds(20260916, retryRecordOracle))( +oracleReplayTest(it).each([20260916])( + `keeps admitted transactions pending when a peer's retry record cannot be updated (seed %s)`, + checkRetryRecordFailure, +) +oracleReplayTest(it, RETRY_RECORD_PROPERTY).each([undefined])( `keeps admitted transactions pending when a peer's retry record cannot be updated (seed %s)`, checkRetryRecordFailure, ) @@ -2861,13 +2863,17 @@ async function checkDurableAdmissionFailure(seed: number | undefined) { } }), { - ...oracleOptions(admissionOracle, seed), + ...oracleLaneOptions(admissionOracle, seed), examples: [[0], [1], [2]], }, ) } -it.each(oracleSeeds(20260916, admissionOracle))( +oracleReplayTest(it).each([20260916])( + `rejects only the transaction whose durable admission fails (seed %s)`, + checkDurableAdmissionFailure, +) +oracleReplayTest(it, ADMISSION_PROPERTY).each([undefined])( `rejects only the transaction whose durable admission fails (seed %s)`, checkDurableAdmissionFailure, ) diff --git a/packages/offline-transactions/tsconfig.json b/packages/offline-transactions/tsconfig.json index 648131b1ea..88e1750367 100644 --- a/packages/offline-transactions/tsconfig.json +++ b/packages/offline-transactions/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.json", "compilerOptions": { "outDir": "./dist", - "rootDir": "." + "rootDir": "../.." }, "include": ["src/**/*", "tests/**/*", "*.ts"], "exclude": ["dist", "node_modules"] diff --git a/packages/powersync-db-collection/tests/correctness-oracle.test.ts b/packages/powersync-db-collection/tests/correctness-oracle.test.ts index a974b7b781..a378af3dff 100644 --- a/packages/powersync-db-collection/tests/correctness-oracle.test.ts +++ b/packages/powersync-db-collection/tests/correctness-oracle.test.ts @@ -48,9 +48,12 @@ import { TEST_DATABASE_IMPLEMENTATION } from './test-db-implementation' import type { WatchOnChangeHandler } from '@powersync/common' import type { PowerSyncLogger } from '@powersync/node' -const describePowerSync = TEST_DATABASE_IMPLEMENTATION - ? describe - : describe.skip +// Without a SQLite binding the oracle would silently not run, so fail loudly. +if (TEST_DATABASE_IMPLEMENTATION === undefined) { + throw new Error( + `PowerSync correctness oracle needs a SQLite binding: install better-sqlite3, or run on Node 24+ where node:sqlite is available`, + ) +} type CrudRow = { data: string } @@ -137,7 +140,7 @@ async function cleanupAfterOracle( } } -describePowerSync(`PowerSync correctness oracle`, () => { +describe(`PowerSync correctness oracle`, () => { afterEach(() => vi.restoreAllMocks()) it(`rejects stale full-row patches and undeclared public keys`, () => { diff --git a/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver-oracle.test.ts b/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver-oracle.test.ts index 78d2a099c7..591ce9f3af 100644 --- a/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver-oracle.test.ts +++ b/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver-oracle.test.ts @@ -3,6 +3,12 @@ import { tmpdir } from 'node:os' import { dirname, join } from 'node:path' import { fc } from '@fast-check/vitest' import { afterEach, expect, it } from 'vitest' +import { + oracleRandomParameters, + oracleReplayTest, + oracleRuns, + readOracleRunConfig, +} from '../../db/tests/oracle-config.js' import { OpSQLiteDriver } from '../src/op-sqlite-driver' import { InvalidPersistedCollectionConfigError } from '../../db-sqlite-persistence-core/src' import { runSQLiteDriverContractSuite } from '../../db-sqlite-persistence-core/tests/contracts/sqlite-driver-contract' @@ -133,7 +139,8 @@ async function withColumnarDriver( * adapter. Named empty, reordered, duplicated, missing-key, and swapped-value * wrong answers challenge the checker. Normal execution runs the same alias * properties and budgets in retained fixed-seed and seedless-random lanes. - * Seed, numeric shrink path, and property coordinates select one exact replay. + * TANSTACK_DB_ORACLE_SEED, TANSTACK_DB_ORACLE_PATH, and TANSTACK_DB_ORACLE_PROPERTY + * select one exact replay. * Real-provider refinement: frozen receipts from the published 15.2.7 Native * and Node sources, plus forward receipts from 18.2.1 Native, Node, and browser * sources, prove the shim's accepted envelopes match provider output. @@ -503,127 +510,61 @@ const writeResultFieldNames = [ `lastInsertRowId`, ] as const -const aliasOracleProperties = [ - `write-marker-aliases`, - `result-carrier-equivalence`, - `columnar-sql-aliases`, -] as const - -type AliasOracleProperty = (typeof aliasOracleProperties)[number] +type AliasOracleProperty = + | `op-sqlite.write-marker-aliases` + | `op-sqlite.result-carrier-equivalence` + | `op-sqlite.columnar-sql-aliases` type AliasOracleEnvironment = Record -type AliasOracleConfig = { - runs: number - replaySeed: number | undefined - replayPath: string | undefined - replayProperty: AliasOracleProperty | undefined -} -type AliasOracleCampaign = { - name: `fixed` | `random` | `replay` +type AliasOracleParameters = { numRuns: number - seed: number | undefined - path: string | undefined + seed?: number + path?: string } const FIXED_ALIAS_ORACLE_SEED = 165903 -function isAliasOracleProperty(value: string): value is AliasOracleProperty { - return aliasOracleProperties.some((property) => property === value) -} - -function readAliasOracleConfig( +function readAliasOracleRuns( environment: AliasOracleEnvironment = process.env, -): AliasOracleConfig { - const runsText = environment.TANSTACK_DB_OP_SQLITE_ORACLE_RUNS ?? `50` +): number { + // A run-count control, not a replay coordinate. Its default is the shared + // base count, so TANSTACK_DB_ORACLE_RUNS_MULTIPLIER still scales it. + const runsText = + environment.TANSTACK_DB_OP_SQLITE_ORACLE_RUNS ?? String(oracleRuns(50)) const runs = Number(runsText) if (runsText.trim() === `` || !Number.isSafeInteger(runs) || runs < 1) { throw new Error( `TANSTACK_DB_OP_SQLITE_ORACLE_RUNS must be a positive integer`, ) } - - const seedText = environment.TANSTACK_DB_OP_SQLITE_ORACLE_SEED - const path = environment.TANSTACK_DB_OP_SQLITE_ORACLE_PATH - const propertyText = environment.TANSTACK_DB_OP_SQLITE_ORACLE_PROPERTY - if ( - seedText === undefined && - path === undefined && - propertyText === undefined - ) { - return { - runs, - replaySeed: undefined, - replayPath: undefined, - replayProperty: undefined, - } - } - - if (seedText === undefined) { - throw new Error( - `TANSTACK_DB_OP_SQLITE_ORACLE_PATH and PROPERTY require SEED`, - ) - } - const replaySeed = Number(seedText) - if (seedText.trim() === `` || !Number.isSafeInteger(replaySeed)) { - throw new Error(`TANSTACK_DB_OP_SQLITE_ORACLE_SEED must be an integer`) - } - if (path === undefined || !/^\d+(?::\d+)*$/.test(path)) { - throw new Error( - `TANSTACK_DB_OP_SQLITE_ORACLE_PATH must be a numeric shrink path`, - ) - } - if (propertyText === undefined || !isAliasOracleProperty(propertyText)) { - throw new Error( - `TANSTACK_DB_OP_SQLITE_ORACLE_PROPERTY must name a generated property`, - ) - } - - return { - runs, - replaySeed, - replayPath: path, - replayProperty: propertyText, - } + return runs } -function aliasOracleCampaigns( - property: AliasOracleProperty, - config: AliasOracleConfig, -): ReadonlyArray { - if (config.replayProperty !== undefined) { - if (config.replayProperty !== property) return [] - return [ - { - name: `replay`, - numRuns: 1, - seed: config.replaySeed, - path: config.replayPath, - }, - ] - } +const aliasOracleRuns = readAliasOracleRuns() +// Replay coordinates come from the shared TANSTACK_DB_ORACLE_* variables. +const aliasOracleReplay = readOracleRunConfig() - return [ - { - name: `fixed`, - numRuns: config.runs, - seed: FIXED_ALIAS_ORACLE_SEED, - path: undefined, - }, - { - name: `random`, - numRuns: config.runs, - seed: undefined, - path: undefined, - }, - ] -} - -function aliasOracleParameters(campaign: AliasOracleCampaign) { - return { - numRuns: campaign.numRuns, - ...(campaign.seed === undefined ? {} : { seed: campaign.seed }), - ...(campaign.path === undefined ? {} : { path: campaign.path }), - ...(campaign.name === `replay` ? { endOnFailure: true } : {}), - } +/** + * Registers a property's two campaigns. The fixed campaign pins its seed and + * stands aside on every path replay. The random campaign runs in ordinary and + * seed-only runs, and as the replay of its property, where the shared helper + * returns the recorded path. + */ +function registerAliasOracle( + property: AliasOracleProperty, + title: string, + check: (parameters: AliasOracleParameters) => Promise, +): void { + const randomParameters = oracleRandomParameters( + aliasOracleRuns, + aliasOracleReplay, + property, + ) + oracleReplayTest(it)(`${title} [fixed]`, () => + check({ numRuns: aliasOracleRuns, seed: FIXED_ALIAS_ORACLE_SEED }), + ) + oracleReplayTest(it, property)(`${title} [random]`, () => + check(randomParameters), + ) } const writeResultAliasesArbitrary = fc.uniqueArray( @@ -695,99 +636,88 @@ function expectAliasGrammar( } it(`configures identical fixed and seedless-random alias campaigns`, () => { - const config = readAliasOracleConfig({}) - expect(aliasOracleCampaigns(`write-marker-aliases`, config)).toEqual([ - { - name: `fixed`, - numRuns: 50, - seed: FIXED_ALIAS_ORACLE_SEED, - path: undefined, - }, - { - name: `random`, - numRuns: 50, - seed: undefined, - path: undefined, - }, - ]) + const runs = readAliasOracleRuns({}) + expect(runs).toBe(oracleRuns(50)) + expect( + oracleRandomParameters(runs, undefined, `op-sqlite.write-marker-aliases`), + ).toEqual({ numRuns: runs }) +}) + +it(`reseeds the random alias campaigns from a seed-only replay`, () => { + expect( + oracleRandomParameters( + 50, + readOracleRunConfig({ TANSTACK_DB_ORACLE_SEED: `271828` }), + `op-sqlite.write-marker-aliases`, + ), + ).toEqual({ numRuns: 50, seed: 271828 }) }) it(`selects exactly one alias property and shrink path for replay`, () => { - const config = readAliasOracleConfig({ - TANSTACK_DB_OP_SQLITE_ORACLE_SEED: `271828`, - TANSTACK_DB_OP_SQLITE_ORACLE_PATH: `0:1`, - TANSTACK_DB_OP_SQLITE_ORACLE_PROPERTY: `columnar-sql-aliases`, + const replay = readOracleRunConfig({ + TANSTACK_DB_ORACLE_SEED: `271828`, + TANSTACK_DB_ORACLE_PATH: `0:1`, + TANSTACK_DB_ORACLE_PROPERTY: `op-sqlite.columnar-sql-aliases`, }) - expect(aliasOracleCampaigns(`write-marker-aliases`, config)).toEqual([]) - expect(aliasOracleCampaigns(`result-carrier-equivalence`, config)).toEqual([]) - const [replay] = aliasOracleCampaigns(`columnar-sql-aliases`, config) - expect(replay).toEqual({ - name: `replay`, - numRuns: 1, - seed: 271828, - path: `0:1`, - }) - expect(aliasOracleParameters(replay!)).toEqual({ - numRuns: 1, - seed: 271828, - path: `0:1`, - endOnFailure: true, - }) + expect( + oracleRandomParameters(50, replay, `op-sqlite.write-marker-aliases`), + ).toEqual({ numRuns: 50, seed: 271828 }) + expect( + oracleRandomParameters(50, replay, `op-sqlite.columnar-sql-aliases`), + ).toMatchObject({ numRuns: 1, seed: 271828, path: `0:1` }) }) it.each([ { name: `path without seed`, - environment: { TANSTACK_DB_OP_SQLITE_ORACLE_PATH: `0` }, + environment: { TANSTACK_DB_ORACLE_PATH: `0` }, }, { name: `property without seed`, environment: { - TANSTACK_DB_OP_SQLITE_ORACLE_PROPERTY: `write-marker-aliases`, + TANSTACK_DB_ORACLE_PROPERTY: `op-sqlite.write-marker-aliases`, }, }, - { - name: `seed without path`, - environment: { TANSTACK_DB_OP_SQLITE_ORACLE_SEED: `1` }, - }, { name: `seed and path without property`, environment: { - TANSTACK_DB_OP_SQLITE_ORACLE_SEED: `1`, - TANSTACK_DB_OP_SQLITE_ORACLE_PATH: `0`, + TANSTACK_DB_ORACLE_SEED: `1`, + TANSTACK_DB_ORACLE_PATH: `0`, }, }, { name: `non-integer seed`, environment: { - TANSTACK_DB_OP_SQLITE_ORACLE_SEED: `1.5`, - TANSTACK_DB_OP_SQLITE_ORACLE_PATH: `0`, - TANSTACK_DB_OP_SQLITE_ORACLE_PROPERTY: `write-marker-aliases`, + TANSTACK_DB_ORACLE_SEED: `1.5`, + TANSTACK_DB_ORACLE_PATH: `0`, + TANSTACK_DB_ORACLE_PROPERTY: `op-sqlite.write-marker-aliases`, }, }, - { - name: `non-integer run budget`, - environment: { TANSTACK_DB_OP_SQLITE_ORACLE_RUNS: `1.5` }, - }, { name: `invalid shrink path`, environment: { - TANSTACK_DB_OP_SQLITE_ORACLE_SEED: `1`, - TANSTACK_DB_OP_SQLITE_ORACLE_PATH: `first`, - TANSTACK_DB_OP_SQLITE_ORACLE_PROPERTY: `write-marker-aliases`, + TANSTACK_DB_ORACLE_SEED: `1`, + TANSTACK_DB_ORACLE_PATH: `first`, + TANSTACK_DB_ORACLE_PROPERTY: `op-sqlite.write-marker-aliases`, }, }, { name: `unknown property`, environment: { - TANSTACK_DB_OP_SQLITE_ORACLE_SEED: `1`, - TANSTACK_DB_OP_SQLITE_ORACLE_PATH: `0`, - TANSTACK_DB_OP_SQLITE_ORACLE_PROPERTY: `unknown`, + TANSTACK_DB_ORACLE_SEED: `1`, + TANSTACK_DB_ORACLE_PATH: `0`, + TANSTACK_DB_ORACLE_PROPERTY: `unknown`, }, }, ])(`rejects incomplete alias replay coordinates: $name`, ({ environment }) => { - expect(() => readAliasOracleConfig(environment)).toThrow() + expect(() => readOracleRunConfig(environment)).toThrow() +}) + +it(`rejects a non-integer alias run budget`, () => { + expect(() => + readAliasOracleRuns({ TANSTACK_DB_OP_SQLITE_ORACLE_RUNS: `1.5` }), + ).toThrow(`TANSTACK_DB_OP_SQLITE_ORACLE_RUNS must be a positive integer`) }) it(`controls alias grammar reconstruction, ablation, range, and exclusion`, () => { @@ -795,13 +725,10 @@ it(`controls alias grammar reconstruction, ablation, range, and exclusion`, () = expectAliasGrammar(statementResultAliasesArbitrary, statementResultAliasNames) }) -const aliasOracleConfig = readAliasOracleConfig() - -for (const campaign of aliasOracleCampaigns( - `write-marker-aliases`, - aliasOracleConfig, -)) { - it(`preserves generated single-row write-marker aliases in direct row arrays [${campaign.name}]`, async () => { +registerAliasOracle( + `op-sqlite.write-marker-aliases`, + `preserves generated single-row write-marker aliases in direct row arrays`, + async (parameters) => { await fc.assert( fc.asyncProperty(writeResultAliasesArbitrary, async (aliases) => { const row = Object.fromEntries( @@ -811,22 +738,19 @@ for (const campaign of aliasOracleCampaigns( expectExactAliasRows(actual, [row]) }), { - ...aliasOracleParameters(campaign), - ...(campaign.name === `replay` - ? {} - : { - examples: [[[`rowsAffected`]], [[...writeResultFieldNames]]], - }), + ...parameters, + ...(parameters.path === undefined + ? { examples: [[[`rowsAffected`]], [[...writeResultFieldNames]]] } + : { endOnFailure: true }), }, ) - }) -} + }, +) -for (const campaign of aliasOracleCampaigns( - `result-carrier-equivalence`, - aliasOracleConfig, -)) { - it(`returns the same rows across equivalent documented result carriers [${campaign.name}]`, async () => { +registerAliasOracle( + `op-sqlite.result-carrier-equivalence`, + `returns the same rows across equivalent documented result carriers`, + async (parameters) => { await fc.assert( fc.asyncProperty(statementResultAliasesArbitrary, async (aliases) => { const { expected } = aliasQueryCase(aliases) @@ -872,19 +796,19 @@ for (const campaign of aliasOracleCampaigns( } }), { - ...aliasOracleParameters(campaign), - ...(campaign.name === `replay` - ? {} - : { + ...parameters, + ...(parameters.path === undefined + ? { examples: [ [[`rowsAffected`]], [[`rows`, `rawRows`, `columnNames`, `ordinary_name`]], ], - }), + } + : { endOnFailure: true }), }, ) - }) -} + }, +) function quoteIdentifier(identifier: string): string { return `"${identifier.replaceAll(`"`, `""`)}"` @@ -965,11 +889,10 @@ it(`preserves statement-result field names in direct row arrays`, async () => { ) }) -for (const campaign of aliasOracleCampaigns( - `columnar-sql-aliases`, - aliasOracleConfig, -)) { - it(`preserves generated legal SQL aliases through v14 columnar rows [${campaign.name}]`, async () => { +registerAliasOracle( + `op-sqlite.columnar-sql-aliases`, + `preserves generated legal SQL aliases through v14 columnar rows`, + async (parameters) => { await withColumnarDriver(async ({ driver, queryExecutions }) => { await fc.assert( fc.asyncProperty(statementResultAliasesArbitrary, async (aliases) => { @@ -986,15 +909,15 @@ for (const campaign of aliasOracleCampaigns( expectExactAliasRows(actual, expected) }), { - ...aliasOracleParameters(campaign), - ...(campaign.name === `replay` - ? {} - : { examples: [[[...statementResultFieldNames]]] }), + ...parameters, + ...(parameters.path === undefined + ? { examples: [[[...statementResultFieldNames]]] } + : { endOnFailure: true }), }, ) }) - }) -} + }, +) it(`returns an exact empty row set for an empty columnar SELECT`, async () => { await withColumnarDriver(async ({ driver, queryExecutions }) => { diff --git a/scripts/agent-lint-feedback.mjs b/scripts/agent-lint-feedback.mjs index fb4f22ff2f..63e80186af 100644 --- a/scripts/agent-lint-feedback.mjs +++ b/scripts/agent-lint-feedback.mjs @@ -40,6 +40,11 @@ import { parseArgs } from 'node:util' import tsParser from '@typescript-eslint/parser' import { ESLint } from 'eslint' import oraclePlugin from './eslint-plugin-oracle/index.js' +import { + describeLoweredRunBudget, + loweredRunBudgets, + mergeBase, +} from './check-oracle-run-counts.mjs' const root = fileURLToPath(new URL(`../`, import.meta.url)) @@ -82,24 +87,11 @@ function trackedOracleFiles() { /** * Every file that the branch adds or changes, including uncommitted work. - * Without a main branch to compare against, the branch's own commits cannot - * be found, so this fails instead of linting only the uncommitted work. + * `mergeBase` fails without a main branch, instead of linting only the + * uncommitted work. */ function changedFiles() { - const base = [`origin/main`, `main`] - .map((branch) => { - try { - return git(`merge-base`, `HEAD`, branch) - } catch { - return undefined - } - }) - .find((commit) => commit !== undefined) - if (base === undefined) { - throw new Error( - `Cannot find origin/main or main to compare this branch against. Run \`git fetch origin main\`.`, - ) - } + const base = mergeBase() return repositoryFiles( git(`diff`, `--name-only`, `--diff-filter=ACMR`, base), git(`ls-files`, `--others`, `--exclude-standard`), @@ -262,6 +254,27 @@ function selectFiles(input) { : { scope: `changed`, files: changedFiles() } } +/** + * Lowered run budgets, for a report over the branch's changed files. A + * lowered budget can be deliberate, so it is a report for review, not a lint + * error. + */ +function runBudgetReport(scope) { + if (scope !== `changed`) return `` + return loweredRunBudgets() + .map( + (budget) => + `${budget.file}: ${describeLoweredRunBudget(budget)}. Restore it, or explain why in the pull request.`, + ) + .join(`\n`) +} + const input = readHookInput() const { scope, files } = selectFiles(input) -respond(reportText(await lint(createLinter(), files)), input, scope) +const report = [ + reportText(await lint(createLinter(), files)), + runBudgetReport(scope), +] + .filter(Boolean) + .join(`\n`) +respond(report, input, scope) diff --git a/scripts/check-oracle-run-counts.mjs b/scripts/check-oracle-run-counts.mjs new file mode 100644 index 0000000000..24cdfb2b55 --- /dev/null +++ b/scripts/check-oracle-run-counts.mjs @@ -0,0 +1,204 @@ +#!/usr/bin/env node +/** + * Reports oracle run budgets that a branch lowered. + * + * A generated property checks more histories with a larger budget. Lowering + * one can make a failing property pass without any change that a lint rule can + * see, because the new count is still a valid count. This check compares each + * changed oracle file with its version at the merge-base and reports every + * budget that went down. A lowered budget can be a deliberate choice, so the + * report is a tripwire for the reviewer, not a failure: CI prints it as a + * warning, and the agent hooks return it at the end of a turn. + * + * A budget is keyed by its registered property where the code names one + * (`oraclePropertyOptions(n, 'p')`, `oracleRandomParameters(n, _, 'p')`, or + * `{ property: 'p', defaultRuns: n }`), and otherwise by its fixed seed + * (`{ seed: s, numRuns: oracleRuns(n) }`). A constant bound to a number counts + * as that number. + * + * node scripts/check-oracle-run-counts.mjs [--format=text|github] + */ +import { execFileSync } from 'node:child_process' +import { readFileSync } from 'node:fs' +import { basename, join } from 'node:path' +import { fileURLToPath, pathToFileURL } from 'node:url' +import { parseArgs } from 'node:util' +import tsParser from '@typescript-eslint/parser' + +const root = fileURLToPath(new URL(`../`, import.meta.url)) +function git(...args) { + // Pipe stderr so that a failed attempt in `mergeBase` prints nothing. + return execFileSync(`git`, args, { + cwd: root, + encoding: `utf8`, + stdio: [`ignore`, `pipe`, `pipe`], + }).trim() +} + +/** + * The commit where this branch left main. Without one, the branch's own + * history cannot be found, so this fails instead of comparing too little. + */ +export function mergeBase() { + for (const branch of [`origin/main`, `main`]) { + try { + return git(`merge-base`, `HEAD`, branch) + } catch { + // Try the next branch. + } + } + throw new Error( + `Cannot find origin/main or main to compare this branch against. Run \`git fetch origin main\`.`, + ) +} + +/** The number a node is, or that a constant in the same file is bound to. */ +function numberOf(node, constants) { + if (node?.type === `Literal` && typeof node.value === `number`) { + return node.value + } + if (node?.type === `Identifier`) return constants.get(node.name) + if (node?.type === `CallExpression` && node.callee.name === `oracleRuns`) { + return numberOf(node.arguments[0], constants) + } + return undefined +} + +function stringOf(node) { + if (node?.type === `Literal` && typeof node.value === `string`) { + return node.value + } + if (node?.type === `TemplateLiteral` && node.expressions.length === 0) { + return node.quasis[0].value.cooked + } + return undefined +} + +function walk(node, visit) { + if (node === null || typeof node !== `object`) return + if (typeof node.type === `string`) visit(node) + for (const [key, child] of Object.entries(node)) { + if (key === `parent`) continue + if (Array.isArray(child)) child.forEach((item) => walk(item, visit)) + else if (child && typeof child.type === `string`) walk(child, visit) + } +} + +function propertyValue(object, name) { + return object.properties.find( + (property) => + property.type === `Property` && + !property.computed && + (property.key.name ?? property.key.value) === name, + )?.value +} + +/** Every keyed run budget in one source file. */ +function runBudgets(source, file) { + const ast = tsParser.parse(source, { + range: true, + loc: true, + jsx: file.endsWith(`.tsx`), + }) + const constants = new Map() + walk(ast, (node) => { + if ( + node.type === `VariableDeclaration` && + node.kind === `const` && + node.declarations.length === 1 + ) { + const [declaration] = node.declarations + const value = numberOf(declaration.init, constants) + if (declaration.id.type === `Identifier` && value !== undefined) { + constants.set(declaration.id.name, value) + } + } + }) + const budgets = new Map() + const record = (key, value) => { + if (key !== undefined && value !== undefined) budgets.set(key, value) + } + walk(ast, (node) => { + if (node.type === `CallExpression` && node.callee.type === `Identifier`) { + const [first, second, third] = node.arguments + if (node.callee.name === `oraclePropertyOptions`) { + const property = stringOf(second) + if (property !== undefined) { + record(`property ${property}`, numberOf(first, constants)) + } + } + if (node.callee.name === `oracleRandomParameters`) { + const property = stringOf(third) + if (property !== undefined) { + record(`property ${property}`, numberOf(first, constants)) + } + } + } + if (node.type === `ObjectExpression`) { + const property = stringOf(propertyValue(node, `property`)) + const defaultRuns = numberOf( + propertyValue(node, `defaultRuns`), + constants, + ) + if (property !== undefined) record(`property ${property}`, defaultRuns) + const seed = numberOf(propertyValue(node, `seed`), constants) + const numRuns = numberOf(propertyValue(node, `numRuns`), constants) + if (seed !== undefined) record(`fixed seed ${seed}`, numRuns) + } + }) + return budgets +} + +function changedOracleFiles(base) { + return git(`diff`, `--name-only`, `--diff-filter=M`, base) + .split(`\n`) + .filter((file) => /oracle.*\.tsx?$/.test(basename(file))) +} + +function show(revision, file) { + try { + return git(`show`, `${revision}:${file}`) + } catch { + return undefined + } +} + +/** Every keyed run budget that the branch lowered in a changed oracle file. */ +export function loweredRunBudgets() { + const base = mergeBase() + const lowered = [] + for (const file of changedOracleFiles(base)) { + const before = show(base, file) + if (before === undefined) continue + const after = readFileSync(join(root, file), `utf8`) + const previous = runBudgets(before, file) + const current = runBudgets(after, file) + for (const [key, was] of previous) { + const now = current.get(key) + if (now !== undefined && now < was) lowered.push({ file, key, was, now }) + } + } + return lowered +} + +export function describeLoweredRunBudget({ key, was, now }) { + return `${key}: run budget lowered from ${was} to ${now}` +} + +if (import.meta.url === pathToFileURL(process.argv[1]).href) { + const { values: options } = parseArgs({ + options: { format: { type: `string`, default: `text` } }, + }) + const lowered = loweredRunBudgets() + for (const budget of lowered) { + const message = describeLoweredRunBudget(budget) + if (options.format === `github`) { + console.log( + `::warning file=${budget.file},title=Lowered oracle run budget::${message}`, + ) + } else { + console.log(`${budget.file}: ${message}`) + } + } + process.exitCode = lowered.length > 0 && options.format === `text` ? 1 : 0 +} diff --git a/scripts/eslint-plugin-oracle/no-disabled.test.js b/scripts/eslint-plugin-oracle/no-disabled.test.js index c241383bd3..1f19462fe1 100644 --- a/scripts/eslint-plugin-oracle/no-disabled.test.js +++ b/scripts/eslint-plugin-oracle/no-disabled.test.js @@ -12,6 +12,27 @@ const ruleTester = new RuleTester({ ruleTester.run(`oracle/no-disabled`, noDisabled, { valid: [ + // A selection helper's result used as a registration, with no modifier. + { + code: `import { oracleReplayTest } from './oracle-config.js'\nimport { it as vitestIt } from 'vitest'\nconst fixedCase = oracleReplayTest(vitestIt)\nfixedCase('checks the fixed history', () => check(history))`, + languageOptions: { sourceType: `module` }, + }, + // A constant that is not bound to a test function registers nothing. + { + code: `const helper = makeHelper()\nhelper.skip('a', () => check())`, + languageOptions: { sourceType: `module` }, + }, + // The shared replay variables, and variables that select no replay. + `const seed = process.env.TANSTACK_DB_ORACLE_SEED`, + `const { TANSTACK_DB_ORACLE_PATH, TANSTACK_DB_ORACLE_PROPERTY } = process.env`, + `const statistics = process.env.TANSTACK_DB_ORACLE_STATISTICS`, + `const debug = process.env.RELOAD_ORACLE_DEBUG`, + `const seed = config.TANSTACK_DB_LIKE_ORACLE_SEED`, + // oracle-config implements replay selection, so it reads every variable. + { + code: `const seed = process.env.UTILS_ORACLE_SEED`, + filename: `/repo/packages/db/tests/oracle-config.ts`, + }, // A replay that does not select this property registers an empty test. `it.skip('runs only when its replay property is selected', () => {})`, `it.todo('loads every subset from one generation')`, @@ -28,6 +49,8 @@ ruleTester.run(`oracle/no-disabled`, noDisabled, { }, `fc.assert(property, oraclePropertyOptions(12, 'cleanup-queue.history'))`, `fc.assert(property, { numRuns: oracleRuns(20) })`, + `const BINDING_RUNS = 12\nfc.assert(property, { numRuns: oracleRuns(BINDING_RUNS) })`, + `const fixedSeed = 7\nconst runs = 30\nfc.assert(property, { seed: fixedSeed, numRuns: runs })`, // A replay passes the recorded path, and a witness a fixed seed. `fc.check(property, { seed: failed.seed, path: failed.counterexamplePath, numRuns: 1 })`, `fc.check(property, { seed: 205203, numRuns: 1 })`, @@ -40,6 +63,36 @@ ruleTester.run(`oracle/no-disabled`, noDisabled, { `fc.sample(historyArbitrary, { numRuns: 1 })`, // A known gap is pinned as failing on a fixed seed, not skipped. `it.fails('checks the fixed history', () => fc.assert(property, { seed: 2128, numRuns: oracleRuns(12) }))`, + // The shared replay helpers may select a replay, in either polarity for + // a registration and through the alias helper. + ...[ + `it.skipIf(oracleReplaySkips('cursor.exact-width'))('a', () => check())`, + `if (!oracleReplayRequested()) { it('a', () => check()) }`, + `if (oracleReplayRequested()) { replayOnly() } else { it('a', () => check()) }`, + `const propertyTest = oracleReplayTest(fcTest, 'cursor.exact-width')\npropertyTest.prop([a])('a', check)`, + `describe('s', () => { if (oracleReplaySkips('cursor.exact-width')) return\n it('a', () => check()) })`, + ].map((code) => ({ + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\n${code}`, + languageOptions: { sourceType: `module` }, + })), + // Playwright's test API calls that register nothing. + `test('a', async () => { if (rejected) { expect(test.info().project.name).not.toBe('webkit') } })`, + // A guard that skips a loop item filters data, as \`.filter\` would. + `for (const c of cases) { if (!c.legal) continue\n it(c.name, () => check(c)) }`, + // A loop over a table that a replay helper empties during a replay, and a + // data filter, run every item in ordinary runs. + ...[ + `for (const c of oracleReplayRequested() ? [] : cases) { it(c.name, () => check(c)) }`, + `it.each(cases.filter((c) => c.legal))('%s', (c) => check(c))`, + `const campaigns = oracleReplaySkips('cursor.exact-width') ? [] : cases\nfor (const c of campaigns) { it(c.name, () => check(c)) }`, + `const registration = oracleReplayTest(it)\nregistration('a', () => check())`, + ].map((code) => ({ + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\n${code}`, + languageOptions: { sourceType: `module` }, + })), + // Registrations in a loop or over a table run in every ordinary run. + `for (const campaign of campaigns) { it(campaign.name, () => check(campaign)) }`, + `const fixedRuns = { seed: 1846, numRuns: 40 }`, // A skip on something other than the test context is not a test skip. `it('reads a page', (ctx) => { const cursor = open(); cursor.skip() })`, // An inner variable that shadows the context is not the test context. @@ -47,6 +100,38 @@ ruleTester.run(`oracle/no-disabled`, noDisabled, { `it('reads a page', (_, ctx) => { ctx.skip() })`, ], invalid: [ + // A private variable that selects a replay or a run count. + { + code: `const seed = process.env.TANSTACK_DB_LIKE_ORACLE_SEED`, + errors: [ + { + messageId: `replayVariable`, + data: { name: `TANSTACK_DB_LIKE_ORACLE_SEED` }, + }, + ], + }, + { + code: `fc.assert(property, { path: process.env['BTREE_MAP_ORACLE_PATH'] })`, + errors: [{ messageId: `replayVariable` }], + }, + { + code: `const runs = Number(process.env.TANSTACK_DB_OPFS_CLOSE_ORACLE_RUNS ?? 20)`, + errors: [{ messageId: `replayVariable` }], + }, + { + code: `const { INDEXEDDB_ORACLE_SEED: seed, TANSTACK_DB_ORACLE_PATH } = process.env`, + errors: [ + { + messageId: `replayVariable`, + data: { name: `INDEXEDDB_ORACLE_SEED` }, + }, + ], + }, + { + code: `const property = process.env.UTILS_ORACLE_PROPERTY`, + filename: `/repo/packages/db/tests/utils-oracle.property.test.ts`, + errors: [{ messageId: `replayVariable` }], + }, { code: `it.only('checks the history', () => {})`, errors: [{ messageId: `only` }], @@ -217,6 +302,165 @@ ruleTester.run(`oracle/no-disabled`, noDisabled, { code: `fc.assert(property, { seed: replay.replaySeed, path: undefined, numRuns: 1 })`, errors: [{ messageId: `singleRun` }], }, + { + code: `const propertyTest = replay.replayPath === undefined ? fcTest : fcTest.skip`, + errors: [ + { messageId: `aliasTest`, data: { name: `fcTest` } }, + { messageId: `alias`, data: { name: `skip` } }, + ], + }, + { + code: `const run = it.skip\nrun('a', () => check())`, + errors: [{ messageId: `alias` }], + }, + { + code: `const run = fcTest.skip.prop([a])\nrun('a', check)`, + errors: [{ messageId: `skip` }], + }, + { + code: `if (!replayRequested) { it('a', () => check()) }`, + errors: [{ messageId: `guarded` }], + }, + { + code: `describe('s', () => { if (requested !== id) { return }\n it('a', () => check()) })`, + errors: [{ messageId: `guarded` }], + }, + { + code: `process.env.CI || it('a', () => check())`, + errors: [{ messageId: `guarded` }], + }, + { + code: `const oracleReplayRequested = () => true\nif (!oracleReplayRequested()) { it('a', () => check()) }`, + errors: [{ messageId: `guarded` }], + }, + { + code: `fc.assert(property, { seed: replay.replaySeed, path: replay.replayPath, numRuns: 1 })`, + errors: [{ messageId: `singleRun` }], + }, + { + code: `fc.assert(property, { numRuns: 300 })`, + errors: [{ messageId: `literalRuns` }], + }, + { + code: `fcTest.prop([history], { numRuns: 200 })('checks the history', check)`, + errors: [{ messageId: `literalRuns` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nif (process.env.CI) { oracleReplayTest(it)('a', () => check()) }`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\noracleReplayTest(it).only('a', () => check())`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `only` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\noracleReplayTest(it).skip('a', () => check())`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `skip` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\noracleReplayTest(it).skipIf(process.env.CI)('a', () => check())`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `conditional` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nif (oracleReplayRequested()) { it('law', () => check()) }`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\ndescribe('s', () => { if (!oracleReplayRequested()) return\n it('a', () => check()) })`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\noracleReplayRequested() && it('law', () => check())`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `const BINDING_RUNS = 12\nfc.assert(property, { numRuns: BINDING_RUNS })`, + errors: [{ messageId: `literalRuns` }], + }, + { + code: `const replayRuns = 1\nfc.assert(property, { numRuns: replayRuns })`, + errors: [{ messageId: `singleRun` }], + }, + { + code: `const base = 1\nfc.assert(property, oraclePropertyOptions(base, 'p'))`, + errors: [{ messageId: `singleRun` }], + }, + { + // The rule reports the alias, and it also sees the skip through it. + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nconst t = it\nt.skip('a', () => check())`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `aliasTest` }, { messageId: `skip` }], + }, + // A registration through a selection helper's result is a registration of + // its test, so its modifiers and guards are checked. + { + code: `import { oracleReplayTest } from './oracle-config.js'\nconst fixedCase = oracleReplayTest(it)\nfixedCase.skip('checks the fixed history', () => check(history))`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `skip` }], + }, + { + code: `import { oracleReplayTest } from './oracle-config.js'\nconst fixedCase = oracleReplayTest(it)\nfixedCase.only('checks the fixed history', () => check(history))`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `only` }], + }, + { + code: `import { oracleReplayTest } from './oracle-config.js'\nconst fixedCase = oracleReplayTest(it)\nif (process.env.CI) fixedCase('checks the fixed history', () => check(history))`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleCalibrationTest } from './oracle-config.js'\nconst witness = oracleCalibrationTest(it, 'p', 'fault')\nconst again = witness\nagain.skipIf(process.env.CI)('rejects the mutant', () => check(bad))`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `conditional` }], + }, + // A renamed import of a test function is that test function. + { + code: `import { it as vitestIt } from 'vitest'\nvitestIt.skip('checks a history', () => check(history))`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `skip` }], + }, + { + code: `import { oracleReplayTest } from './oracle-config.js'\nimport { test as fcTestAlias } from '@fast-check/vitest'\nconst campaign = oracleReplayTest(fcTestAlias, 'p')\ncampaign.todo.prop([history])('checks histories', (h) => check(h))`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `skip` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nconst tests = [it, test]\ntests[0]('a', () => check())`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `aliasTest` }, { messageId: `aliasTest` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nfor (const c of process.env.CI ? [] : cases) { it(c.name, () => check(c)) }`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nfor (const c of cases.filter(() => !process.env.CI)) { it(c.name, () => check(c)) }`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nit.each(process.env.CI ? [] : cases)('%s', (c) => check(c))`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nconst campaigns = process.env.CI ? [] : cases\nfor (const c of campaigns) { it(c.name, () => check(c)) }`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, + { + code: `import { oracleReplayRequested, oracleReplaySkips, oracleReplayTest } from './oracle-config.js'\nfor (const c of oracleReplayRequested() ? cases : []) { it(c.name, () => check(c)) }`, + languageOptions: { sourceType: `module` }, + errors: [{ messageId: `guarded` }], + }, { // A comment does not grant an exception. code: ` diff --git a/scripts/eslint-plugin-oracle/rules/no-disabled.js b/scripts/eslint-plugin-oracle/rules/no-disabled.js index 19c0f6313a..15a07bbda5 100644 --- a/scripts/eslint-plugin-oracle/rules/no-disabled.js +++ b/scripts/eslint-plugin-oracle/rules/no-disabled.js @@ -8,33 +8,87 @@ * disables nothing, such as the test that a replay registers when it does * not select its property. * - A test that skips itself through its context, as in `ctx.skip()`. - * - `.skipIf(condition)` or `.runIf(condition)`, unless the call is exactly - * `.skipIf(oracleReplayRequested())`. A replay may stand a test aside, and - * nothing else may switch an oracle test off. The shared helper fixes the - * polarity, so a condition cannot turn the test off in ordinary runs. + * - `.skipIf(condition)` or `.runIf(condition)`, unless the call is + * `.skipIf(oracleReplayRequested())` or `.skipIf(oracleReplaySkips(...))`. + * A replay may stand a test aside, and nothing else may switch an oracle + * test off. The shared helpers in oracle-config are false in an ordinary + * run, so no condition built from them can turn a test off there. + * - A test modifier passed around as a value, as in + * `const run = replaying ? it.skip : it`. The rule cannot see what such an + * alias switches off. `oracleReplayTest(test, property)` selects a replay. + * - A test registered under a condition, or after an early `return`, unless + * the condition is a shared replay helper and the test registers when the + * helper is false, as in `if (!oracleReplayRequested())`. Each helper is + * false in an ordinary run, so such a test runs there. An early `throw` + * fails the run loudly, and `continue` filters a loop's data, so neither + * guards a registration. A registration made through `oracleReplayTest` + * or `oracleCalibrationTest`, directly or through a constant bound to its + * result, or through a renamed import such as `import { it as vitestIt }`, + * is checked as a registration of its test. * - Playwright's runtime `test.skip()` or `test.fixme()` inside a test, * unless its condition is `browserName !== ''`: a control that * only one browser supports. * - A generated property limited to one run that pins no history: a literal * `numRuns` of 1 or 0, or a base of 1 passed to the run helpers. One random - * history is not a generated property. A replay pins its history with a - * `path` beside `numRuns`, and a witness with a seed that is a number - * literal or a constant bound to one. `oraclePropertyOptions` returns one - * run for a path replay, so a campaign needs no one-run option of its own. + * history is not a generated property. A witness pins its history with a + * recorded `path` (a literal, or a failure's `counterexamplePath`) or with + * a seed that is a number literal or a constant bound to one. A replay from + * configuration takes its options from `oraclePropertyOptions`, which + * returns one run for a path replay. + * - A literal run count above 1 that pins no history. It does not scale with + * `TANSTACK_DB_ORACLE_RUNS_MULTIPLIER`, so `oracleRuns` must wrap it. + * - A private replay variable: a `process.env` read whose name ends in + * `_SEED`, `_PATH`, `_PROPERTY`, or `_RUNS`, other than the shared + * `TANSTACK_DB_ORACLE_SEED`, `TANSTACK_DB_ORACLE_PATH`, and + * `TANSTACK_DB_ORACLE_PROPERTY`. The replay manifest, the guarded replay + * runner, and the retired-variable check see only the shared variables. * * The rule has no inline exception. A known gap is pinned with `it.fails` on * a fixed seed, not skipped. * - * The rule cannot see a run count lowered to any value above 1, a `path` or - * seed read from configuration that is absent in ordinary runs, a skip - * reached through an alias such as `const run = it.skip`, or a registration - * inside an `if`. Reviewers still check those. + * The rule cannot see a run count lowered to any value above 1, a run count + * or seed held in a variable that is not a constant bound to a number, a call + * to a local helper that registers tests under a condition, or a table built + * by a function that returns different items in different runs. Reviewers + * still check those. */ const testRoots = new Set([`describe`, `fcTest`, `it`, `suite`, `test`]) const disablingModifiers = new Set([`fixme`, `skip`, `todo`]) const conditionalModifiers = new Set([`runIf`, `skipIf`]) -const replaySelection = `oracleReplayRequested` +const aliasModifiers = new Set([ + ...disablingModifiers, + ...conditionalModifiers, + `only`, +]) +const replayConditions = new Set([`oracleReplayRequested`, `oracleReplaySkips`]) +// The members that keep a call a test registration. Playwright's `test` also +// has calls such as `test.info()` and `test.step()` that register nothing. +const registrationMembers = new Set([ + ...aliasModifiers, + `concurrent`, + `describe`, + `each`, + `fails`, + `parallel`, + `prop`, + `sequential`, + `serial`, +]) +// oracle-config implements replay selection, so its options may carry a +// replay path from configuration. +const replaySelectionModule = + /(^|[/\\])packages[/\\]db[/\\]tests[/\\]oracle-config\.ts$/ +// The shared helpers that return a test registration or its `skip` form. +const selectionHelpers = new Set([`oracleCalibrationTest`, `oracleReplayTest`]) +// The variables that select a replay. Any other variable whose name ends +// in one of these suffixes is a private replay selection. +const sharedReplayVariables = new Set([ + `TANSTACK_DB_ORACLE_PATH`, + `TANSTACK_DB_ORACLE_PROPERTY`, + `TANSTACK_DB_ORACLE_SEED`, +]) +const replayVariableName = /_(PATH|PROPERTY|RUNS|SEED)$/ const runHelpers = new Set([ `oraclePropertyOptions`, `oracleRandomParameters`, @@ -49,13 +103,53 @@ function calleeChain(callee) { if (node.type === `MemberExpression`) { if (node.property.type === `Identifier`) names.unshift(node.property.name) node = node.object + } else if ( + node.type === `CallExpression` && + node.callee.type === `Identifier` && + selectionHelpers.has(node.callee.name) && + node.arguments.length > 0 + ) { + // `oracleReplayTest(it)` returns `it` or `it.skip`, so a registration + // made through it is checked as a registration of `it`. + node = node.arguments[0] } else if (node.type === `CallExpression`) { node = node.callee } else { break } } - return { root: node.type === `Identifier` ? node.name : undefined, names } + return { + root: node.type === `Identifier` ? node.name : undefined, + rootNode: node, + names, + } +} + +/** + * Whether an identifier names a test function: `it`, `test`, and the + * other test roots, a renamed import of one (`import { it as vitestIt }`), + * or a constant bound to one through a selection helper or a member chain + * (`const fixedCase = oracleReplayTest(it)`). A registration made through + * such a constant is checked as a registration of its test. + */ +function isTestRoot(node, sourceCode, seen = new Set()) { + if (node?.type !== `Identifier`) return false + if (testRoots.has(node.name)) return true + const definition = findVariable(sourceCode.getScope(node), node.name)?.defs[0] + if (definition === undefined || seen.has(definition.node)) return false + seen.add(definition.node) + if (definition.type === `ImportBinding`) { + const imported = definition.node.imported + return ( + imported !== undefined && testRoots.has(imported.name ?? imported.value) + ) + } + return ( + definition.type === `Variable` && + definition.parent.kind === `const` && + definition.node.init !== null && + isTestRoot(calleeChain(definition.node.init).rootNode, sourceCode, seen) + ) } function isFunction(node) { @@ -94,30 +188,207 @@ function registrationCondition(callee) { return undefined } -/** Exactly `.skipIf(oracleReplayRequested())`. */ -function isReplaySelection({ name, args }, sourceCode) { - const [condition] = args +/** + * A call to one of the shared oracle-config helpers in `names`. The binding + * must be the import, not a local function with the same name. + */ +function isSharedHelperCall(node, names, sourceCode) { if ( - name !== `skipIf` || - args.length !== 1 || - condition.type !== `CallExpression` || - condition.callee.type !== `Identifier` || - condition.callee.name !== replaySelection || - condition.arguments.length > 0 + node?.type !== `CallExpression` || + node.callee.type !== `Identifier` || + !names.has(node.callee.name) ) { return false } - // The helper must be the shared one, not a local function of that name. - const definition = findVariable( - sourceCode.getScope(condition), - replaySelection, - )?.defs[0] + const definition = findVariable(sourceCode.getScope(node), node.callee.name) + ?.defs[0] return ( definition?.type === `ImportBinding` && /(^|\/)oracle-config(\.js)?$/.test(definition.parent.source.value) ) } +/** Exactly `.skipIf()`, not negated. */ +function isReplaySelection({ name, args }, sourceCode) { + return ( + name === `skipIf` && + args.length === 1 && + isSharedHelperCall(args[0], replayConditions, sourceCode) + ) +} + +/** + * Whether a statement always returns from its function. A `throw` fails the + * run loudly, and `continue` or `break` skips one item of a loop's data, as + * `.filter` would, so neither guards a registration. + */ +function returns(statement) { + if (statement === null || statement === undefined) return false + if (statement.type === `ReturnStatement`) return true + if (statement.type === `BlockStatement`) { + return returns(statement.body.at(-1)) + } + if (statement.type === `IfStatement`) { + return returns(statement.consequent) && returns(statement.alternate) + } + return false +} + +function isEmptyArray(node) { + return node?.type === `ArrayExpression` && node.elements.length === 0 +} + +/** + * The conditions that decide which items a loop's iterable or an `.each()` + * table holds, each with the value under which the items register. A branch + * that is `[]` registers nothing, a `.filter` callback that reads + * `process.env` filters by environment, and a constant is read through to + * its initializer. + */ +function iterableConditions(node, sourceCode, seen = new Set()) { + if (node === null || node === undefined || seen.has(node)) return [] + seen.add(node) + if (node.type === `Identifier`) { + const definition = findVariable(sourceCode.getScope(node), node.name) + ?.defs[0] + return definition?.type === `Variable` && definition.parent.kind === `const` + ? iterableConditions(definition.node.init, sourceCode, seen) + : [] + } + if (node.type === `ConditionalExpression`) { + const conditions = [] + if (isEmptyArray(node.alternate)) { + conditions.push({ test: node.test, registersWhen: true }) + } else if (isEmptyArray(node.consequent)) { + conditions.push({ test: node.test, registersWhen: false }) + } + return [ + ...conditions, + ...iterableConditions(node.consequent, sourceCode, seen), + ...iterableConditions(node.alternate, sourceCode, seen), + ] + } + if (node.type === `LogicalExpression`) { + return [ + { + test: node.left, + registersWhen: node.operator === `&&` ? true : undefined, + }, + ...iterableConditions(node.right, sourceCode, seen), + ] + } + if ( + node.type === `CallExpression` && + node.callee.type === `MemberExpression` && + node.callee.property.name === `filter` + ) { + const [callback] = node.arguments + const readsEnvironment = + callback !== undefined && + /process\.env/.test(sourceCode.getText(callback)) + return [ + ...(readsEnvironment + ? [{ test: callback, registersWhen: undefined }] + : []), + ...iterableConditions(node.callee.object, sourceCode, seen), + ] + } + return [] +} + +/** The table of an `.each(table)` call in a registration's callee chain. */ +function eachTable(callee) { + let node = callee + while (node.type === `CallExpression` || node.type === `MemberExpression`) { + if ( + node.type === `CallExpression` && + node.callee.type === `MemberExpression` && + node.callee.property.name === `each` + ) { + return node.arguments[0] + } + node = node.type === `CallExpression` ? node.callee : node.object + } + return undefined +} + +/** + * The conditions that decide whether `node` runs, each with the value that + * the condition must have for `node` to run: every enclosing `if`, + * conditional, or logical branch, and every earlier `if` in an enclosing + * block that returns. `registersWhen` is undefined when no single value + * decides it, as for `??`. + */ +function guardingConditions(node, sourceCode) { + const conditions = [] + let child = node + for (let parent = node.parent; parent; parent = parent.parent) { + if (parent.type === `ForOfStatement` && child === parent.body) { + conditions.push(...iterableConditions(parent.right, sourceCode)) + } + if ( + (parent.type === `IfStatement` || + parent.type === `ConditionalExpression`) && + child !== parent.test + ) { + conditions.push({ + test: parent.test, + registersWhen: child === parent.consequent, + }) + } else if (parent.type === `LogicalExpression` && child === parent.right) { + conditions.push({ + test: parent.left, + registersWhen: + parent.operator === `&&` + ? true + : parent.operator === `||` + ? false + : undefined, + }) + } else if ( + parent.type === `BlockStatement` || + parent.type === `Program` || + parent.type === `SwitchCase` + ) { + const body = + parent.type === `SwitchCase` ? parent.consequent : parent.body + for (const statement of body.slice(0, body.indexOf(child))) { + if (statement.type === `IfStatement` && returns(statement.consequent)) { + conditions.push({ test: statement.test, registersWhen: false }) + } + } + } + child = parent + } + return conditions +} + +/** + * Whether a guard keeps a registration in every ordinary run. Each shared + * replay helper is false in an ordinary run, so the registration must run + * when the helper is false: under `!helper()`, in the `else` branch of + * `helper()`, or after `if (helper()) return`. A registration that runs only + * when a helper is true would run only during a replay. + */ +function keepsOrdinaryRun({ test, registersWhen }, sourceCode) { + if (registersWhen === undefined) return false + const negated = test.type === `UnaryExpression` && test.operator === `!` + const helper = negated ? test.argument : test + if (!isSharedHelperCall(helper, replayConditions, sourceCode)) return false + return negated ? registersWhen : !registersWhen +} + +/** A `path` that a witness recorded: a literal, or a failure's path. */ +function isRecordedPath(node) { + if (node.type === `TemplateLiteral`) return node.expressions.length === 0 + if (node.type === `Literal`) return typeof node.value === `string` + return ( + node.type === `MemberExpression` && + !node.computed && + node.property.name === `counterexamplePath` + ) +} + /** Playwright's `browserName !== ''`. */ function isBrowserCondition(node) { return ( @@ -169,16 +440,46 @@ function findVariable(scope, name) { return undefined } +function isProcessEnv(node) { + return ( + node?.type === `MemberExpression` && + !node.computed && + node.object.type === `Identifier` && + node.object.name === `process` && + node.property.name === `env` + ) +} + +/** The name in `process.env.NAME` or `process.env['NAME']`. */ +function environmentKey(node) { + if (!isProcessEnv(node.object)) return undefined + if (!node.computed) return node.property.name + return node.property.type === `Literal` && + typeof node.property.value === `string` + ? node.property.value + : undefined +} + +function isPrivateReplayVariable(name) { + return replayVariableName.test(name) && !sharedReplayVariables.has(name) +} + /** A number literal, or a constant bound to one. */ -function isFixedSeed(node, sourceCode) { - if (isNumberLiteral(node)) return true - if (node.type !== `Identifier`) return false +/** The number literal that `node` is, or that a constant `node` names. */ +function constantNumber(node, sourceCode) { + if (isNumberLiteral(node)) return node + if (node?.type !== `Identifier`) return undefined const definition = findVariable(sourceCode.getScope(node), node.name)?.defs[0] - return ( - definition?.type === `Variable` && + return definition?.type === `Variable` && definition.parent.kind === `const` && isNumberLiteral(definition.node.init) - ) + ? definition.node.init + : undefined +} + +/** A number literal, or a constant bound to one. */ +function isFixedSeed(node, sourceCode) { + return constantNumber(node, sourceCode) !== undefined } function propertyName(property) { @@ -197,9 +498,7 @@ function isPinnedOrObservation(object, sourceCode) { const option = (name) => object.properties.find((property) => propertyName(property) === name) const path = option(`path`) - if (path !== undefined && sourceCode.getText(path.value) !== `undefined`) { - return true - } + if (path !== undefined && isRecordedPath(path.value)) return true const seed = option(`seed`) if (seed !== undefined && isFixedSeed(seed.value, sourceCode)) return true const { parent } = object @@ -235,10 +534,47 @@ function isContextSkip(node, sourceCode) { return ( registration.type === `CallExpression` && registration.arguments.includes(fn) && - testRoots.has(calleeChain(registration.callee).root) + isTestRoot(calleeChain(registration.callee).rootNode, sourceCode) ) } +/** + * Whether a test function is passed around as a value: assigned, collected, + * returned, or passed to any call except a shared selection helper. + */ +function isPassedAsValue(node) { + const { parent } = node + switch (parent.type) { + case `VariableDeclarator`: + return parent.init === node + case `AssignmentExpression`: + return parent.right === node + case `ArrayExpression`: + case `ReturnStatement`: + case `SpreadElement`: + return true + case `Property`: + return parent.value === node && parent.parent.type === `ObjectExpression` + case `ConditionalExpression`: + return parent.test !== node + case `LogicalExpression`: + return true + case `ArrowFunctionExpression`: + return parent.body === node + case `CallExpression`: + return ( + parent.arguments.includes(node) && + !( + parent.callee.type === `Identifier` && + selectionHelpers.has(parent.callee.name) && + parent.arguments[0] === node + ) + ) + default: + return false + } +} + /** @type {import('eslint').Rule.RuleModule} */ export default { meta: { @@ -250,8 +586,13 @@ export default { messages: { only: `\`.only\` runs only this test, so every other law in the file stops running. Remove it before committing.`, skip: `\`.{{name}}\` stops this test from enforcing its law. Remove it. To keep a known gap visible, pin a failing history with \`it.fails\` and a fixed seed.`, - conditional: `\`.{{name}}\` can switch this test off in ordinary runs. Only \`.skipIf(oracleReplayRequested())\` may switch an oracle test off, so that a replay can stand it aside. Remove the condition.`, - singleRun: `A generated property with one run that pins no history checks one random history. Pass a base run count to oracleRuns or oraclePropertyOptions. A replay passes \`path\`, and a witness passes a seed that is a number literal or a constant.`, + conditional: `\`.{{name}}\` can switch this test off in ordinary runs. Only \`.skipIf(oracleReplayRequested())\` or \`.skipIf(oracleReplaySkips(property))\` may switch an oracle test off, so that a replay can stand it aside. Remove the condition.`, + alias: `\`.{{name}}\` is passed around as a value, so the rule cannot see what it switches off. Call it directly, or select a replay with \`oracleReplayTest(test, property)\` from oracle-config.`, + aliasTest: `\`{{name}}\` is passed around as a value, so the rule cannot see what it registers or switches off. Call it directly, or select a replay with \`oracleReplayTest({{name}}, property)\` from oracle-config.`, + guarded: `This test registers only under a condition, so it can stop running in ordinary runs. Register it unconditionally. A replay selection uses \`oracleReplayRequested()\` or \`oracleReplaySkips(property)\` from oracle-config, or \`oracleReplayTest(test, property)\`.`, + singleRun: `A generated property with one run that pins no history checks one random history. Pass a base run count to oracleRuns or oraclePropertyOptions, which also selects a replay. A witness passes a recorded \`path\` or a seed that is a number literal or a constant.`, + replayVariable: `\`{{name}}\` selects a replay or a run count outside the shared variables, so the replay manifest, the guarded replay runner, and the retired-variable check cannot see it. Take the options from \`oraclePropertyOptions\` or \`oracleRandomParameters\` in oracle-config, which read TANSTACK_DB_ORACLE_SEED, TANSTACK_DB_ORACLE_PATH, and TANSTACK_DB_ORACLE_PROPERTY, and scale run counts with \`oracleRuns\`.`, + literalRuns: `A literal run count does not scale with TANSTACK_DB_ORACLE_RUNS_MULTIPLIER. Wrap it in oracleRuns, or pass it to oraclePropertyOptions.`, }, }, create(context) { @@ -282,17 +623,30 @@ export default { context.report({ node, messageId: `skip`, data: { name: `skip` } }) return } - const { root, names } = calleeChain(node.callee) + const { root, rootNode, names } = calleeChain(node.callee) if (runHelpers.has(root) && names.length === 0) { if ( - isSingleRunLiteral(node.arguments[0]) && + isSingleRunLiteral(constantNumber(node.arguments[0], sourceCode)) && !isPinnedRunCount(node) ) { context.report({ node: node.arguments[0], messageId: `singleRun` }) } return } - if (!testRoots.has(root)) return + if ( + !isTestRoot(rootNode, sourceCode) || + !names.every((member) => registrationMembers.has(member)) + ) { + return + } + const unguarded = [ + ...guardingConditions(node, sourceCode), + ...iterableConditions(eachTable(node.callee), sourceCode), + ].every((condition) => keepsOrdinaryRun(condition, sourceCode)) + if (!unguarded) { + context.report({ node, messageId: `guarded` }) + return + } if (names.includes(`only`)) { context.report({ node: node.callee, messageId: `only` }) return @@ -313,14 +667,88 @@ export default { context.report({ node, messageId: `skip`, data: { name: disabled } }) } }, + Identifier(node) { + if (!testRoots.has(node.name) || !isPassedAsValue(node)) return + // A local variable or parameter that shares a test function's name + // is not that function. + const definition = findVariable(sourceCode.getScope(node), node.name) + ?.defs[0] + if (definition !== undefined && definition.type !== `ImportBinding`) { + return + } + context.report({ + node, + messageId: `aliasTest`, + data: { name: node.name }, + }) + }, + MemberExpression(node) { + const variable = environmentKey(node) + if ( + variable !== undefined && + isPrivateReplayVariable(variable) && + !replaySelectionModule.test(context.filename) + ) { + context.report({ + node, + messageId: `replayVariable`, + data: { name: variable }, + }) + } + // Report a chain such as `fcTest.skip` or `it.skip.prop` once, at its + // outermost member, when nothing calls it. + if ( + node.parent.type === `MemberExpression` && + node.parent.object === node + ) { + return + } + if ( + node.parent.type === `CallExpression` && + node.parent.callee === node + ) { + return + } + const { rootNode, names } = calleeChain(node) + const name = names.find((member) => aliasModifiers.has(member)) + if (isTestRoot(rootNode, sourceCode) && name !== undefined) { + context.report({ node, messageId: `alias`, data: { name } }) + } + }, + VariableDeclarator(node) { + if ( + node.id.type !== `ObjectPattern` || + !isProcessEnv(node.init) || + replaySelectionModule.test(context.filename) + ) { + return + } + for (const property of node.id.properties) { + const name = + property.type === `Property` ? propertyName(property) : undefined + if (name !== undefined && isPrivateReplayVariable(name)) { + context.report({ + node: property, + messageId: `replayVariable`, + data: { name }, + }) + } + } + }, Property(node) { if ( - propertyName(node) === `numRuns` && - isSingleRunLiteral(node.value) && - !isPinnedRunCount(node.value) + replaySelectionModule.test(context.filename) || + propertyName(node) !== `numRuns` ) { - context.report({ node, messageId: `singleRun` }) + return } + // A constant bound to a number literal counts as that literal. + const count = constantNumber(node.value, sourceCode) + if (count === undefined || isPinnedRunCount(node.value)) return + context.report({ + node, + messageId: isSingleRunLiteral(count) ? `singleRun` : `literalRuns`, + }) }, } },