Skip to content

fix(db): complete loadSubset refinement lifecycle - #1780

Open
KyleAMathews wants to merge 334 commits into
codex/loadsubset-total-orderfrom
codex/loadsubset-refinement-oracle
Open

fix(db): complete loadSubset refinement lifecycle#1780
KyleAMathews wants to merge 334 commits into
codex/loadsubset-total-orderfrom
codex/loadsubset-refinement-oracle

Conversation

@KyleAMathews

@KyleAMathews KyleAMathews commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Completes the in-scope loadSubset RFC implementation on top of the outcome,
coverage, and total-order stack. Refinement, replay, publication, readiness, and
adapter cleanup now use one demand/acquisition/applied-result model.

What changed

  • Keep relational state and routing in the existing D2 graph. Custom state is
    limited to async source work and public publication boundaries.
  • Publish reusable coverage only from an exact, authoritative result after its
    rows are visible.
  • Make ordered replay and graph publication atomic. A failed replacement keeps
    the last complete public snapshot and remains retryable.
  • Track logical leases, shared acquisitions, row-key provenance, and physical
    release as separate lifetimes.
  • Use one total order for indexes, top-K, cursors, and refills.
  • Apply the same restart, cancellation, late-settlement, and cleanup laws in
    core, Query DB, Electric, and PowerSync.

Key invariants

  • A successful subset load settles only after its establishing writes are
    visible.
  • Stale, canceled, failed, and released attempts establish no current coverage.
  • Failed replay or publication cannot replace the last complete snapshot.
  • Coverage compaction cannot erase lease, acquisition, row-ownership, or
    release facts.
  • Each physical acquisition is released once after its last logical owner
    leaves; failed release remains retryable.
  • Callback errors keep their exact occurrence and order without leaving partial
    graph state.

Compatibility and limits

  • No new query or pagination syntax.
  • The registry stores provenance and row keys, not copies of row values. Its
    state is bounded by active leases, acquisitions, and owned key slots, and is
    retired on release, reset, truncate, and cleanup.
  • Outcome-free ordered adapters remain supported without inventing coverage.
    Core advances the caller-local boundary until a request makes no progress;
    later growth refreshes from the start. Authoritative continuation,
    replacement, source extent, and reusable coverage still require explicit
    outcomes.
  • When source ordering cannot prove a safe cursor, core loads the filtered
    source region and lets D2 refine it.
  • Electric cannot cancel a snapshot after ShapeStream.requestSnapshot()
    starts because the upstream API exposes neither a request signal nor request
    identity.
  • The generic DBSP operator-law harness remains tracked separately in test(db-ivm): add DBSP incrementalization law properties #1741.

Verification

The integrated gate passed on 303a19ca. Follow-ups through 2143f481 close
the outcome-free adapter boundary found by framework CI and the exact-commit
audit: local settlement never becomes reusable coverage, shrink/regrow gets a
fresh request generation, Effect refill stops at a no-progress fixed point, and
all reset paths retire local settlement.

  • DB: 148 files, 4,691 tests passed, 6 skipped; no type errors.
  • Registered oracle gate: 26 files, 1,518 tests; no type errors.
  • Query DB: 338 tests. Electric: 616 runtime tests / 636 checks. PowerSync: 136
    tests, including the exact prior CI seed.
  • React, Vue, and Svelte shared infinite-query conformance: 33/33 each.
  • Fixed-seed and fresh-random 100× campaigns each passed all 1,518 oracle tests.
  • The four explicitly run package builds passed. ESLint had no errors; Prettier
    and git diff --check passed.
  • The final whole-ledger audit found no dropped defect, false-green oracle,
    adapter law, surviving product mutant, or architecture contradiction. A
    follow-up exact-commit audit found one masked flag-retirement assertion; the
    final test kills that mutant across reset, authoritative settlement, and live
    invalidation.
  • GitHub CI is green on final commit 2143f481.

Main areas changed

  • Core sync, subscription, transaction, rollback, readiness, and publication.
  • Exact subset outcomes, coverage ownership, demand identity, total ordering,
    and window state.
  • D2-backed live-query publication and correlated materialization boundaries.
  • Query DB, Electric, and PowerSync lifecycle conformance.
  • Architecture and oracle documentation.

Stacked on #1778. Part of #1657.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 8359039a-9fb6-4edb-8198-ce0af989bcd4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Aug 26, 2026

Copy link
Copy Markdown
More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1780

@tanstack/browser-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1780

@tanstack/capacitor-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1780

@tanstack/cloudflare-durable-objects-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1780

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1780

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1780

@tanstack/db-sqlite-persistence-core

npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1780

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1780

@tanstack/electron-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1780

@tanstack/expo-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1780

@tanstack/node-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1780

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1780

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1780

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1780

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1780

@tanstack/react-native-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1780

@tanstack/react-router-with-db

npm i https://pkg.pr.new/@tanstack/react-router-with-db@1780

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1780

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1780

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1780

@tanstack/tauri-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1780

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1780

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1780

commit: 2143f48

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Size Change: +9.74 kB (+5.35%) 🔍

Total Size: 192 kB

📦 View Changed
Filename Size Change
packages/db/dist/esm/collection/change-events.js 1.65 kB +175 B (+11.87%) ⚠️
packages/db/dist/esm/collection/changes.js 2.48 kB +537 B (+27.57%) 🚨
packages/db/dist/esm/collection/index.js 4.08 kB +91 B (+2.28%)
packages/db/dist/esm/collection/indexes.js 2.04 kB +48 B (+2.41%)
packages/db/dist/esm/collection/lifecycle.js 2 kB +135 B (+7.25%) 🔍
packages/db/dist/esm/collection/mutations.js 2.62 kB +83 B (+3.27%)
packages/db/dist/esm/collection/state.js 7.47 kB +960 B (+14.75%) ⚠️
packages/db/dist/esm/collection/subscription.js 13 kB +446 B (+3.56%)
packages/db/dist/esm/collection/sync.js 7.63 kB +142 B (+1.9%)
packages/db/dist/esm/indexes/auto-index.js 833 B +4 B (+0.48%)
packages/db/dist/esm/indexes/base-index.js 1 kB +218 B (+27.81%) 🚨
packages/db/dist/esm/indexes/basic-index.js 2.39 kB +227 B (+10.48%) ⚠️
packages/db/dist/esm/indexes/btree-index.js 2.34 kB +41 B (+1.79%)
packages/db/dist/esm/indexes/reverse-index.js 652 B +95 B (+17.06%) ⚠️
packages/db/dist/esm/query/compiler/order-by.js 1.97 kB +160 B (+8.86%) 🔍
packages/db/dist/esm/query/effect.js 5.62 kB +148 B (+2.71%)
packages/db/dist/esm/query/expression-value-context.js 1.49 kB +1.49 kB (new file) 🆕
packages/db/dist/esm/query/ir-stable-identity.js 4.35 kB +188 B (+4.51%) 🔍
packages/db/dist/esm/query/live/bucket-facade-adapter.js 3.73 kB +973 B (+35.25%) 🚨
packages/db/dist/esm/query/live/collection-config-builder.js 7.69 kB +370 B (+5.05%) 🔍
packages/db/dist/esm/query/live/collection-subscriber.js 3.09 kB +451 B (+17.1%) ⚠️
packages/db/dist/esm/query/live/subset-demand-controller.js 1.29 kB +53 B (+4.29%)
packages/db/dist/esm/query/live/utils.js 1.48 kB +94 B (+6.78%) 🔍
packages/db/dist/esm/query/live/window-state.js 2.66 kB +500 B (+23.18%) 🚨
packages/db/dist/esm/query/load-subset-options.js 1.29 kB +207 B (+19.17%) ⚠️
packages/db/dist/esm/query/load-subset-outcome.js 682 B +125 B (+22.44%) 🚨
packages/db/dist/esm/query/predicate-utils.js 3.44 kB +55 B (+1.63%)
packages/db/dist/esm/query/subset-dedupe.js 2.34 kB +759 B (+48.07%) 🚨
packages/db/dist/esm/query/total-order.js 684 B +80 B (+13.25%) ⚠️
packages/db/dist/esm/scheduler.js 1.6 kB +166 B (+11.59%) ⚠️
packages/db/dist/esm/transactions.js 3.52 kB +24 B (+0.69%)
packages/db/dist/esm/utils/array-utils.js 208 B -65 B (-23.81%) 🎉
packages/db/dist/esm/utils/callbacks.js 174 B +174 B (new file) 🆕
packages/db/dist/esm/utils/comparison.js 1.97 kB +579 B (+41.56%) 🚨
ℹ️ View Unchanged
Filename Size
packages/db/dist/esm/client.js 3.66 kB
packages/db/dist/esm/collection-options.js 236 B
packages/db/dist/esm/collection/cleanup-queue.js 810 B
packages/db/dist/esm/collection/events.js 434 B
packages/db/dist/esm/collection/transaction-metadata.js 144 B
packages/db/dist/esm/deferred.js 207 B
packages/db/dist/esm/errors.js 5.39 kB
packages/db/dist/esm/event-emitter.js 1.08 kB
packages/db/dist/esm/index.js 3.81 kB
packages/db/dist/esm/indexes/index-registry.js 820 B
packages/db/dist/esm/live-query-adapter.js 318 B
packages/db/dist/esm/live-query-observer.js 3.65 kB
packages/db/dist/esm/live-query-options.js 702 B
packages/db/dist/esm/live-query-window-controller.js 4.43 kB
packages/db/dist/esm/load-subset-request-provenance.js 412 B
packages/db/dist/esm/local-only.js 975 B
packages/db/dist/esm/local-storage.js 2.18 kB
packages/db/dist/esm/optimistic-action.js 359 B
packages/db/dist/esm/paced-mutations.js 496 B
packages/db/dist/esm/proxy.js 3.75 kB
packages/db/dist/esm/query/builder/functions.js 1.47 kB
packages/db/dist/esm/query/builder/get-query-ir.js 121 B
packages/db/dist/esm/query/builder/index.js 6.6 kB
packages/db/dist/esm/query/builder/ref-proxy.js 1.24 kB
packages/db/dist/esm/query/compiler/evaluators.js 1.9 kB
packages/db/dist/esm/query/compiler/expressions.js 430 B
packages/db/dist/esm/query/compiler/group-by.js 3.69 kB
packages/db/dist/esm/query/compiler/index.js 8.71 kB
packages/db/dist/esm/query/compiler/joins.js 2.95 kB
packages/db/dist/esm/query/compiler/lazy-targets.js 1.11 kB
packages/db/dist/esm/query/compiler/parent-routes.js 319 B
packages/db/dist/esm/query/compiler/route-metadata.js 419 B
packages/db/dist/esm/query/compiler/select.js 1.58 kB
packages/db/dist/esm/query/coverage-registry.js 4.75 kB
packages/db/dist/esm/query/expression-helpers.js 1.43 kB
packages/db/dist/esm/query/ir.js 1.59 kB
packages/db/dist/esm/query/live-query-collection.js 391 B
packages/db/dist/esm/query/live/collection-registry.js 264 B
packages/db/dist/esm/query/live/internal.js 145 B
packages/db/dist/esm/query/live/materialized-pipeline.js 2.47 kB
packages/db/dist/esm/query/optimizer.js 2.92 kB
packages/db/dist/esm/query/query-once.js 359 B
packages/db/dist/esm/query/runtime-reference-identity.js 463 B
packages/db/dist/esm/SortedMap.js 1.3 kB
packages/db/dist/esm/strategies/debounceStrategy.js 247 B
packages/db/dist/esm/strategies/queueStrategy.js 428 B
packages/db/dist/esm/strategies/throttleStrategy.js 246 B
packages/db/dist/esm/utils.js 927 B
packages/db/dist/esm/utils/browser-polyfills.js 304 B
packages/db/dist/esm/utils/btree.js 5.61 kB
packages/db/dist/esm/utils/cursor.js 699 B
packages/db/dist/esm/utils/index-optimization.js 2.39 kB
packages/db/dist/esm/utils/type-guards.js 157 B
packages/db/dist/esm/utils/uuid.js 449 B
packages/db/dist/esm/virtual-props.js 360 B

compressed-size-action::db-package-size

@github-actions

Copy link
Copy Markdown
Contributor

Size Change: 0 B

Total Size: 7.25 kB

ℹ️ View Unchanged
Filename Size
packages/react-db/dist/esm/DbProvider.js 317 B
packages/react-db/dist/esm/HydrationBoundary.js 263 B
packages/react-db/dist/esm/index.js 330 B
packages/react-db/dist/esm/live-query-internals.js 282 B
packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.81 kB
packages/react-db/dist/esm/useLiveQuery.js 2.68 kB
packages/react-db/dist/esm/useLiveQueryEffect.js 355 B
packages/react-db/dist/esm/useLiveSuspenseQuery.js 812 B
packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

@KyleAMathews KyleAMathews changed the title test(db): add loadSubset refinement oracle foundation fix(db): complete loadSubset refinement lifecycle Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant