Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/gc-waits-for-persisting-transactions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/db': patch
---

Garbage collection no longer cleans up a collection while a transaction that touches it is persisting. Cleaning it up dropped the transaction's optimistic rows while its mutation handler still ran, and rejected waits such as Electric's `awaitTxId`, so a write could fail after the screen that made it unmounted. The `gcTime` delay now starts again when the last such transaction completes or fails. An explicit `cleanup()` is unchanged.
2 changes: 1 addition & 1 deletion docs/contributing/oracle-coverage.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion packages/db/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"test": "vitest --run",
"test:dist": "vitest run --config vitest.dist.config.ts",
"test:facade-retention": "node --expose-gc --import tsx tests/facade-retention.probe.ts",
"test:oracles": "vitest --run --coverage.enabled=false tests/reverse-index-oracle.property.test.ts tests/query/compiler/in-evaluator-oracle.property.test.ts tests/cleanup-queue-oracle.property.test.ts tests/comparison-oracle.property.test.ts tests/duration-query-equality-oracle.test.ts tests/temporal-index-order-oracle.test.ts tests/cursor-oracle.property.test.ts tests/index-update-oracle.property.test.ts tests/utils-oracle.property.test.ts tests/change-event-history-oracle.test.ts tests/index-suggestion-oracle.test.ts tests/btree-map-oracle.test.ts tests/paced-mutations-oracle.test.ts tests/db-client-hydration-authority-oracle.test.ts tests/local-storage-order-oracle.test.ts tests/local-storage-peer-oracle.test.ts tests/local-storage-value-fidelity-oracle.property.test.ts tests/collection-mutation-startup-oracle.test.ts tests/collection-cleanup-restart-oracle.test.ts tests/effect-disposal-oracle.test.ts tests/optimistic-transaction-oracle.property.test.ts tests/transaction-ownership-oracle.property.test.ts tests/optimistic-settlement-boundaries-oracle.test.ts tests/optimistic-history-publication-oracle.test.ts tests/optimistic-history-outcomes-oracle.test.ts tests/collection-metadata-publication-oracle.property.test.ts tests/collection-index-value-oracle.test.ts tests/collection-state-retention-oracle.property.test.ts tests/collection-subscription-lifecycle-history-oracle.property.test.ts tests/collection-subscription-lifecycle-oracle.test.ts tests/collection-subscription-reentrancy-oracle.test.ts tests/collection-subscription-lifecycle-publication-oracle.property.test.ts tests/collection-subscription-replay-oracle.property.test.ts tests/d2-source-reconciliation-oracle.property.test.ts tests/live-query-observer-history-oracle.property.test.ts tests/live-query-observer-snapshot-oracle.property.test.ts tests/query/cold-join-reconciliation-oracle.test.ts tests/query/identity-output-shape-oracle.test.ts tests/query/optimizer-semantics-oracle.test.ts tests/query/index-path-collision-oracle.test.ts tests/query/custom-index-order-selection-oracle.test.ts tests/query/mixed-index-order-oracle.test.ts tests/query/includes-collection-oracle.property.test.ts tests/query/includes-functional-projection-oracle.test.ts tests/query/includes-functional-input-boundary.test.ts tests/query/includes-context-transport-oracle.test.ts tests/query/includes-cross-formulation-oracle.property.test.ts tests/query/includes-optimistic-oracle.property.test.ts tests/query/includes-oracle.property.test.ts tests/query/includes-alias-shadowing-oracle.test.ts tests/query/includes-outer-join-route-work-oracle.test.ts tests/query/includes-publication-oracle.test.ts tests/query/includes-query-shape-oracle.test.ts tests/query/subquery-user-value-oracle.test.ts tests/query/includes-temporal-oracle.test.ts tests/query/includes-work-counter-oracle.test.ts tests/query/load-subset-oracle.property.test.ts tests/query/load-subset-replay-refinement-oracle.test.ts tests/query/load-subset-source-readiness-refinement-oracle.test.ts tests/query/load-subset-transaction-refinement-oracle.test.ts tests/query/ordered-source-loader-state-oracle.test.ts tests/query/ordered-demand-retirement.test.ts tests/query/ordered-default-work.test.ts tests/query/ordered-lifecycle-oracle.property.test.ts tests/query/ordered-work-oracle.property.test.ts tests/query/pagination-oracle.property.test.ts tests/query/includes-space-oracle.test.ts tests/query/virtual-row-fields-oracle.test.ts tests/query/where-predicate-publication-oracle.property.test.ts tests/query/join-result-key-oracle.property.test.ts tests/query/pooled-live-query-oracle.property.test.ts tests/live-query-deferred-acquisition-oracle.test.ts tests/flat-change-tracking-oracle.property.test.ts tests/truncate-readiness-oracle.test.ts tests/proxy-detachment-contract-oracle.test.ts tests/production-error-messages-oracle.test.ts tests/query/bucket-facade-rollback-oracle.property.test.ts tests/query/aggregate-select-oracle.property.test.ts",
"test:oracles": "vitest --run --coverage.enabled=false tests/reverse-index-oracle.property.test.ts tests/query/compiler/in-evaluator-oracle.property.test.ts tests/cleanup-queue-oracle.property.test.ts tests/comparison-oracle.property.test.ts tests/duration-query-equality-oracle.test.ts tests/temporal-index-order-oracle.test.ts tests/cursor-oracle.property.test.ts tests/index-update-oracle.property.test.ts tests/utils-oracle.property.test.ts tests/change-event-history-oracle.test.ts tests/index-suggestion-oracle.test.ts tests/btree-map-oracle.test.ts tests/paced-mutations-oracle.test.ts tests/db-client-hydration-authority-oracle.test.ts tests/local-storage-order-oracle.test.ts tests/local-storage-peer-oracle.test.ts tests/local-storage-value-fidelity-oracle.property.test.ts tests/collection-mutation-startup-oracle.test.ts tests/collection-cleanup-restart-oracle.test.ts tests/collection-gc-transaction-oracle.property.test.ts tests/effect-disposal-oracle.test.ts tests/optimistic-transaction-oracle.property.test.ts tests/transaction-ownership-oracle.property.test.ts tests/optimistic-settlement-boundaries-oracle.test.ts tests/optimistic-history-publication-oracle.test.ts tests/optimistic-history-outcomes-oracle.test.ts tests/collection-metadata-publication-oracle.property.test.ts tests/collection-index-value-oracle.test.ts tests/collection-state-retention-oracle.property.test.ts tests/collection-subscription-lifecycle-history-oracle.property.test.ts tests/collection-subscription-lifecycle-oracle.test.ts tests/collection-subscription-reentrancy-oracle.test.ts tests/collection-subscription-lifecycle-publication-oracle.property.test.ts tests/collection-subscription-replay-oracle.property.test.ts tests/d2-source-reconciliation-oracle.property.test.ts tests/live-query-observer-history-oracle.property.test.ts tests/live-query-observer-snapshot-oracle.property.test.ts tests/query/cold-join-reconciliation-oracle.test.ts tests/query/identity-output-shape-oracle.test.ts tests/query/optimizer-semantics-oracle.test.ts tests/query/index-path-collision-oracle.test.ts tests/query/custom-index-order-selection-oracle.test.ts tests/query/mixed-index-order-oracle.test.ts tests/query/includes-collection-oracle.property.test.ts tests/query/includes-functional-projection-oracle.test.ts tests/query/includes-functional-input-boundary.test.ts tests/query/includes-context-transport-oracle.test.ts tests/query/includes-cross-formulation-oracle.property.test.ts tests/query/includes-optimistic-oracle.property.test.ts tests/query/includes-oracle.property.test.ts tests/query/includes-alias-shadowing-oracle.test.ts tests/query/includes-outer-join-route-work-oracle.test.ts tests/query/includes-publication-oracle.test.ts tests/query/includes-query-shape-oracle.test.ts tests/query/subquery-user-value-oracle.test.ts tests/query/includes-temporal-oracle.test.ts tests/query/includes-work-counter-oracle.test.ts tests/query/load-subset-oracle.property.test.ts tests/query/load-subset-replay-refinement-oracle.test.ts tests/query/load-subset-source-readiness-refinement-oracle.test.ts tests/query/load-subset-transaction-refinement-oracle.test.ts tests/query/ordered-source-loader-state-oracle.test.ts tests/query/ordered-demand-retirement.test.ts tests/query/ordered-default-work.test.ts tests/query/ordered-lifecycle-oracle.property.test.ts tests/query/ordered-work-oracle.property.test.ts tests/query/pagination-oracle.property.test.ts tests/query/includes-space-oracle.test.ts tests/query/virtual-row-fields-oracle.test.ts tests/query/where-predicate-publication-oracle.property.test.ts tests/query/join-result-key-oracle.property.test.ts tests/query/pooled-live-query-oracle.property.test.ts tests/live-query-deferred-acquisition-oracle.test.ts tests/flat-change-tracking-oracle.property.test.ts tests/truncate-readiness-oracle.test.ts tests/proxy-detachment-contract-oracle.test.ts tests/production-error-messages-oracle.test.ts tests/query/bucket-facade-rollback-oracle.property.test.ts tests/query/aggregate-select-oracle.property.test.ts",
"bench:nested-includes": "vitest bench tests/query/includes-performance.bench.ts --run",
"test:indexed-db:package": "pnpm --filter @tanstack/db... build && vitest run --config vitest.indexed-db.package.config.ts",
"test:indexed-db:browser": "playwright test --config playwright.indexed-db.config.ts",
Expand Down
14 changes: 13 additions & 1 deletion packages/db/src/collection/lifecycle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -288,11 +288,14 @@ export class CollectionLifecycleManager<
this.startGCTimer(UNSUBSCRIBED_GC_FLOOR_MS)
}

// A persisting transaction holds the Collection: collecting it would drop
// the rows the transaction shows and end the sync run its handler may wait on.
private canGarbageCollect(): boolean {
return (
!this.cleaningUp &&
this.changes.activeSubscribersCount === 0 &&
!this.sync.hasPendingPreload
!this.sync.hasPendingPreload &&
!this.state.hasPersistingTransaction()
)
}

Expand Down Expand Up @@ -324,6 +327,15 @@ export class CollectionLifecycleManager<
)
}

/**
* A transaction that touched this Collection settled. When nothing else
* holds the Collection, its `gcTime` delay starts now.
*/
public startGCTimerAfterSettlement(): void {
if (this.status === `idle` || this.status === `cleaned-up`) return
this.startGCTimer()
}

/**
* Cancel the garbage collection timer
* Called when the collection becomes active again
Expand Down
5 changes: 3 additions & 2 deletions packages/db/src/transactions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -710,12 +710,13 @@ class Transaction<T extends object = Record<string, unknown>> {
touchCollection(): void {
const collections = new Set(this.collections)
for (const mutation of this.mutations) collections.add(mutation.collection)
if (this.state === `completed` || this.state === `failed`)
this.collections.clear()
const settled = this.state === `completed` || this.state === `failed`
if (settled) this.collections.clear()
const errors: Array<unknown> = []
for (const collection of collections) {
try {
collection._state.onTransactionStateChange()
if (settled) collection._lifecycle.startGCTimerAfterSettlement()

// Only call commitPendingTransactions if there are pending sync transactions
if (collection._state.pendingSyncedTransactions.length > 0) {
Expand Down
4 changes: 3 additions & 1 deletion packages/db/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -789,7 +789,9 @@ export interface BaseCollectionConfig<
* when it has no active subscribers. Defaults to 5 minutes (300000ms).
* Sync started without subscribers gets a minimum 50ms grace period.
* Pending preloads retain the collection until they settle. Preloading ready
* data refreshes the retention period. A non-positive or non-finite value
* data refreshes the retention period. A persisting transaction that touches
* the collection also retains it, and the period starts again when the last
* one completes or fails. A non-positive or non-finite value
* disables automatic garbage collection.
*/
gcTime?: number
Expand Down
Loading