From dcaffeee9a868205ecee13414ad877919e625bb9 Mon Sep 17 00:00:00 2001 From: Simon Binder Date: Thu, 13 Aug 2026 14:51:39 +0200 Subject: [PATCH 1/4] Document client-side performance, statement cache --- .../advanced/performance-optimization.mdx | 120 ++++++++++++++++++ debugging/troubleshooting.mdx | 32 +---- docs.json | 3 +- 3 files changed, 124 insertions(+), 31 deletions(-) create mode 100644 client-sdks/advanced/performance-optimization.mdx diff --git a/client-sdks/advanced/performance-optimization.mdx b/client-sdks/advanced/performance-optimization.mdx new file mode 100644 index 00000000..5204859f --- /dev/null +++ b/client-sdks/advanced/performance-optimization.mdx @@ -0,0 +1,120 @@ +--- +title: "SQLite performance optimization" +description: "Optimize query performance in PowerSync client SDKs" +--- + +PowerSync client SDKs use SQLite, and apply reasonable defaults to optimize performance: + +1. For all native targets, [write-ahead logging](https://www.sqlite.org/wal.html) is enabled to support concurrent reads and writes. + PowerSync also dispatches queries to multiple threads for parallelism. + With the JavaScript web SDK, a WAL-like option is [available on Chromium browsers](/client-sdks/reference/javascript-web#2-opfs-based-alternatives). +2. SDKs configure connections for performance: We enable a large page cache of around 50MB and set `pragma synchronous = normal` to avoid frequent fsync operations. + +## Debugging query performance + +Slow queries can have two causes: + +1. The query itself is slow, e.g. because it reads a lot of rows or uses unoptimized joins not backed by an index. + The [Fixing Slow queries](#fixing-slow-queries) section describes how these queries can be optimized. +2. The database is blocked: There can only be a single writer at a time, the amount of concurrent readers is bound by an + option set when opening the database. So an apparently slow query might actually be fast, but ran at an unfortunate + time when the database was busy. + +To understand the problem, having a trace of query runtimes available helps. + +Enabling the `debugMode` flag in the [Web SDK](/client-sdks/reference/javascript-web) logs all SQL queries on the Performance timeline in Chrome's Developer Tools (after recording). This can help identify slow-running queries. +With the Dart SDK, queries are logged to the [Performance View in DevTools](https://docs.flutter.dev/tools/devtools/performance) by default outside of release builds. + + + + + +This includes: + +* PowerSync queries from client code. +* Internal statements from PowerSync, including queries saving sync data, and begin/commit statements. + +This excludes: + +* The time waiting for the global transaction lock, but includes all overhead in worker communication. This means you won't see concurrent queries in most cases. +* Internal statements from `powersync-sqlite-core`, used for Sync Stream bookkeeping and the Sync client. + +Enable this mode when instantiating `PowerSyncDatabase`: + +```js +export const db = new PowerSyncDatabase({ + schema: AppSchema, + database: { + dbFilename: 'powersync.db', + debugMode: true // Defaults to false. To enable in development builds, use + // debugMode: process.env.NODE_ENV !== 'production' + } +}); +``` + +If this reveals it took too long for a read connection to become available, consider increasing the size of the connection pool +with the [maxReaders option](https://pub.dev/documentation/sqlite_async/latest/sqlite_async/SqliteOptions/maxReaders.html) (Dart), +[readWorkerCount](https://powersync-ja.github.io/powersync-js/node-sdk/globals) (Node.JS) or +[additionalReaders](https://powersync-ja.github.io/powersync-js/web-sdk/globals#resolvedwebsqlopenoptions) (Web, only with `WASQLiteVFS.OPFSWriteAheadVFS`). +The Swift and Kotlin SDK always use four read connections, React Native uses five. + +For contention on the write connection, note that the PowerSync client processes changes from the PowerSync Service in a single write transaction +after they've been downloaded. Due to consistency requirements, this transaction cannot be split into multiple steps, and especially for a large initial +sync it can hold a write lock for several seconds. + +## Fixing slow queries + +If a query itself is expensive and takes a long time to run, several options can improve performance. +Some of these require a restructuring of your app's schema. + +1. Increase throttle: For auto-updating watched queries on frequently-changed tables, queries might run very often. + Watched queries should typically be cheap to run, as they otherwise risk blocking SQLite connections for too long. + For more expensive queries that still need to be watched, consider increasing their throttle to run them less often. +2. Use high-performance diffs: As an alternative to regular watched queries, [High Performance Diffs](/client-sdks/high-performance-diffs) + use triggers internally to only report changed rows back to your app. +3. Apply filters: Where possible, filtering on the `id` column of tables to reduce the amount of rows read will make + queries more efficient. When filtering on other columns of large tables, make sure these columns are covered by indexes + declared in your app's schema. +4. Use raw tables. For ease of use, PowerSync tables are actually views over JSON data that extract columns by parsing from JSON + each time a row is accessed. While SQLite has a cache for parsed JSON, this can still be inefficient for queries computing + on columns. [Raw Tables](https://docs.powersync.com/client-sdks/advanced/raw-tables) enable you to use plain SQLite tables, + which are _substantially_ faster to query but require special consideration for migrations. + +### Prepared Statement Cache + +For cheap statements that run frequently, the cost of preparing a statement (making SQLite parse the SQL text and come up with a +query plan based on available indexes) can make up a substantial chunk of the total query runtime. + +In the Dart and JavaScript Web SDKs, it is possible to enable a cache of prepared statements: + + + +```dart database.dart +final db = PowerSyncDatabase( + schema: Schema([...]), + path: 'my_database.db', + sqliteOptions: SqliteOptions( + // Cache up to 64 prepared statements + preparedStatementCacheSize: 64, + ), +); +``` + +```javascript database.ts +const db = new PowerSyncDatabase({ + schema, + database: { + dbFilename: 'my_database.db', + // Cache up to 64 prepared statements + preparedStatementsCache: 64, + }, +}); +``` + + + +In both SDKs, each connection uses its own independent cache and the maximum size applies to those caches. +When the cache is full, the least-recently-used statement is evicted. + +For more information, see the [documentation for Dart](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteOptions/preparedStatementCacheSize.html) +or [for JavaScript](https://powersync-ja.github.io/powersync-js/web-sdk/globals#preparedStatementsCache-1). diff --git a/debugging/troubleshooting.mdx b/debugging/troubleshooting.mdx index dfd9d422..e1b89255 100644 --- a/debugging/troubleshooting.mdx +++ b/debugging/troubleshooting.mdx @@ -334,6 +334,8 @@ These are some common pointers when it comes to diagnosing and understanding per 2. The **initial sync** on a client can take a while in cases where the operations history is large. See [Compacting Buckets](/maintenance-ops/compacting-buckets) to optimize sync performance. 3. You can get big performance gains by using **transactions & batching** as explained in this [blog post](https://www.powersync.com/blog/flutter-database-comparison-sqlite-async-sqflite-objectbox-isar). +Struggling with client-side performance issues? This page describes tools to diagnose performance of the PowerSync service. Optimizing client-side query performance is described in [SQLite performance optimization](/client-sdks/advanced/performance-optimization) + ### Diagnosing Sync Latency If writes are slow to reach the client, there is no single trace that covers the full path. Isolate each stage of the pipeline to find the bottleneck. @@ -369,33 +371,3 @@ Sync & API logs in the [PowerSync Dashboard](https://dashboard.powersync.com/) r * **Upload queue blocking downloads**: by default, uploads are processed before downloads, so a backlogged upload queue delays receiving new data. Buckets and streams at [priority 0](/sync/advanced/prioritized-sync) are not blocked by uploads, but come with the trade-off of potential sync inconsistencies. * **Replication lag on the source database**: high write volume, long-running transactions, bulk updates, or backfills can cause replication to fall behind faster than the service can drain it. See [Replication Lag](/maintenance-ops/replication-lag) for source-specific causes and fixes. * **Too many buckets per user**: incremental sync overhead scales roughly linearly with the number of buckets per user. See [Too Many Buckets](#too-many-buckets-psync_s2305) above. - -### Web: Logging Queries on the Performance Timeline - -Enabling the `debugMode` flag in the [Web SDK](/client-sdks/reference/javascript-web) logs all SQL queries on the Performance timeline in Chrome's Developer Tools (after recording). This can help identify slow-running queries. - - - - -This includes: - -* PowerSync queries from client code. -* Internal statements from PowerSync, including queries saving sync data, and begin/commit statements. - -This excludes: - -* The time waiting for the global transaction lock, but includes all overhead in worker communication. This means you won't see concurrent queries in most cases. -* Internal statements from `powersync-sqlite-core`. - -Enable this mode when instantiating `PowerSyncDatabase`: - -```js -export const db = new PowerSyncDatabase({ - schema: AppSchema, - database: { - dbFilename: 'powersync.db', - debugMode: true // Defaults to false. To enable in development builds, use - // debugMode: process.env.NODE_ENV !== 'production' - } -}); -``` diff --git a/docs.json b/docs.json index 0e97b4da..794be7ee 100644 --- a/docs.json +++ b/docs.json @@ -370,7 +370,8 @@ "client-sdks/advanced/background-syncing", "client-sdks/advanced/checkpoint-requests", "client-sdks/advanced/data-encryption", - "client-sdks/advanced/sqlite-extensions" + "client-sdks/advanced/sqlite-extensions", + "client-sdks/advanced/performance-optimization" ] } ] From e4cf2b05eb2b587857975816ad5a4b70a445f7a7 Mon Sep 17 00:00:00 2001 From: Simon Binder Date: Thu, 13 Aug 2026 15:20:29 +0200 Subject: [PATCH 2/4] AI feedback --- .../advanced/performance-optimization.mdx | 78 +++++++++---------- 1 file changed, 39 insertions(+), 39 deletions(-) diff --git a/client-sdks/advanced/performance-optimization.mdx b/client-sdks/advanced/performance-optimization.mdx index 5204859f..daa7b65a 100644 --- a/client-sdks/advanced/performance-optimization.mdx +++ b/client-sdks/advanced/performance-optimization.mdx @@ -1,5 +1,5 @@ --- -title: "SQLite performance optimization" +title: "SQLite Performance Optimization" description: "Optimize query performance in PowerSync client SDKs" --- @@ -8,25 +8,25 @@ PowerSync client SDKs use SQLite, and apply reasonable defaults to optimize perf 1. For all native targets, [write-ahead logging](https://www.sqlite.org/wal.html) is enabled to support concurrent reads and writes. PowerSync also dispatches queries to multiple threads for parallelism. With the JavaScript web SDK, a WAL-like option is [available on Chromium browsers](/client-sdks/reference/javascript-web#2-opfs-based-alternatives). -2. SDKs configure connections for performance: We enable a large page cache of around 50MB and set `pragma synchronous = normal` to avoid frequent fsync operations. +2. SDKs also configure connections for performance by enabling a large page cache of around 50MB and setting `pragma synchronous = normal` to avoid frequent fsync operations. -## Debugging query performance +## Debugging Query Performance Slow queries can have two causes: 1. The query itself is slow, e.g. because it reads a lot of rows or uses unoptimized joins not backed by an index. - The [Fixing Slow queries](#fixing-slow-queries) section describes how these queries can be optimized. -2. The database is blocked: There can only be a single writer at a time, the amount of concurrent readers is bound by an - option set when opening the database. So an apparently slow query might actually be fast, but ran at an unfortunate - time when the database was busy. + The [Fixing Slow Queries](#fixing-slow-queries) section describes how these queries can be optimized. +2. The database is blocked. There can only be a single writer at a time, and the number of concurrent readers is bound + by an option set when opening the database. So a query that appears slow might actually be fast, having run + at an unfortunate time when the database was busy. To understand the problem, having a trace of query runtimes available helps. Enabling the `debugMode` flag in the [Web SDK](/client-sdks/reference/javascript-web) logs all SQL queries on the Performance timeline in Chrome's Developer Tools (after recording). This can help identify slow-running queries. With the Dart SDK, queries are logged to the [Performance View in DevTools](https://docs.flutter.dev/tools/devtools/performance) by default outside of release builds. - - + + ![Performance timeline showing PowerSync query durations](/images/resources/performance-timeline.png) This includes: @@ -36,8 +36,8 @@ This includes: This excludes: -* The time waiting for the global transaction lock, but includes all overhead in worker communication. This means you won't see concurrent queries in most cases. -* Internal statements from `powersync-sqlite-core`, used for Sync Stream bookkeeping and the Sync client. +* The time spent waiting for the global transaction lock. It still includes all overhead in worker communication, so you generally won't see concurrent queries reflected in the trace. +* Internal statements from `powersync-sqlite-core`, used by the Sync client for Sync Stream bookkeeping. Enable this mode when instantiating `PowerSyncDatabase`: @@ -56,39 +56,50 @@ If this reveals it took too long for a read connection to become available, cons with the [maxReaders option](https://pub.dev/documentation/sqlite_async/latest/sqlite_async/SqliteOptions/maxReaders.html) (Dart), [readWorkerCount](https://powersync-ja.github.io/powersync-js/node-sdk/globals) (Node.JS) or [additionalReaders](https://powersync-ja.github.io/powersync-js/web-sdk/globals#resolvedwebsqlopenoptions) (Web, only with `WASQLiteVFS.OPFSWriteAheadVFS`). -The Swift and Kotlin SDK always use four read connections, React Native uses five. +The Swift and Kotlin SDKs always use four read connections; React Native uses five. For contention on the write connection, note that the PowerSync client processes changes from the PowerSync Service in a single write transaction after they've been downloaded. Due to consistency requirements, this transaction cannot be split into multiple steps, and especially for a large initial sync it can hold a write lock for several seconds. -## Fixing slow queries +## Fixing Slow Queries If a query itself is expensive and takes a long time to run, several options can improve performance. Some of these require a restructuring of your app's schema. -1. Increase throttle: For auto-updating watched queries on frequently-changed tables, queries might run very often. - Watched queries should typically be cheap to run, as they otherwise risk blocking SQLite connections for too long. - For more expensive queries that still need to be watched, consider increasing their throttle to run them less often. -2. Use high-performance diffs: As an alternative to regular watched queries, [High Performance Diffs](/client-sdks/high-performance-diffs) - use triggers internally to only report changed rows back to your app. -3. Apply filters: Where possible, filtering on the `id` column of tables to reduce the amount of rows read will make - queries more efficient. When filtering on other columns of large tables, make sure these columns are covered by indexes - declared in your app's schema. -4. Use raw tables. For ease of use, PowerSync tables are actually views over JSON data that extract columns by parsing from JSON - each time a row is accessed. While SQLite has a cache for parsed JSON, this can still be inefficient for queries computing - on columns. [Raw Tables](https://docs.powersync.com/client-sdks/advanced/raw-tables) enable you to use plain SQLite tables, - which are _substantially_ faster to query but require special consideration for migrations. +1. For auto-updating watched queries on frequently-changed tables, queries might run very often. Watched queries should + typically be cheap to run, as they otherwise risk blocking SQLite connections for too long. For more expensive queries + that still need to be watched, consider increasing their throttle to run them less often. +2. As an alternative to regular watched queries, use [High Performance Diffs](/client-sdks/high-performance-diffs), + which use triggers internally to only report changed rows back to your app. +3. Where possible, filter on the `id` column of tables to reduce the number of rows read, which makes queries more + efficient. When filtering on other columns of large tables, make sure these columns are covered by indexes declared + in your app's schema. +4. PowerSync tables are views over JSON data that extract columns by parsing from JSON each time a row is accessed. + While SQLite has a cache for parsed JSON, this can still be inefficient for queries computing on columns. + [Raw Tables](/client-sdks/advanced/raw-tables) let you use plain SQLite tables instead, which are faster to query + but require special consideration for migrations. ### Prepared Statement Cache For cheap statements that run frequently, the cost of preparing a statement (making SQLite parse the SQL text and come up with a query plan based on available indexes) can make up a substantial chunk of the total query runtime. -In the Dart and JavaScript Web SDKs, it is possible to enable a cache of prepared statements: +In the JavaScript Web and Dart SDKs, it is possible to enable a cache of prepared statements: +```typescript database.ts +const db = new PowerSyncDatabase({ + schema, + database: { + dbFilename: 'my_database.db', + // Cache up to 64 prepared statements + preparedStatementsCache: 64, + }, +}); +``` + ```dart database.dart final db = PowerSyncDatabase( schema: Schema([...]), @@ -100,21 +111,10 @@ final db = PowerSyncDatabase( ); ``` -```javascript database.ts -const db = new PowerSyncDatabase({ - schema, - database: { - dbFilename: 'my_database.db', - // Cache up to 64 prepared statements - preparedStatementsCache: 64, - }, -}); -``` - In both SDKs, each connection uses its own independent cache and the maximum size applies to those caches. When the cache is full, the least-recently-used statement is evicted. -For more information, see the [documentation for Dart](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteOptions/preparedStatementCacheSize.html) -or [for JavaScript](https://powersync-ja.github.io/powersync-js/web-sdk/globals#preparedStatementsCache-1). +For more information, see the [documentation for JavaScript](https://powersync-ja.github.io/powersync-js/web-sdk/globals#preparedStatementsCache-1) +or [for Dart](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteOptions/preparedStatementCacheSize.html). From 3fee5cc9f604cacdb4e4ab589cab9241e77004c7 Mon Sep 17 00:00:00 2001 From: Simon Binder Date: Thu, 13 Aug 2026 15:37:20 +0200 Subject: [PATCH 3/4] More Claude feedback --- client-sdks/advanced/performance-optimization.mdx | 10 +++++----- debugging/troubleshooting.mdx | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/client-sdks/advanced/performance-optimization.mdx b/client-sdks/advanced/performance-optimization.mdx index daa7b65a..88fc5310 100644 --- a/client-sdks/advanced/performance-optimization.mdx +++ b/client-sdks/advanced/performance-optimization.mdx @@ -5,7 +5,7 @@ description: "Optimize query performance in PowerSync client SDKs" PowerSync client SDKs use SQLite, and apply reasonable defaults to optimize performance: -1. For all native targets, [write-ahead logging](https://www.sqlite.org/wal.html) is enabled to support concurrent reads and writes. +1. For all native targets, [write-ahead logging](https://www.sqlite.org/wal.html) (WAL) is enabled to support concurrent reads and writes. PowerSync also dispatches queries to multiple threads for parallelism. With the JavaScript web SDK, a WAL-like option is [available on Chromium browsers](/client-sdks/reference/javascript-web#2-opfs-based-alternatives). 2. SDKs also configure connections for performance by enabling a large page cache of around 50MB and setting `pragma synchronous = normal` to avoid frequent fsync operations. @@ -54,7 +54,7 @@ export const db = new PowerSyncDatabase({ If this reveals it took too long for a read connection to become available, consider increasing the size of the connection pool with the [maxReaders option](https://pub.dev/documentation/sqlite_async/latest/sqlite_async/SqliteOptions/maxReaders.html) (Dart), -[readWorkerCount](https://powersync-ja.github.io/powersync-js/node-sdk/globals) (Node.JS) or +[readWorkerCount](https://powersync-ja.github.io/powersync-js/node-sdk/globals) (Node.js) or [additionalReaders](https://powersync-ja.github.io/powersync-js/web-sdk/globals#resolvedwebsqlopenoptions) (Web, only with `WASQLiteVFS.OPFSWriteAheadVFS`). The Swift and Kotlin SDKs always use four read connections; React Native uses five. @@ -85,11 +85,11 @@ Some of these require a restructuring of your app's schema. For cheap statements that run frequently, the cost of preparing a statement (making SQLite parse the SQL text and come up with a query plan based on available indexes) can make up a substantial chunk of the total query runtime. -In the JavaScript Web and Dart SDKs, it is possible to enable a cache of prepared statements: +In the JavaScript Web and Dart SDKs, enable a cache of prepared statements when opening the database: -```typescript database.ts +```typescript TypeScript const db = new PowerSyncDatabase({ schema, database: { @@ -100,7 +100,7 @@ const db = new PowerSyncDatabase({ }); ``` -```dart database.dart +```dart Dart final db = PowerSyncDatabase( schema: Schema([...]), path: 'my_database.db', diff --git a/debugging/troubleshooting.mdx b/debugging/troubleshooting.mdx index e1b89255..8c3b8c76 100644 --- a/debugging/troubleshooting.mdx +++ b/debugging/troubleshooting.mdx @@ -334,7 +334,7 @@ These are some common pointers when it comes to diagnosing and understanding per 2. The **initial sync** on a client can take a while in cases where the operations history is large. See [Compacting Buckets](/maintenance-ops/compacting-buckets) to optimize sync performance. 3. You can get big performance gains by using **transactions & batching** as explained in this [blog post](https://www.powersync.com/blog/flutter-database-comparison-sqlite-async-sqflite-objectbox-isar). -Struggling with client-side performance issues? This page describes tools to diagnose performance of the PowerSync service. Optimizing client-side query performance is described in [SQLite performance optimization](/client-sdks/advanced/performance-optimization) +Struggling with client-side performance issues? This page describes tools to diagnose performance of the PowerSync service. Optimizing client-side query performance is described in [SQLite Performance Optimization](/client-sdks/advanced/performance-optimization). ### Diagnosing Sync Latency From 923a2020a31e63f693fd0515e9b6bcd666c4ce9a Mon Sep 17 00:00:00 2001 From: benitav Date: Thu, 13 Aug 2026 17:27:29 +0200 Subject: [PATCH 4/4] Apply suggestion from @benitav --- debugging/troubleshooting.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/debugging/troubleshooting.mdx b/debugging/troubleshooting.mdx index 8c3b8c76..64e0a660 100644 --- a/debugging/troubleshooting.mdx +++ b/debugging/troubleshooting.mdx @@ -334,7 +334,7 @@ These are some common pointers when it comes to diagnosing and understanding per 2. The **initial sync** on a client can take a while in cases where the operations history is large. See [Compacting Buckets](/maintenance-ops/compacting-buckets) to optimize sync performance. 3. You can get big performance gains by using **transactions & batching** as explained in this [blog post](https://www.powersync.com/blog/flutter-database-comparison-sqlite-async-sqflite-objectbox-isar). -Struggling with client-side performance issues? This page describes tools to diagnose performance of the PowerSync service. Optimizing client-side query performance is described in [SQLite Performance Optimization](/client-sdks/advanced/performance-optimization). +Struggling with client-side performance issues? This page describes tools to diagnose performance of the PowerSync Service. Optimizing client-side query performance is described in [SQLite Performance Optimization](/client-sdks/advanced/performance-optimization). ### Diagnosing Sync Latency