Add chunked payload generation and network transport

.claude/commands/stress-test-sync-sqlitecloud.md

@@ -114,13 +114,13 @@ Create a bash script at `/tmp/stress_test_concurrent.sh` that:
    - Each iteration does:
      a. **UPDATE** — run `UPDATE <table> SET value = value + 1;` repeated `NUM_UPDATES` times (skip if 0)
      b. **DELETE** — run `DELETE FROM <table> WHERE rowid IN (SELECT rowid FROM <table> ORDER BY RANDOM() LIMIT 10);` repeated `NUM_DELETES` times (skip if 0)
-     c. **Sync using the 3-step send/check/check pattern:**
+     c. **Sync using the 3-step send/receive/receive pattern:**
         1. `SELECT cloudsync_network_send_changes();` — send local changes to the server
-        2. `SELECT cloudsync_network_check_changes();` — ask the server to prepare a payload of remote changes
+        2. `SELECT cloudsync_network_receive_changes();` — ask the server to prepare a payload of remote changes
         3. Sleep 1 second (outside sqlite3, between two separate sqlite3 invocations)
-        4. `SELECT cloudsync_network_check_changes();` — download the prepared payload, if any
+        4. `SELECT cloudsync_network_receive_changes();` — download the prepared payload, if any
    - Each sqlite3 session must: `.load` the extension, call `cloudsync_network_init()`/`cloudsync_network_init_custom()`, `cloudsync_network_set_apikey()`/`cloudsync_network_set_token()` (depending on RLS mode), do the work, call `cloudsync_terminate()`
-   - **Timing**: Log the wall-clock execution time (in milliseconds) for each `cloudsync_network_send_changes()`, `cloudsync_network_check_changes()` call. Define a `now_ms()` helper function at the top of the script and use it before and after each sqlite3 invocation that calls a network function, computing the delta. On **macOS**, `date` does not support `%3N` (nanoseconds) — use `python3 -c 'import time; print(int(time.time()*1000))'` instead. On **Linux**, `date +%s%3N` works fine. The script should detect the platform and define `now_ms()` accordingly. Log lines like: `[DB<N>][iter <I>] send_changes: 123ms`, `[DB<N>][iter <I>] check_changes_1: 45ms`, `[DB<N>][iter <I>] check_changes_2: 67ms`
+   - **Timing**: Log the wall-clock execution time (in milliseconds) for each `cloudsync_network_send_changes()`, `cloudsync_network_receive_changes()` call. Define a `now_ms()` helper function at the top of the script and use it before and after each sqlite3 invocation that calls a network function, computing the delta. On **macOS**, `date` does not support `%3N` (nanoseconds) — use `python3 -c 'import time; print(int(time.time()*1000))'` instead. On **Linux**, `date +%s%3N` works fine. The script should detect the platform and define `now_ms()` accordingly. Log lines like: `[DB<N>][iter <I>] send_changes: 123ms`, `[DB<N>][iter <I>] receive_changes_1: 45ms`, `[DB<N>][iter <I>] receive_changes_2: 67ms`
    - Include labeled output lines like `[DB<N>][iter <I>] updated count=<C>, deleted count=<D>` for grep-ability
 
 3. **Launches all workers in parallel** using `&` and collects PIDs
@@ -138,7 +138,7 @@ Create a bash script at `/tmp/stress_test_concurrent.sh` that:
 - Use `echo -e` to pipe generated SQL (with `\n` separators) into sqlite3
 - During database initialization (Step 1), insert `ROWS` initial rows per database in a single transaction so each DB starts with data to update/delete. Row IDs should be unique across databases: `db<N>_r<J>`
 - User IDs for rows must match the token's userId for RLS to work
-- The sync pattern requires **separate sqlite3 invocations** for send_changes and each check_changes call (with a 1-second sleep between the two check_changes calls), so that timing can be measured per-call from bash
+- The sync pattern requires **separate sqlite3 invocations** for send_changes and each receive_changes call (with a 1-second sleep between the two receive_changes calls), so that timing can be measured per-call from bash
 - **stderr capture**: All sqlite3 invocations must redirect both stdout and stderr to the log file. Use `>> "$LOG" 2>&1` (in this order — stdout redirect first, then stderr to stdout). For timed calls that capture output in a variable, redirect stderr to the log file separately: `RESULT=$(echo -e "$SQL" | $SQLITE3 "$DB" 2>> "$LOG")` and then echo `$RESULT` to the log as well. This ensures "Runtime error" messages from sqlite3 are never lost.
 - Use `/bin/bash` (not `/bin/sh`) for arrays and process management
 
@@ -191,7 +191,7 @@ Report the test results including:
 | Rows per iteration | ROWS |
 | Iterations per database | ITERATIONS |
 | Total CRUD operations | N × ITERATIONS × (UPDATE_ALL + DELETE_FEW) |
-| Total sync operations | N × ITERATIONS × 3 (1 send_changes + 2 check_changes) |
+| Total sync operations | N × ITERATIONS × 3 (1 send_changes + 2 receive_changes) |
 | Duration | start to finish time |
 | Total errors | count |
 | Error types | categorized list |

.claude/commands/test-sync-roundtrip-sqlitecloud-rls.md

@@ -267,16 +267,16 @@ For each of the four SQLite databases, execute the sync operations:
 SELECT cloudsync_network_send_changes();
 
 -- Check for changes from server (repeat with 2-3 second delays)
-SELECT cloudsync_network_check_changes();
--- Repeat check_changes 3-5 times with delays until it returns more than 0 received rows or stabilizes
+SELECT cloudsync_network_receive_changes();
+-- Repeat receive_changes 3-5 times with delays until it returns more than 0 received rows or stabilizes
 ```
 
 **Recommended sync order:**
 1. Sync Database 1A (send + check)
 2. Sync Database 2A (send + check)
 3. Sync Database 1B (send + check)
 4. Sync Database 2B (send + check)
-5. Re-sync all databases (check_changes) to ensure full propagation
+5. Re-sync all databases (receive_changes) to ensure full propagation
 
 ### Step 10: Verify RLS Enforcement
 
@@ -333,7 +333,7 @@ SELECT COUNT(*) FROM <table_name> WHERE id = 'malicious_1';
 **Also verify the malicious row does NOT appear in User 2's databases after syncing:**
 ```sql
 -- In Database 2A or 2B (User 2)
-SELECT cloudsync_network_check_changes();
+SELECT cloudsync_network_receive_changes();
 SELECT * FROM <table_name> WHERE id = 'malicious_1';
 -- Expected: 0 rows (the malicious row should not sync to legitimate User 2 databases)
 ```
@@ -405,7 +405,7 @@ The test FAILS if:
 - Always use the Homebrew sqlite3 binary, NOT `/usr/bin/sqlite3`
 - The cloudsync extension must be built first with `make`
 - SQLiteCloud tables need cleanup before re-running tests
-- `cloudsync_network_check_changes()` may need multiple calls with delays
+- `cloudsync_network_receive_changes()` may need multiple calls with delays
 - Run `SELECT cloudsync_terminate();` on SQLite connections before closing to properly cleanup memory
 - Ensure both test users exist in Supabase auth before running the test
 - The RLS policies must use `auth_userid()` to work with SQLiteCloud token authentication

.claude/commands/test-sync-roundtrip-supabase-rls.md

@@ -275,16 +275,16 @@ For each of the four SQLite databases, execute the sync operations:
 SELECT cloudsync_network_send_changes();
 
 -- Check for changes from server (repeat with 2-3 second delays)
-SELECT cloudsync_network_check_changes();
--- Repeat check_changes 3-5 times with delays until it returns more than 0 received rows or stabilizes
+SELECT cloudsync_network_receive_changes();
+-- Repeat receive_changes 3-5 times with delays until it returns more than 0 received rows or stabilizes
 ```
 
 **Recommended sync order:**
 1. Sync Database 1A (send + check)
 2. Sync Database 2A (send + check)
 3. Sync Database 1B (send + check)
 4. Sync Database 2B (send + check)
-5. Re-sync all databases (check_changes) to ensure full propagation
+5. Re-sync all databases (receive_changes) to ensure full propagation
 
 ### Step 10: Verify RLS Enforcement
 
@@ -341,7 +341,7 @@ SELECT COUNT(*) FROM <table_name> WHERE id = 'malicious_1';
 **Also verify the malicious row does NOT appear in User 2's databases after syncing:**
 ```sql
 -- In Database 2A or 2B (User 2)
-SELECT cloudsync_network_check_changes();
+SELECT cloudsync_network_receive_changes();
 SELECT * FROM <table_name> WHERE id = 'malicious_1';
 -- Expected: 0 rows (the malicious row should not sync to legitimate User 2 databases)
 ```
@@ -414,7 +414,7 @@ The test FAILS if:
 - Always use the Homebrew sqlite3 binary, NOT `/usr/bin/sqlite3`
 - The cloudsync extension must be built first with `make`
 - PostgreSQL tables need cleanup before re-running tests
-- `cloudsync_network_check_changes()` may need multiple calls with delays
+- `cloudsync_network_receive_changes()` may need multiple calls with delays
 - Run `SELECT cloudsync_terminate();` on SQLite connections before closing to properly cleanup memory
 - Ensure both test users exist in Supabase auth before running the test
 - The RLS policies must use `auth.uid()` to work with Supabase JWT authentication

.claude/commands/test-sync-roundtrip-supabase.md

@@ -136,8 +136,8 @@ In the SQLite session:
 SELECT cloudsync_network_send_changes();
 
 -- Check for changes from server (repeat with 2-3 second delays)
-SELECT cloudsync_network_check_changes();
--- Repeat check_changes 3-5 times with delays until it returns more than 0 received rows or stabilizes
+SELECT cloudsync_network_receive_changes();
+-- Repeat receive_changes 3-5 times with delays until it returns more than 0 received rows or stabilizes
 
 -- Verify final data
 SELECT * FROM <table_name>;
@@ -164,7 +164,7 @@ Report the test results including:
 - Always use the Homebrew sqlite3 binary, NOT `/usr/bin/sqlite3`
 - The cloudsync extension must be built first with `make`
 - PostgreSQL tables need cleanup before re-running tests
-- `cloudsync_network_check_changes()` may need multiple calls with delays
+- `cloudsync_network_receive_changes()` may need multiple calls with delays
 - run `SELECT cloudsync_terminate();` on SQLite connections before closing the properly cleanup the memory
 
 ## Permissions

.github/workflows/main.yml

@@ -12,6 +12,13 @@ permissions:
   pages: write
   id-token: write
 
+# Cancel an in-progress run when the same branch is pushed again. This also keeps two
+# runs of the same branch from hitting the shared integration tenants concurrently
+# (the chunked negative-cache test needs an idle tenant — see the chunked env below).
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   build:
     if: ${{ !contains(github.event.head_commit.message, '[auto-update]') }}
@@ -89,6 +96,12 @@ jobs:
       INTEGRATION_TEST_CLOUDSYNC_ADDRESS: ${{ secrets.INTEGRATION_TEST_CLOUDSYNC_ADDRESS }}
       INTEGRATION_TEST_OFFLINE_DATABASE_ID: ${{ secrets.INTEGRATION_TEST_OFFLINE_DATABASE_ID }}
       INTEGRATION_TEST_FAILURE_DATABASE_ID: ${{ secrets.INTEGRATION_TEST_FAILURE_DATABASE_ID }}
+      # Chunked tests need an EXCLUSIVE tenant (the negative-cache test's Phase 2 asserts
+      # an idle tenant, so concurrent writers from other matrix legs cause false failures).
+      # Run them on a single leg only; every other leg gets '' and skips them (treated as
+      # unset by test_chunked_pair_open). The chunked network path is platform-agnostic C,
+      # so per-OS coverage adds little here.
+      INTEGRATION_TEST_CHUNKED_DATABASE_ID: ${{ (matrix.name == 'linux' && matrix.arch == 'x86_64') && secrets.INTEGRATION_TEST_CHUNKED_DATABASE_ID || '' }}
 
     steps:
 
@@ -137,6 +150,7 @@ jobs:
             -e INTEGRATION_TEST_CLOUDSYNC_ADDRESS="${{ env.INTEGRATION_TEST_CLOUDSYNC_ADDRESS }}" \
             -e INTEGRATION_TEST_OFFLINE_DATABASE_ID="${{ env.INTEGRATION_TEST_OFFLINE_DATABASE_ID }}" \
             -e INTEGRATION_TEST_FAILURE_DATABASE_ID="${{ env.INTEGRATION_TEST_FAILURE_DATABASE_ID }}" \
+            -e INTEGRATION_TEST_CHUNKED_DATABASE_ID="${{ env.INTEGRATION_TEST_CHUNKED_DATABASE_ID }}" \
             alpine:latest \
             tail -f /dev/null
           docker exec alpine sh -c "apk update && apk add --no-cache gcc make curl sqlite openssl-dev musl-dev linux-headers"
@@ -212,6 +226,7 @@ jobs:
             export INTEGRATION_TEST_CLOUDSYNC_ADDRESS="$INTEGRATION_TEST_CLOUDSYNC_ADDRESS"
             export INTEGRATION_TEST_OFFLINE_DATABASE_ID="$INTEGRATION_TEST_OFFLINE_DATABASE_ID"
             export INTEGRATION_TEST_FAILURE_DATABASE_ID="$INTEGRATION_TEST_FAILURE_DATABASE_ID"
+            export INTEGRATION_TEST_CHUNKED_DATABASE_ID="$INTEGRATION_TEST_CHUNKED_DATABASE_ID"
             $(make test PLATFORM=$PLATFORM ARCH=$ARCH -n)
           EOF
           echo "::endgroup::"