You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(plugin-state): transact writer mutations before rollout - #4101
The first writer-only production trial of #4038 exposed replayed queue-mutation results. Fifteen distinct Worker invocations reported 112 claims within about 77 seconds, while the confirmed origin had allocated only eight claim generations. A later pg_stat_statements inspection showed eleven actual claim executions. Each successful claim must allocate one generation.
Run sync claim/ACK, garbage-collection claim/ACK, and explicit repair enqueue in separate short read-write transactions. Commit each before KV/R2 I/O. Keep shared Hyperdrive caching, public plugin reads, generation fencing, retry ceilings and provider budgets unchanged.
Select a newly provisioned KV namespace and R2 pointer bucket for the API and all nine production plugin Workers. Nonproduction bindings are unchanged. These fresh resources are required because the writer was paused and source changes during that gap were not queued.
Motivation (AI generated)
These custom PostgreSQL functions mutate leases and progress even though their callers issue SELECT statements. Replaying a cached claim makes the drain process already-settled leases, inflates its progress counters, and slows backfill. The regression tests model an outside-transaction cache and verify that successive leases and GC pages actually advance.
Business Impact (AI generated)
This fixes a blocker found before enabling any KV device reads. All production read flags remain off, the writer is paused, and replicas remain available. No response-time improvement or cost saving is claimed from this patch; those remain production rollout gates.
Test Plan (AI generated)
Backend lint and backend TypeScript checks.
113 unit tests across 12 files, including plugin-state and resource-isolation coverage.
Nine focused source tests cover successive mutation leases, GC cursor progress, explicit repair enqueue, and committing before provider I/O.
Independent review found no blocking defect and independently passed 13 source/isolation tests.
Production API Wrangler dry-run build.
Parsed configuration verifies one matching fresh KV/R2 pair across the API and all nine production plugin environments, with nonproduction unchanged.
CI on 0dc8fb21d6: all 46 latest checks are complete, with 35 successful and 11 skipped. The Cloudflare Workers shard 1/2 retry passed on unchanged code after the original Docker port-bind failure before tests.
After deployment, compare real origin claim executions/generations with actual queue progress.
With old drains stopped, reset namespace-specific GC cursor/lease and backfill bookkeeping, explicitly enqueue full repair, and verify coverage, archives, leases and purges before restarting reads.
Verify the deployed snippet KV-learning guard, usable regional cohort telemetry and real source-change propagation before the 0.1% read canary.
No migration, permissions, primary request-path reads, global query-cache settings or queue retry budgets are changed.
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.
Summary by CodeRabbit
Bug Fixes
Improved the reliability of plugin state synchronization, cleanup progress, and repair requests when operations overlap or cached results are involved.
Documentation
Clarified how state-changing operations are committed and how they relate to storage work.
No actionable comments were generated in the recent review.
Walkthrough
Production plugin-state storage bindings change in the API and plugin configurations. Plugin-state queue mutations now run in database transactions separate from KV/R2 operations, with tests and documentation covering the transaction boundaries.
Changes: Production plugin-state bindings
Layer / File(s)
Summary
Update production storage bindings cloudflare_workers/api/wrangler.jsonc, cloudflare_workers/plugin/wrangler.jsonc
The production API configuration and all nine plugin production environments now use the new PLUGIN_STATE KV namespace ID and dated PLUGIN_STATE_POINTERS R2 bucket name.
Plugin-state mutation transactions
Layer / File(s)
Summary
Run queue mutations in separate transactions supabase/functions/_backend/triggers/plugin_state_sync.ts, tests/plugin-state-source.unit.test.ts, docs/plugin-state-kv.md
A mutate helper runs selected queue RPCs in database transactions. Tests cover sync leases, garbage-collection cursor acknowledgements, repair enqueue ordering, and storage operations outside transactions. Documentation describes this execution model.
Plugin-state queue mutations now commit in short transactions before KV/R2 work, which should stop replayed lease results. Production bindings move to a new KV namespace and pointer bucket. No actionable merge-blocking issue was found; complete the planned backfill verification and canary checks during rollout.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: writer mutations (sync claim/ACK, GC claim/ACK, repair enqueue) are moved into short read-write transactions before KV/R2 I/O. It also names the affected area, plugin-…
Description check
The description follows the template structure. It has a Summary section covering the transaction change, a Test Plan section with checked items and steps, and a checklist. It does not include a Scree…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the main change: writer mutations (sync claim/ACK, GC claim/ACK, repair enqueue) are moved into short read-write transactions before KV/R2 I/O. It also names the affected area, plugin-state, and the rollout context. The title is concise, specific, and follows conventional commit form, so a teammate scanning history would understand the primary change.
Full details: Description check
Explanation
The description follows the template structure. It has a Summary section covering the transaction change, a Test Plan section with checked items and steps, and a checklist. It does not include a Screenshots section, which the template marks as skippable for backend changes. The description also reports manual testing only partly. Some checklist items are not addressed, such as whether documentation was updated (the diff changes docs/plugin-state-kv.md, so the box should be reviewed) and whether E2E coverage is adequate. The Test Plan lists automated checks and unchecked post-deployment items, which is acceptable for a backend change. Overall the description is mostly complete.
Resolution
Optionally tick or address the documentation checklist item, since docs/plugin-state-kv.md changes, and state whether E2E coverage applies. Manual reproduction steps are not given, but the existing test plan covers the backend change.
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary (AI generated)
The first writer-only production trial of #4038 exposed replayed queue-mutation results. Fifteen distinct Worker invocations reported 112 claims within about 77 seconds, while the confirmed origin had allocated only eight claim generations. A later pg_stat_statements inspection showed eleven actual claim executions. Each successful claim must allocate one generation.
Run sync claim/ACK, garbage-collection claim/ACK, and explicit repair enqueue in separate short read-write transactions. Commit each before KV/R2 I/O. Keep shared Hyperdrive caching, public plugin reads, generation fencing, retry ceilings and provider budgets unchanged.
Select a newly provisioned KV namespace and R2 pointer bucket for the API and all nine production plugin Workers. Nonproduction bindings are unchanged. These fresh resources are required because the writer was paused and source changes during that gap were not queued.
Motivation (AI generated)
These custom PostgreSQL functions mutate leases and progress even though their callers issue SELECT statements. Replaying a cached claim makes the drain process already-settled leases, inflates its progress counters, and slows backfill. The regression tests model an outside-transaction cache and verify that successive leases and GC pages actually advance.
Business Impact (AI generated)
This fixes a blocker found before enabling any KV device reads. All production read flags remain off, the writer is paused, and replicas remain available. No response-time improvement or cost saving is claimed from this patch; those remain production rollout gates.
Test Plan (AI generated)
0dc8fb21d6: all 46 latest checks are complete, with 35 successful and 11 skipped. The Cloudflare Workers shard 1/2 retry passed on unchanged code after the original Docker port-bind failure before tests.No migration, permissions, primary request-path reads, global query-cache settings or queue retry budgets are changed.
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.Summary by CodeRabbit