docs: add RFC draft for Cloud Bigtable storage backend

[annguy3n]

What

This PR introduces the draft design for the Cloud Bigtable storage backend (RFC-0003).

Key design highlights include:

• Lock-then-Read 2PC Flow
• Single-Row Write Guarding
• Transactional Streams
• Sharded TTL Index shadow table
• GSI Validation and Reconciler
• Secure Credentials Path

Why

Cloud Bigtable is GCP's managed wide-column store and serves as the natural target for DynamoDB workloads migrating to GCP. A robust connector allows AWS-to-GCP migration for high-throughput, low-latency applications with zero code changes.

Checklist

• I have read CONTRIBUTING.md

ADR / RFC: #185

---

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

[LeeroyHannigan]

Thank you for this RFC. At a quick glance, this looks solid. Let me do a thorough review and circle back to you early next week (week commencing 6 July).

[LeeroyHannigan]

Thanks for the detailed RFC, @annguy3n this is a strong design and we're happy to accept a Cloud Bigtable backend on these lines. You've clearly thought hard about the DynamoDB semantics: the lock-then-read 2PC, single-row write guarding, stream replay via the txn log, and the sharded TTL index are exactly the right calls given Bigtable's single-row atomicity.

A few DynamoDB-parity points to fold in, split into what must match vs. what's fine to differ if documented:

Must match DynamoDB

• TTL + Streams: the TTL sweeper's deletes must emit DynamoDB-compatible REMOVE stream records (with the TTL principal) when streams are enabled, please make that explicit in the design, since silently dropping them is a breaking behavior gap.

• Key encoding: DynamoDB sorts Number keys numerically (exact to 38 significant digits) and String/Binary keys by byte order. In a byte-lexicographic store that means numeric keys need an order-preserving byte encoding, and numbers must round-trip as exact decimals (not floats). This currently only shows up under Testing but it's a core design detail and should be promoted into the row-key format and GSI index-key sections.

• Transaction rules: TransactWriteItems needs ClientRequestToken idempotency (10-minute window) and must reject two operations on the same item, worth noting alongside the 2PC flow.

• Read isolation: DynamoDB gives read-committed for GetItem/Query/Scan/BatchGetItem and serializable for TransactGetItems. Two things fall out of the 2PC apply window: (1) a row's new value must not be visible until the transaction's commit point, applying mutations before the commit decision is durable would expose a write that can still roll back, which is a dirty read and breaks read-committed; and (2) TransactGetItems must observe the committed set atomically (read through the coordinator / at the commit boundary), otherwise it can see partial state and violate serializable. Please spell out how both are guaranteed.

Fine to differ, just call it out

• GSI consistency: shadow tables make GSIs eventually consistent, which matches DynamoDB, good. Please still reject ConsistentRead=true on a GSI (API parity), and make sure the reconciler keeps GSI reads from returning orphaned/phantom rows during the divergence window.

• Cross-row visibility for standard reads: during the apply window a Query/Scan may see some rows of a committed transaction updated and others not. That's within read-committed (each row still returns a committed value), so it's fine, just document it, and confirm the read-committed and TransactGetItems guarantees above hold.

• Backup / PITR / export: not covered here, fine to defer to a follow-up RFC, but let's note that the effective PITR/export window will differ from DynamoDB's 35 days.

Overall: accepted in principle, let's tighten the TTL-stream path, key-encoding, and the read-isolation guarantees in the doc, then move to implementation behind the opt-in feature. Really nice write-up.

[annguy3n]

@LeeroyHannigan Thanks for the feedback, I've updated the RFC to address them. Let me know if you have any other questions/comments.

[LeeroyHannigan]

Hi @annguy3n ,

Thanks for the update to the RFC, please find below my findings

Blocking

B1. Internal phase-numbering is self-contradictory. The 2PC mermaid in 2.B defines exactly three labeled phases (Phase 1 Open, Phase 2 Commit, Phase 3 Clean up), with intent placement and data apply as unlabeled steps. But the prose references non-existent/contradictory phases: 2.C says intents are written in "Phase 3" (the diagram's Phase 3 is Clean up) and data is applied "only in Phase 7"; 3 says log enrichment in "Phase 2" and stream records written in "Phase 5"; the recovery sweeper (2.E) and streams (3.4) roll forward against these. There is no Phase 4–7 anywhere. This must be reconciled to a single consistent phase list before implementation, because the commit/visibility ordering (see B2) hinges on which phase is the durable commit point.

B2. TransactGetItems serializable guarantee is asserted but the mechanism is missing. 2.C claims TransactGetItems "coordinates with __extenddb_txn_log__ to observe the snapshot at the transaction commit boundary." No snapshot/MVCC mechanism is specified. The design keeps a single committed value per row in family d (mutations applied post-commit), and Bigtable cell timestamps are per-row, not a global read timestamp, so there is no described way to obtain a cross-row atomic snapshot at a commit boundary. As written this reads as read-committed, not serializable. Either specify the mechanism (e.g. read-through-log to reconstruct pre/post images, or a commit-timestamp read) or downgrade the claimed guarantee.

Non-blocking

N1. Commit-point vs client-ack visibility gap (needs to be pinned down). 2.C states data in d is applied only after COMMITTED is durable, and the sweeper rolls forward on crash. It never states when success is returned to the client. If the coordinator acks at the durable-COMMITTED point but applies to d afterward, a strongly-consistent GetItem/Query issued between ack and apply returns the pre-transaction (still-committed) value. DynamoDB reflects a succeeded TransactWriteItems in a subsequent strongly-consistent read. Read-committed per-row is preserved, but read-your-writes after a 200 is not, unless the client ack is deferred until apply completes. State the ack point explicitly.

N2. GSI row-key delimiter contradicts the string-key encoding rule. 1.B first says String/Binary keys use "null-byte / length-prefixed terminators to preserve lexicographical sort order," then defines the GSI shadow row key as [pk_val]\0[sk_val]\0[base_pk_val]\0[base_sk_val] — bare \0 separators. DynamoDB String/Binary values can contain 0x00 bytes; bare-\0 delimiting breaks both uniqueness and sort parity when a value embeds a null, and is inconsistent with the length-prefixed scheme stated one paragraph earlier. Use the same length-prefixed/order-preserving composite encoding for the GSI key.

N3. Trait name is stale vs main. 5.B says the reconciler is "spawned via RuntimeHooks." The actual trait is ServerRuntimeHooks with async fn spawn_workers(&self, ctx: &WorkerContext) at crates/storage/src/hooks.rs:47,58; postgres/sqlite/mongodb all implement it (crates/storage-postgres/src/lib.rs:376, crates/storage-sqlite/src/hooks.rs, crates/storage-mongodb/src/lib.rs:50). Align the name so the RFC points at the real extension surface (TTL/GSI/recovery workers should all register through spawn_workers).

Notes (positive / low-risk)

• Config shape is consistent with repo convention: [storage] backend = "..." selects the backend and backend tables are [storage.<name>] (crates/config/src/lib.rs:186), so [storage.bigtable] with pool_size = 20 matches the existing postgres default. Good.
• The design is docs-only and adds no code; it does not touch or break any current subsystem (backends on main: postgres, sqlite, mongodb; no bigtable dir).
• Core-level operations the RFC leans on already exist, so the Bigtable work is a backend implementation of established contracts, not new surface: TransactWriteItemsInput/ClientRequestToken (crates/core/src/types/transaction.rs:83-89), IdempotentParameterMismatch (crates/core/src/error/mod.rs:37), transact_write_helpers.rs.

Thanks again, looking forward to seeing your implementation of Bigtable here.