Skip to content

fix: honour TableThroughputMode and emit TableThroughputModeSummary - #320

Open
yesyayen wants to merge 2 commits into
ExtendDB:mainfrom
yesyayen:fix/table-throughput-mode
Open

fix: honour TableThroughputMode and emit TableThroughputModeSummary#320
yesyayen wants to merge 2 commits into
ExtendDB:mainfrom
yesyayen:fix/table-throughput-mode

Conversation

@yesyayen

@yesyayen yesyayen commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

What

Amazon DynamoDB accepts TableThroughputMode as a request member on CreateTable and UpdateTable and emits TableThroughputModeSummary in table descriptions. ExtendDB ignored the request member (so a CreateTable carrying only TableThroughputMode: PAY_PER_REQUEST fell through to the PROVISIONED default and failed on missing throughput) and never emitted the response member. Both members are absent from the public SDK models (checked at aws-sdk-dynamodb 1.119.0 and botocore 1.33.13), but the service honours them; this was measured directly against Amazon DynamoDB with raw signed requests (2026-08-21).

Request side, implemented exactly as measured:

  • TableThroughputMode is a fallback alias of BillingMode: when both members are present, BillingMode decides and the other member is ignored. A conflict is not refused. Resolution happens once in extenddb-core (resolve_table_throughput_mode(), which consumes the member), so all existing billing-mode validation and both storage backends run unchanged and every downstream message phrases the mode as BillingMode, matching the service.
  • Enum validation is per member and pre-semantic: an invalid value fails at tableThroughputMode even when a valid BillingMode would win, before required-member validation and before the table lookup on UpdateTable. Multiple invalid members join under N validation errors detected: with billingMode first. UpdateTable gains this pre-lookup enum validation for both members, expressed through the same typed enum-field specs the other handlers use.

Response side: TableDescription gains TableThroughputModeSummary, an exact mirror of BillingModeSummary (present if and only if the sibling is present, same mode, identical LastUpdateToPayPerRequestDateTime). The mirror is derived From<&BillingModeSummary> and populated centrally in the engine on every path that emits a description (CreateTable, DescribeTable, UpdateTable, DeleteTable, RestoreTableFromBackup), so the two members cannot disagree and every backend inherits it; backends only gained a mechanical None field.

This reverses a deliberate earlier decision: two unit tests asserted the member must be ignored because it is not in the SDK model. The original decision was sound and the SDK model lagged the service. The tests are inverted with the measurement dated in the comment so the member is not re-removed on SDK-model grounds.

Behavior: today vs Amazon DynamoDB

Captured from Amazon DynamoDB via raw SigV4-signed JSON (the SDKs cannot express these requests):

Scenario ExtendDB (before) Amazon DynamoDB
CreateTable with only TableThroughputMode: PAY_PER_REQUEST ValidationException: No provisioned throughput specified for the table 200, table is PAY_PER_REQUEST
CreateTable with BillingMode: PROVISIONED + throughput + TableThroughputMode: PAY_PER_REQUEST 200, PROVISIONED (member ignored) 200, PROVISIONED (BillingMode wins; conflict not refused)
CreateTable with TableThroughputMode: PROVISIONED, no throughput ValidationException (defaulted-PROVISIONED path) ValidationException phrased as BillingMode: ReadCapacityUnits and WriteCapacityUnits must both be specified when BillingMode is PROVISIONED
CreateTable with TableThroughputMode: BOGUS member ignored, table created or unrelated error ValidationException: 1 validation error detected: Value 'BOGUS' at 'tableThroughputMode' failed to satisfy constraint: Member must satisfy enum value set: [PROVISIONED, PAY_PER_REQUEST]
CreateTable with both members invalid member ignored, billingMode error only 2 validation errors detected: ... 'billingMode' ...; ... 'tableThroughputMode' ...
UpdateTable switching mode via TableThroughputMode alone ValidationException: At least one of ... must be specified 200, mode switches
UpdateTable with TableThroughputMode: BOGUS on a missing table deserialization error enum ValidationException at tableThroughputMode (fires before the table lookup)
DescribeTable on a PAY_PER_REQUEST table BillingModeSummary only BillingModeSummary and TableThroughputModeSummary, same mode, identical timestamp
Any table description on a table created PROVISIONED neither summary neither summary (mirror invariant holds in the absent case too)

Why

Fix conformance gaps: an accept-direction divergence (the service accepts and honours a request shape ExtendDB refuses) plus a missing response member emitted by the service on every table description.

Testing done

The branch was rebased onto current main after the vector-search backend PRs and the enum-violation-shape work merged; the enum validation was re-expressed through the typed field specs introduced there, with the original wire captures as the oracle, and all gates below were re-run at the rebased tip.

  • New tests/test_table_throughput_mode.py: 18 dual-target tests covering the request matrix
  • At the rebased tip: 18/18 against a fixed PostgreSQL-backend instance, 18/18 against a fixed SQLite-backend instance, and the full suite against Amazon DynamoDB.
  • Regression sweep: table-lifecycle suites against the fixed instance show the same failure set as an unfixed build of the same base commit running side by side (environment-dependent suites only), so the change introduces zero regressions.

Checklist

  • I have read CONTRIBUTING.md
  • All tests pass (cargo test --workspace)
  • Code is formatted (cargo fmt --check)
  • Clippy is clean (cargo clippy -- -W clippy::pedantic)
  • I have added or updated tests for new functionality
  • I have updated documentation if behavior changed
  • Breaking changes are noted below (if any)
  • If this changes the wire protocol, Storage trait, auth model, on-disk
    format, or public CLI surface, an RFC has been accepted or is linked
    below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a (additive wire members matching Amazon DynamoDB; no trait, auth, on-disk, or CLI change)


By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

Dual-target coverage for the TableThroughputMode request member on
CreateTable and UpdateTable (fallback alias of BillingMode, per-member
enum validation, BillingMode precedence when both are present) and for
TableThroughputModeSummary mirroring BillingModeSummary in table
descriptions. Measured against Amazon DynamoDB; passes there, fails on
ExtendDB before the fix.
Amazon DynamoDB accepts TableThroughputMode on CreateTable and
UpdateTable as a fallback alias of BillingMode: BillingMode wins when
both members are present (a conflict is not refused), enum validation
is per member and pre-semantic, and every downstream message phrases
the mode as BillingMode. Measured 2026-08-21; the earlier decision to
ignore the member was based on the SDK model, which lags the service.

Table descriptions now also carry TableThroughputModeSummary, an exact
mirror of BillingModeSummary (same mode, identical timestamp, emitted
together or not at all), populated centrally in the engine so both
members cannot disagree and every backend inherits it.
@yesyayen
yesyayen marked this pull request as ready for review August 27, 2026 16:40
return resp.json()


def _wait_active_raw(name: str, timeout: float = 120.0) -> None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should-fix The 120s budget is too short when this suite runs against real DynamoDB: billing mode switches can take several minutes to return to ACTIVE, so the three UpdateTable mode-switch tests time out there. All three pass with a 900s budget. Bump the default, or scale it up when EXTENDDB_TEST_ENDPOINT is unset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants