diff --git a/.github/workflows/verify.yml b/.github/workflows/verify.yml index 0d2d81c..319aabc 100644 --- a/.github/workflows/verify.yml +++ b/.github/workflows/verify.yml @@ -52,3 +52,13 @@ jobs: - name: Python-frame JS cross-check (zero-dep independent reader, full round-trip) run: node tools/frame-crosscheck.mjs + + # Narrow on purpose: catches one facet of ONE of the six matrix incidents + # listed in decisions/matrix-version-verification.md — a version in the SDK + # Overview table stated as a snapshot, true when written and false at the + # next release. The other five need a reader, not a regex. The mutation + # suite runs first so the guard cannot degrade to silently reporting OK. + - name: SDK Overview versions are floors, not snapshots + run: | + python3 tools/test_check_version_floors.py + python3 tools/check-version-floors.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 9be360b..3bd9ffc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,64 @@ All notable changes to the CacheKit Protocol Specification. ## [Unreleased] +### SDK Feature Matrix + +- Consolidated ten conflicting open matrix PRs into one code-verified end-state + (LAB-1400), regenerated from current SDK code rather than from the stale PR + diffs. Cells that **reversed** — check these if you built on them: key + rotation (py/rs ✅ → ❌ fleet-wide; `rotate_key()` is a `NotImplemented` + stub, and cachekit-py's importable PyO3 `KeyRotationState` succeeds while + rotating nothing), Rust `::secure` preset and Rust sync support (both ✅ → + never existed), Builder API (py/ts ✅ → ❌), hardware-acceleration detection + (rs ✅ → not re-exported; ts N/A → ❌), TypeScript Arrow (🔜 → ❌), and + Python's encrypted read path (documented fail-closed → **fail-open by + default**), and `cache.secure.wrap()` in TypeScript (implied encryption → no + guarantee at all; LAB-513, CWE-311). New rows: Retry, Graceful degradation, + Cross-instance L1 invalidation (LAB-520), client-L1 stale-while-revalidate + (LAB-728), Orjson serializer, tamper/wrong-key failure mode, `secure`-API + enforcement, plus an Observability section (LAB-275). Supersedes protocol#25, + #28, #29, #31, #32, #33, #35, #37, #40, #43 — per-PR fold verdicts below. + +- **Every version-keyed claim re-verified against published artifacts**, after an + expert-panel review found the first pass had introduced two new false cells of + the very class it was fixing. `cachekit-rs` 0.6.0 published 74 minutes before + that pass's final commit, so six Rust reliability cells shipped marked 🚧 + unreleased when the tier was in fact released and **on by default**; and + "cachekit-ts ships the protocol-1.1 `bin` flip as of 0.1.5" was false — + published `@cachekit-io/cachekit@0.1.5` carries dependency pins byte-identical + to 0.1.4's, on `cachekit-core-ts@0.1.2` (native addons embed core **0.2.0**) + and `cachekit-core-wasm@0.1.1` (core **0.3.0**), so TypeScript emits legacy on + both paths. The matrix now carries a per-artifact rollout table with the + embedded-core evidence, and the method is recorded in + [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md): + registry metadata establishes which artifact is current, and where an embedded + dependency decides the claim the `.crate`/`.tgz` is opened. Versions in the SDK + Overview are now floors (`X+`), enforced by + [`tools/check-version-floors.py`](tools/check-version-floors.py) in `verify.yml` + — four failures of this one mechanical class in six weeks. + +- Footnote namespace repaired: markers `¹`–`⁴` were each defined **twice** with + unrelated content (Cache Backends and Protocol Compliance), so half the + evidence pointers in the file resolved to the wrong note — including the + "version cells are floors" note. The Protocol Compliance block is now `¹⁴`–`¹⁷` + and every marker is defined exactly once. + +#### Per-PR fold verdicts (LAB-1400) + +| PR | Ticket | Verdict | +| :--- | :--- | :--- | +| [#25](https://github.com/cachekit-io/protocol/pull/25) | LAB-423 | Incorporated — `spec/wire-format.md` lacked the CI-enforcement note; both enforcement points now named | +| [#28](https://github.com/cachekit-io/protocol/pull/28) | LAB-274 | Incorporated incl. the intent-preset semantics table; rejected its stale "rs has no circuit breaker" line | +| [#29](https://github.com/cachekit-io/protocol/pull/29) | LAB-275 | Partly incorporated (key rotation, hardware accel, Observability, serializer rows, MSRV 1.85); versions / ts-Workers / rs-stampede / interop claims rejected as stale | +| [#31](https://github.com/cachekit-io/protocol/pull/31) | LAB-520 | Incorporated as-is | +| [#32](https://github.com/cachekit-io/protocol/pull/32) | LAB-426 | Incorporated — rs Workers locking + TTL was still missing from `main` | +| [#33](https://github.com/cachekit-io/protocol/pull/33) | LAB-427 | Already on `main`; would have reintroduced a stale rs-Redis-lock ❌ | +| [#35](https://github.com/cachekit-io/protocol/pull/35) | LAB-518 | Incorporated; rejected its "backpressure stays ❌" line (LAB-729) | +| [#37](https://github.com/cachekit-io/protocol/pull/37) | LAB-430 | Already on `main`; same stale-cell problem as #33 | +| [#40](https://github.com/cachekit-io/protocol/pull/40) | LAB-751 | Incorporated — `main` still claimed "SWR forced off" on Workers | +| [#43](https://github.com/cachekit-io/protocol/pull/43) | LAB-728 | Incorporated; extended the py cell, which understated its gate (needs an explicit `ttl=`) | +| [#17](https://github.com/cachekit-io/protocol/pull/17) | — | Out of scope, left open | + ### Specs - StorageEnvelope `compressed_data` canonical encoding flipped from MessagePack diff --git a/decisions/key-rotation.md b/decisions/key-rotation.md index df556b0..abceea5 100644 --- a/decisions/key-rotation.md +++ b/decisions/key-rotation.md @@ -8,7 +8,7 @@ | **Date** | 2026-07-23 | | **Ticket** | LAB-516 (filed by the LAB-275 cross-SDK feature-gap audit) | | **Normative spec** | [`spec/encryption.md` → Key Rotation (Keyring)](../spec/encryption.md#key-rotation-keyring) — the spec section owns the rules; this record owns the rationale and runbooks. | -| **Implementation** | Not yet shipped in any SDK — tracked as LAB-516 sub-issues. The [feature matrix](../sdk-feature-matrix.md) rotation row is corrected to ❌ fleet-wide by the LAB-275 audit PR ([protocol#29](https://github.com/cachekit-io/protocol/pull/29)) and flips per SDK only as each implementation ships. | +| **Implementation** | Not yet shipped in any SDK — tracked as LAB-516 sub-issues. `ZeroKnowledgeEncryptor::rotate_key()` returns `NotImplemented` (`cachekit-core/src/encryption/core.rs:492`). The [feature matrix](../sdk-feature-matrix.md#encryption) rotation row reads ❌ fleet-wide as of the LAB-1400 consolidation and flips per SDK only as each implementation ships. | --- diff --git a/decisions/matrix-version-verification.md b/decisions/matrix-version-verification.md new file mode 100644 index 0000000..f1e6b3c --- /dev/null +++ b/decisions/matrix-version-verification.md @@ -0,0 +1,68 @@ +**[Protocol](../README.md)** > **Decisions** > **Matrix Version Verification** + +# Decision Record: Verify the Feature Matrix Against Published Artifacts, Not Branches + +| | | +| :--- | :--- | +| **Status** | Accepted | +| **Date** | 2026-08-04 | +| **Ticket** | LAB-1400 (consolidation of ten conflicting `sdk-feature-matrix.md` PRs) | +| **Applies to** | [`sdk-feature-matrix.md`](../sdk-feature-matrix.md), and any version- or ship-status claim in [`spec/`](../spec) | + +--- + +## Context + +[`sdk-feature-matrix.md`](../sdk-feature-matrix.md) is a trust surface: readers use it to decide what they can rely on without reading four SDKs. In six weeks it produced **six** failures of one family — a cell asserting something the reader would act on that the shipped artifacts did not support: + +| # | Failure | Direction | +| :--- | :--- | :--- | +| LAB-388 | "SWR ✓" for orphaned, never-called code | claimed more than shipped | +| LAB-998 | interop/v1 ship-status false | claimed less than shipped | +| LAB-1400 pass 1 | six Rust reliability cells marked 🚧 unreleased — `cachekit-rs` 0.6.0 had published 74 minutes earlier | claimed less than shipped | +| LAB-1400 pass 1 | "cachekit-ts ships the `bin` flip as of 0.1.5" — the published 0.1.5 pins a NAPI addon embedding `cachekit-core` 0.2.0 | claimed more than shipped | +| LAB-1400 pass 2 | a floor-scoping instruction that, read literally, told maintainers to strip the seven floors living outside the Overview table | would have reopened LAB-998 | +| LAB-1400 pass 3 | "a reader built against core ≤ 0.3.0 rejects `bin`", with fleet-upgrade sequencing advice derived from it — contradicted by `cachekit-core/tests/dual_decode.rs`, which asserts pre-flip readers accept `bin` | invented a migration risk | + +**Four of the six were committed by the audit that existed to remove the first two.** That is the signal, and it is not carelessness: each was written by someone holding the correct general rule and applying it to a claim they had not opened the artifact for. Reading `main` tells you what the *next* release will contain, never what the current one does. Reading a plausible mechanism tells you what *could* happen, never what the test asserts. The gap in both cases is where the trust bug lives. + +Two properties make it worse than ordinary staleness: + +- **It is bidirectional.** A ❌ hiding a shipped feature makes users hand-roll what they already have; a ✅ they cannot install makes them plan around vapour. Neither is safer. +- **It rots without an edit.** A registry release falsifies a cell while the file is untouched. `cachekit-rs` 0.6.0 falsified six cells and a footnote in one event. + +## Decision + +**A version- or ship-status claim in the matrix is verified against the published artifact, or it is not made.** + +1. **Registry metadata is the floor, not the proof.** Latest version and publish timestamp come from the registry API (crates.io, npm, PyPI). That establishes *which* artifact is current — nothing more. + +2. **When an embedded or transitive dependency version decides the claim, open the artifact.** Downloading and inspecting is mandatory, not optional: + - Rust — fetch the `.crate`, read the *published* `Cargo.toml` for `[features] default` and dependency requirements, and list `src/` for the modules the claim names. + - npm — read the exact `dependencies` pins from the registry document (caret-free pins do **not** float), fetch the `.tgz`, and `strings` the `.node` / `.wasm` for the embedded `cachekit-core-X.Y.Z`. A source-level pin bump in a monorepo does **not** mean the consumed binary was republished. + - Python — the wheel's bundled extension, when a core version decides the claim. + +3. **A merged PR is not a shipped feature.** Cite the release that carries it, not the PR that landed it. `git log` and tags describe intent; the registry describes reality. + +4. **A version answering "which release do I need" is a floor (`X+`), never a snapshot.** A floor stays true as new releases publish; a snapshot is wrong the moment the next one lands and silently misleads until someone notices. That covers the SDK Overview table, the Compliance Status table, and the Architecture Notes release bullets. A version that is instead **evidence about one specific artifact** under rule 2 — an embedded `cachekit-core-0.2.0` in one `.node` binary, a caret-free `0.1.2` npm pin, a historical statement — stays bare, because appending `+` to it would make it false. + + The CI guard polices only the SDK Overview column. The other floors are reviewer discipline: distinguishing "which release do I need" from "what is inside this artifact" needs the claim's intent, which a regex cannot read. + +5. **Record the verification date** next to the claim. A floor plus a date is auditable; a bare number is a guess with a decimal point. + +6. **For a behavioural claim, cite the executed test — not a mechanism you traced.** Reading a code path and reasoning "therefore X rejects Y" produces claims that are plausible and wrong; three of the six incidents above were exactly that. Where a test already asserts the behaviour, cite the test (`cachekit-core/tests/dual_decode.rs` settles the dual-read question in one file). Where none does, trace the **whole** path including its error handling — a claim about what an SDK does on failure is worthless if the operation runs inside a `catch` you did not read — and say which layer you checked. If the two disagree, the test wins. + +## Consequences + +- Refreshing the matrix costs a handful of artifact downloads. That is the price of the document meaning anything, and it is minutes. +- CI partially enforces rule 4: [`.github/workflows/verify.yml`](../.github/workflows/verify.yml) runs [`tools/check-version-floors.py`](../tools/check-version-floors.py), which fails on a non-floor version in the **SDK Overview table only**, and [`tools/test_check_version_floors.py`](../tools/test_check_version_floors.py), a 17-case mutation suite that runs first so the guard cannot silently degrade to reporting OK. + +- **Be honest about the guard's reach: it catches one facet of one of the six incidents above.** The pass-1 `cachekit-rs` event left a `0.5.0` snapshot in the Overview table alongside the six mis-marked Reliability cells; the guard sees the snapshot and nothing else. The other five incidents — LAB-388 (a ✅ on dead code), LAB-998 (a ship-status boolean), the pass-1 ts `bin` claim, the pass-2 floor-scoping instruction, the pass-3 dual-read claim — are all invisible to it, as are the Reliability cells from the very event it partly catches. It also cannot tell whether a floor is *accurate*. Rules 1–3, 5 and 6, and every floor outside that one table, remain reviewer discipline. + + This matters more than it looks. An earlier revision of this record and of `verify.yml` both claimed the guard encoded "the one failure mode that recurred four times" — which would tell the next auditor that CI has this covered when it does not. An overclaiming gate is worse than no gate, because it converts a known gap into an assumed-safe one. The guard's own first version also passed a snapshot hidden behind an ASCII-hyphen placeholder and rejected a valid backticked floor; the mutation suite exists because of that. + +## Rejected alternatives + +- **Generate the version table from the registries in CI.** Removes the class outright, but the matrix's value is the *prose* — "shipped but unreachable", "on by default since 0.6.0" — which no generator produces. A generated table beside hand-written prose would drift from it, trading one inconsistency for another. +- **Drop versions from the matrix entirely.** "Which release do I need?" is the question the document is most often opened to answer. +- **Trust the SDK repos' own CHANGELOGs.** They record merges, and release-please tags can precede or follow publication. `cachekit-ts` 0.1.5's changelog lists the core-0.4.0 bump ([cachekit-ts#91](https://github.com/cachekit-io/cachekit-ts/pull/91)) while its published tree still pins the old addon — the changelog is exactly how the false claim was produced. diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 13a96ea..a1f64e9 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -6,7 +6,9 @@ **Feature parity and compliance status across all CacheKit SDK implementations.** -*Last updated: 2026-07-29 — LAB-903: SDK Overview versions refreshed against the registries after the protocol 1.1 (envelope `bin` encoding, cachekit-core 0.4.0) rollout — py 0.17.0, core 0.4.0, rs 0.5.0 + macros 0.5.0, ts 0.1.4; per-SDK rollout state (which releases carry the flip, which bumps sit merged-unreleased on main) is recorded once, in the [cachekit-core architecture note](#architecture-notes). LAB-998: interop/v1 ship-status corrected — the row no longer reads `unreleased`; all three SDKs have published it (PyPI 0.14.0+, crates.io 0.4.0+, npm 0.1.3+), stated as floors per footnote ⁴, aligned with docs.cachekit.io (LAB-996). LAB-750: ts Workers gains native edge storage — Workers KV + Cache API backend rows added (footnote ¹ updated: phase-1 CachekitIO-only surface superseded; backend instances now accepted by the `minimal`/`production`/`secure` intents in every ts entrypoint). LAB-729: rs backpressure flipped ❌ → ✅ (semaphore + bounded queue in the rs reliability stack; decision footnote records why the LAB-519 ts rationale doesn't transfer to tokio). LAB-430 shipped TypeScript Node-only Memcached and File backends; the protocol-owned File format and vectors now define fail-closed flag negotiation. LAB-446: Python File backend gains full TTL inspection/refresh; Memcached gains `refresh_ttl` (touch) only (see [TTL management note](#reliability-features)). LAB-595 shipped: ts Cloudflare Workers flipped ❌ → ✅ via the `@cachekit-io/cachekit/workers` entrypoint on a wasm32 cachekit-core build (~55 KB gz measured); footnote ¹ records the phase-1 surface and semantics deltas. LAB-519: ts cold-miss single-flight (in-process, always on) + LockableBackend wired into `wrap()`'s miss path (opt-in); ts backpressure decision recorded; ts Redis lock/TTL capability cells refreshed for LAB-427. LAB-272 code-verified protocol-adherence audit (2026-07-22): interop/v1 merged in Python ([cachekit-py#220](https://github.com/cachekit-io/cachekit-py/pull/220)), TypeScript ([cachekit-ts#71](https://github.com/cachekit-io/cachekit-ts/pull/71)), and Rust ([cachekit-rs#33](https://github.com/cachekit-io/cachekit-rs/pull/33)); test-vector CI coverage corrected* +*Last updated: 2026-08-04 — LAB-1400 consolidation of ten open matrix PRs into one end-state. Every version-keyed claim is verified against the **published artifact** (registry metadata, and the `.crate`/`.tgz` contents where an embedded dependency version decides the answer), not against a repo branch — see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for why and how. Per-PR fold verdicts are in [CHANGELOG.md](CHANGELOG.md); per-row history is `git log sdk-feature-matrix.md`.* + +*__Cells that reversed — check these if you built on them:__ Key rotation (py/rs ✅ → ❌ fleet-wide), Rust `::secure` preset and Rust sync support (both ✅ → do not exist), Builder API (py/ts ✅ → ❌), Hardware acceleration (rs ✅ → not re-exported, ts N/A → ❌), TypeScript Arrow (🔜 → ❌), Python's encrypted read path (documented fail-closed → **fail-open by default**), and `cache.secure.wrap()` in TypeScript (implied encryption → **no guarantee**). The TypeScript protocol-1.1 `bin` rollout also reversed twice in two days: it is **not** shipped on either ts path (per-artifact evidence in the [cachekit-core architecture note](#architecture-notes)).* @@ -20,6 +22,7 @@ - [Cache Backends](#cache-backends) - [Backend Abstraction](#backend-abstraction) - [Reliability Features](#reliability-features) +- [Observability](#observability) - [Developer Experience](#developer-experience) - [Protocol Compliance](#protocol-compliance) - [Architecture Notes](#architecture-notes) @@ -30,12 +33,14 @@ | SDK | Package | Version | Language | Status | | :--- | :--- | :---: | :--- | :---: | -| cachekit-py | `cachekit` (PyPI) | 0.17.0 | Python 3.10+ | ✅ Production | -| cachekit-rs | `cachekit-rs` (crates.io) | 0.5.0 | Rust 1.82+ | ✅ Production | -| cachekit-core | `cachekit-core` (crates.io) | 0.4.0 | Rust (shared core) | ✅ Production | -| cachekit-ts | `@cachekit-io/cachekit` (npm) | 0.1.4 | TypeScript | ✅ Production | +| cachekit-py | `cachekit` (PyPI) | 0.17.1+ | Python 3.10+ | ✅ Production | +| cachekit-rs | `cachekit-rs` (crates.io) | 0.6.0+ | Rust 1.85+ | ✅ Production | +| cachekit-core | `cachekit-core` (crates.io) | 0.4.0+ | Rust (shared core) | ✅ Production | +| cachekit-ts | `@cachekit-io/cachekit` (npm) | 0.1.5+ | TypeScript (Node 22+) | ✅ Production | | cachekit-php | — | — | PHP 8.1+ | 🔜 Development | +> Every version in **the table above** is a **floor** (`X+`), never a snapshot — see [Compliance Status](#compliance-status) note ¹⁷. Registry-verified 2026-08-04; check the registry for the current release. Elsewhere in this document **both forms appear, deliberately**: a floor (`X+`) wherever the question is *"which release do I need"* — the [Compliance Status](#compliance-status) table, note ¹³, the [Architecture Notes](#architecture-notes) — and a bare exact version wherever the claim is a fact about **one specific artifact** (an embedded `cachekit-core-0.2.0`, a caret-free npm pin, a historical statement). Do not mechanically convert either into the other: adding `+` to an artifact fact makes it false, and stripping `+` from a floor reopens the staleness class this document has hit repeatedly (see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for the incident list). + --- ## Core Features @@ -44,9 +49,12 @@ | :--- | :---: | :---: | :---: | :---: | | StandardSerializer (MessagePack) | ✅ | ✅ via rmp-serde | ✅ | 🔜 Planned | | AutoSerializer (Python-specific) | ✅ | N/A | N/A | N/A | -| ArrowSerializer (columnar) | ✅ | N/A | 🔜 Planned | ❌ | +| OrjsonSerializer (fast JSON) | ✅ `[json]` extra | N/A | N/A | N/A | +| ArrowSerializer (columnar) | ✅ `[data]` extra | N/A | ❌ (LAB-524)⁰ | ❌ | | ByteStorage (LZ4 + xxHash3-64) | ✅ via Rust FFI | ✅ canonical (cachekit-core) | ✅ via NAPI (Rust) | 🔜 Planned | -| Blake2b-256 key generation | ✅ | N/A | ✅ via @noble/hashes | 🔜 Planned | +| Blake2b-256 key generation | ✅ | ✅ interop mode only — N/A auto mode (see [Compliance Status](#compliance-status) note ¹⁴) | ✅ via @noble/hashes | 🔜 Planned | + +> ⁰ TypeScript Arrow was listed 🔜 Planned from 2026-06-06 with no code, stub, or tracking issue behind it; corrected to ❌, with LAB-524 owning the implement-or-decline decision. Orjson and Arrow live behind cachekit-py's `[json]` / `[data]` extras (`pyproject.toml:73-81`). --- @@ -54,16 +62,37 @@ | Feature | Python | Rust | TypeScript | PHP | | :--- | :---: | :---: | :---: | :---: | -| AES-256-GCM | ✅ via Rust FFI | ✅ ring (native) / aes-gcm (wasm32) | ✅ via NAPI (Rust) | 🔜 Planned | +| AES-256-GCM | ✅ via Rust FFI | ✅ ring (native) / aes-gcm (wasm32) — assumes the `encryption` feature¹⁹ | ✅ via NAPI (Rust) | 🔜 Planned | | HKDF-SHA256 key derivation | ✅ via Rust FFI | ✅ | ✅ via NAPI (Rust) | 🔜 Planned | | Per-tenant key isolation | ✅ | ✅ | ✅ via TenantKeys NAPI | 🔜 Planned | | AAD v0x03 (cache_key binding) | ✅ | ✅ | ✅ | ❌ | -| Key rotation | ✅ | ✅ | ❌ | ❌ | -| Hardware acceleration detection | ✅ | ✅ | N/A | N/A | +| Key rotation | ❌ mismatch detection only⁵ | ❌⁵ | ❌ — nonce-exhaustion detection exists but is **unobservable** to the caller⁵ ⁸ | ❌ | +| **Tamper / wrong-key failure mode** | ⚠️ **fail-OPEN by default** — warn + recompute; switchable with `CACHEKIT_ENCRYPTION_FAIL_CLOSED=true`¹⁸ | ✅ **Fails closed** — `decrypt(…)?` propagates (`client.rs:830`, `:847`), and `#[cachekit(secure)]` emits no fail-open arm (`cachekit-macros/src/lib.rs:439-451`) | ⚠️ **fail-OPEN on reads, silently drops writes, not switchable**⁸ | — | +| **Does the `secure` API enforce encryption?** | ✅ Raises without a key | ✅ `secure()` returns `Err` | ❌ **`cache.secure.wrap()` is an unconditional alias for `wrap()`** — silently caches plaintext on any instance not built by `createCache.secure()` (LAB-513, CWE-311); see [Intent-preset semantics](#intent-preset-semantics-parity-not-presence) | — | +| Hardware acceleration detection | ✅ surfaced (`hardware_acceleration_enabled()`) | ⚠️ core-internal, not re-exported⁶ | ❌ not exposed⁶ | N/A | | Counter-based nonces | ✅ via Rust | ✅ | ✅ via NAPI (Rust) | ❌ use random | > [!IMPORTANT] -> AAD v0x03 is required for protocol compliance. SDKs without it cannot safely interoperate with encrypted payloads from compliant SDKs — the auth tag will fail verification. Python, Rust, and TypeScript construction was code-verified byte-identical on 2026-07-21 (LAB-272); the normative byte layout and the frozen `True`/`False` compressed tokens (protocol#12) are defined in [spec/encryption.md](spec/encryption.md#additional-authenticated-data-aad). Python's auto serializers append the optional `original_type` fifth component; Rust, TypeScript, and interop mode always emit exactly four. Python's read path fails closed when encryption is enabled but a stored entry claims plaintext ([cachekit-py#215](https://github.com/cachekit-io/cachekit-py/pull/215)). +> AAD v0x03 is required for protocol compliance. SDKs without it cannot safely interoperate with encrypted payloads from compliant SDKs — the auth tag will fail verification. Python, Rust, and TypeScript construction was code-verified byte-identical on 2026-07-21 (LAB-272); the normative byte layout and the frozen `True`/`False` compressed tokens (protocol#12) are defined in [spec/encryption.md](spec/encryption.md#additional-authenticated-data-aad). Python's auto serializers append the optional `original_type` fifth component; Rust, TypeScript, and interop mode always emit exactly four. When encryption is enabled but a stored entry claims plaintext, Python never returns it — the entry is converted to a miss and evicted rather than raising (LAB-241, [cachekit-py#215](https://github.com/cachekit-io/cachekit-py/pull/215)); see note ⁵ for what that means if you are monitoring for hard errors. + +> [!WARNING] +> ⁵ **Key rotation ships in no SDK. Rotating a master key today invalidates every encrypted entry.** Corrected 2026-08-04 (LAB-1400) from a previous Python ✅ / Rust ✅ that no code supported. `cachekit-core`'s `ZeroKnowledgeEncryptor::rotate_key()` returns `EncryptionError::NotImplemented` (`src/encryption/core.rs:492`); cachekit-rs neither re-exports nor calls any rotation API; cachekit-ts classifies the core's nonce-exhaustion message into a `NonceExhaustedError` that **no caller can observe**, because it is raised inside the `set` callback and absorbed there (note ⁸). Note that nonce exhaustion does *not* make rotation mandatory — the core fails closed and a fresh encryptor instance clears it; reaching for a master-key rotation instead would destroy every existing entry, which is exactly the trap this note exists to prevent. +> +> **Do not build on cachekit-py's importable `KeyRotationState`.** cachekit-py ships a live PyO3 binding — `#[pyclass(name = "KeyRotationState")]` with `start_rotation()` / `complete_rotation()` / `is_rotating()` (`rust/src/python_bindings.rs:258`, `:283`, registered on `_rust_serializer` at `:394`) — so `from cachekit._rust_serializer import KeyRotationState` works today, `start_rotation(new_key)` **returns success**, and `is_rotating()` then reports `True`. Nothing in `src/cachekit/` consults any of it: no dual-key read path exists, so an operator who takes that success as a grace window and swaps `CACHEKIT_MASTER_KEY` renders every encrypted entry undecryptable. Removal is tracked under LAB-516. `cachekit-core` likewise exports `RotationAwareHeader` (`src/encryption/mod.rs:20`, aliased as the "canonical" `EncryptionHeader`) which no SDK ever writes to the wire. +> +> ¹⁸ **Python's encrypted read path is fail-OPEN by default.** `EncryptionWrapper(fail_closed=False)` (`serializers/encryption_wrapper.py:113`) from `encryption_fail_closed: bool = Field(default=False)` (`config/settings.py:225`): on a key-fingerprint mismatch the default is a warning plus a recompute, not a raise — that flag gates the fingerprint pre-check — and the same setting drives the AES-GCM authentication-failure policy through `handle_decrypt_failure` (`cache_handler.py:1307`, `:1317`; resolved at `:550-552`, L1 path `decorators/wrapper.py:1210`). Both default to recompute. Set `CACHEKIT_ENCRYPTION_FAIL_CLOSED=true` if you are relying on hard errors as your wrong-key or tamper alarm — otherwise the signal is a log line and a billable miss. (Separately, a stored entry that claims plaintext while encryption is enabled is never returned to the caller: it is converted to a miss and evicted, deliberately, for plaintext→encrypted migration — LAB-241, [cachekit-py#215](https://github.com/cachekit-io/cachekit-py/pull/215).) +> +> The rotation **design** is specified, and the retired header recorded, in [decisions/key-rotation.md](decisions/key-rotation.md) (LAB-516). +> +> ⁸ **TypeScript silently absorbs every encryption failure, on reads and writes alike, and ships no switch to stop it.** The mechanism is one layer, not a per-operation quirk: every operation that reaches the backend runs inside `ReliabilityExecutor.execute` (L1 hits short-circuit before it and run no crypto; `acquireLock` sits outside it) — `getEntry` returns `this.run('get', …)` with the decrypt inside the callback (`cache-core.ts:405`, `:415`), and `setEntry` returns `this.run('set', …)` with the encrypt inside its callback (`:478`, `:487`) — and the executor's degradation handler is a bare `catch {}` with **no error-class check** (`reliability/degradation.ts:13`). Retry and the circuit breaker both rethrow, so degradation is the only thing that swallows, and it swallows everything. Consequences, all from that one fact: +> +> - **Read** — a tampered payload or wrong key makes `cache.get()` return `null`, an ordinary miss. Through `wrap()` the function is re-executed and the result re-stored. +> - **Write** — an encrypt, NAPI or key failure makes `set()` **resolve like success while storing nothing**; every later `wrap()` re-executes the origin, forever, silently. +> - **Nonce exhaustion** — `NonceExhaustedError` is raised *inside* `encrypt()` (`encryption/manager-core.ts`), i.e. inside the `set` callback, so it is absorbed too. **Nonce reuse is not the risk:** `cachekit-core` fails closed at 2³² operations per encryptor instance — `generate_nonce()` returns `NonceCounterExhausted` once `counter >= u32::MAX`, and the counter is an `AtomicU64` specifically so it *stays* exhausted rather than wrapping (`encryption/core.rs:168-185`, `:297-311`). The unobservable consequence is instead that **every** encrypted `set()` on that encryptor silently stores nothing for the remaining life of the process. An operator seeing only a rising miss rate has no way to reach the real remedy — a fresh encryptor instance — and per note ⁵ the instinct to rotate the master key would destroy every existing entry. +> +> Degradation is on unless explicitly disabled (`degradationEnabled = config.degradation !== false`, `reliability/executor.ts:39`) and `createCache.secure()` / `.production()` / `.io()` all set it `true` (`intents-core.ts:186`); only `minimal` sets `false`, and `minimal` carries no encryption. **There is no `failClosed` option anywhere in cachekit-ts** — Python's `CACHEKIT_ENCRYPTION_FAIL_CLOSED` has no counterpart, so the only lever is `reliability: { degradation: false }`, which also gives up backend-outage degradation. If you rely on a thrown error as your tamper, wrong-key or nonce alarm, TypeScript raises none; the sole signal is the `errors_total` counter. +> +> ⁶ Runtime AES detection (`is_x86_feature_detected!("aes")`, `cachekit-core/src/encryption/core.rs:243`) lives in the shared core and is **surfaced only by Python** (`encryption_wrapper.py:583`). cachekit-rs never re-exports it — the SDK calls the non-metrics encrypt/decrypt entry points — and the TypeScript NAPI layer exposes nothing — `N/A` there was wrong, since ts runs the same Rust core. Tracked as LAB-523. --- @@ -75,14 +104,20 @@ | Memcached | ✅ `backends/memcached` (`memcached` extra) | ✅ `memcached` feature (rust-memcache)³ | ✅ `backends/memcached.ts` (optional `memjs` peer; Node-only)⁴ | ❌ | | File (local) | ✅ `backends/file` (stdlib + mmap) | ✅ `file` feature (byte-compatible with py)³ | ✅ `backends/file.ts` (Node-only; [format](spec/file-backend-format.md))⁴ | ❌ | | CacheKit SaaS (HTTP) | ✅ `backends/cachekitio` (httpx) | ✅ `cachekitio` feature (reqwest, default) | ✅ `backends/cachekitio.ts` (fetch) | 🔜 Planned | -| Cloudflare Workers | N/A | ✅ `workers` feature (`worker::Fetch`) | ✅ `@cachekit-io/cachekit/workers` (wasm32 core)¹ | N/A | +| Cloudflare Workers | N/A | ✅ `workers` feature (`worker::Fetch`) — needs `--no-default-features`¹⁹ | ✅ `@cachekit-io/cachekit/workers` (wasm32 core)¹ | N/A | | Workers KV (Cloudflare) | N/A | ❌ | ✅ `workersKV` on the `/workers` entry (LAB-750)¹ | N/A | | Cache API (Cloudflare) | N/A | ❌ | ✅ `workersCacheAPI` on the `/workers` entry (LAB-750)¹ | N/A | | DynamoDB | ❌² | ❌ | ❌ | ❌ | -> ¹ Shipped in LAB-595 (2026-07-24), following spike LAB-431's GO verdict: the `@cachekit-io/cachekit/workers` subpath (also the `workerd` condition on the root export) runs crypto and the ByteStorage envelope on a **wasm32 build of cachekit-core** (`@cachekit-io/cachekit-core-wasm`, 137 KB raw / ~55 KB gzipped + ~10 KB JS glue, wasm-bindgen `--target web` + wasm-opt `-Oz`) — counter nonces and the envelope carry over unchanged and the crypto stays single-touchpoint in the Rust core; byte-verified against the Python-ground-truth `test-vectors/encryption.json` and `wire-format.json` suites inside real workerd. Backend surface (phase 2, LAB-750): CachekitIO (pure `fetch`), **Workers KV** (`workersKV({ kv })` over a `KVNamespace` binding; native `expirationTtl` — KV rejects values under its 60s minimum, so the SDK clamps shorter TTLs up to 60s before the `put`, and by an SDK rule (not KV's) `ttl <= 0` stores without expiry; eventually consistent ~60s), the **Cache API** (`workersCacheAPI()` over `caches.default`/named caches; per-data-center only, `Cache-Control: max-age` to the second, best-effort eviction, keys mapped to synthetic never-fetched URLs), or a custom `Backend` instance — all storage transports over the unchanged opaque ByteStorage payload (encryption above the backend; secure caches store ciphertext only). Backend instances are accepted by the `minimal`/`production`/`secure` intents (`{ backend }` in place of `{ url }`, both entrypoints); Redis-URL intents, Redis Pub/Sub invalidation, and Prometheus metrics stay Node-only and are excluded from the edge bundle (CI-guarded: no `node:*`, no NAPI, no ioredis/prom-client — no `nodejs_compat` flag needed); SWR background refresh is forced off (fire-and-forget refreshes aren't tied to `ctx.waitUntil` yet and workerd cancels them at response return). Semantics delta: keys live in wasm linear memory (a host-readable ArrayBuffer), weaker isolation than NAPI's Rust heap but ~JS-heap-equivalent on Workers where the host is your own isolate; zeroized deterministically on `dispose()`. WebCrypto (AES-256-GCM + HKDF-SHA256, random-nonce fallback per [encryption.md → Nonce Generation](spec/encryption.md#nonce-generation)) remains the documented fallback if the wasm path ever hits a wall. +> ¹ Shipped in LAB-595 (2026-07-24), following spike LAB-431's GO verdict: the `@cachekit-io/cachekit/workers` subpath (also the `workerd` condition on the root export) runs crypto and the ByteStorage envelope on a **wasm32 build of cachekit-core** (`@cachekit-io/cachekit-core-wasm`, 137 KB raw / ~55 KB gzipped + ~10 KB JS glue, wasm-bindgen `--target web` + wasm-opt `-Oz`) — counter nonces and the envelope carry over unchanged and the crypto stays single-touchpoint in the Rust core; byte-verified against the Python-ground-truth `test-vectors/encryption.json` and `wire-format.json` suites inside real workerd. Backend surface (phase 2, LAB-750): CachekitIO (pure `fetch`), **Workers KV** (`workersKV({ kv })` over a `KVNamespace` binding; native `expirationTtl` — KV rejects values under its 60s minimum, so the SDK clamps shorter TTLs up to 60s before the `put`, and by an SDK rule (not KV's) `ttl <= 0` stores without expiry; eventually consistent ~60s), the **Cache API** (`workersCacheAPI()` over `caches.default`/named caches; per-data-center only, `Cache-Control: max-age` to the second, best-effort eviction, keys mapped to synthetic never-fetched URLs), or a custom `Backend` instance — all storage transports over the unchanged opaque ByteStorage payload (encryption above the backend; secure caches store ciphertext only). Backend instances are accepted by the `minimal`/`production`/`secure` intents (`{ backend }` in place of `{ url }`, both entrypoints); Redis-URL intents, Redis Pub/Sub invalidation, and Prometheus metrics stay Node-only and are excluded from the edge bundle (CI-guarded: no `node:*`, no NAPI, no ioredis/prom-client — no `nodejs_compat` flag needed); SWR background refresh requires binding the request's `ExecutionContext` — `cache.withExecutionContext(ctx)` returns a cheap per-request view whose refreshes ride `ctx.waitUntil`, because workerd cancels fire-and-forget work at response return; a read on a cache with no bound context fails safe to a plain (no-SWR) L1 get (LAB-751, `packages/cachekit/src/workers/index.ts:20-48`, `workers/runtime.ts:97-126`). Semantics delta: keys live in wasm linear memory (a host-readable ArrayBuffer), weaker isolation than NAPI's Rust heap but ~JS-heap-equivalent on Workers where the host is your own isolate; zeroized deterministically on `dispose()`. WebCrypto (AES-256-GCM + HKDF-SHA256, random-nonce fallback per [encryption.md → Nonce Generation](spec/encryption.md#nonce-generation)) remains the documented fallback if the wasm path ever hits a wall. > > ² DynamoDB has never shipped in any SDK. The previous Python ✅ traced to the [custom-backend tutorial](https://github.com/cachekit-io/cachekit-py/blob/main/docs/backends/custom.md), which shows how a *user* can implement the backend protocol against DynamoDB — that is an extension point, not shipped support (LAB-273). +> +> ¹⁹ **`cargo add cachekit-rs --features workers` does not compile.** Cargo features are additive, so that command keeps the default set — and the published 0.6.0 crate declares `default = ["cachekitio", "encryption", "l1", "reliability"]` while `src/lib.rs` carries `compile_error!` for `workers`×`l1` (moka needs std threads) *and* `workers`×`reliability` (retry/breaker timers need tokio `time`), plus `workers`×`redis`, `workers`×`memcached` and `workers`×`file`. The Workers build is therefore `--no-default-features --features workers,encryption,cachekitio`. +> +> **Do not drop `encryption` from that list.** With the feature off, `CacheKitBuilder::encryption()` and `::encryption_from_bytes()` are compiled as **silent no-op stubs that return `Ok(self)`** (`crates/cachekit/src/client.rs:1019-1032`) — so the documented builder call succeeds, no error surfaces anywhere, and the cache stores **plaintext at rest**. Only `secure()` is `#[cfg]`-gated and fails loudly; the builder path is not. That is the same CWE-311 shape as the `cache.secure.wrap()` row above, reached by following a feature list instead of an API. Every ✅ in this document's Encryption table assumes `encryption` is enabled. +> +> Verified in the published `cachekit-rs-0.6.0.crate`, not the branch. The ✅ is real — the invocation most readers would try is not, which is why it is stated on the cell rather than left to a compiler error. > > ³ Added 2026-07-24 (LAB-429). Both are cargo features on cachekit-rs (`--features memcached` / `--features file`), native targets only (compile-error guarded against `workers`). The rs File backend shares py's on-disk format — Blake2b-128 hashed filenames, the 14-byte `CK` header, atomic write-then-rename, lazy expiry — verified by running the actual py `FileBackend` against an rs-written directory and vice versa. Not yet ported from py: LRU eviction/size caps and the mmap buffer read. rs Memcached is single-server (py's `HashClient` shards across servers); CI exercises it against a live memcached 1.6 container. @@ -111,15 +146,15 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Capability | Python | Rust | TypeScript | | :--- | :--- | :--- | :--- | -| TTL inspect / refresh | `TTLInspectableBackend` — Redis ✅, SaaS ✅, File ✅, Memcached ⚠️ `refresh_ttl` only (LAB-446) | `TtlInspectable` — Redis ✅, SaaS ✅, File ✅, Memcached ⚠️ `refresh_ttl` only (LAB-429) | `TTLBackend` — Redis ✅, SaaS ✅ (`TTLCachekitIO`), File ✅; Memcached ⚠️ `refreshTTL` only (LAB-430) | -| Distributed locking | `LockableBackend` — Redis ✅ (`redis.lock.Lock`), SaaS ✅ | `LockableBackend` — SaaS ✅, Redis ✅ (`SET NX PX` + Lua compare-and-delete, `:lock` namespace shared with py; LAB-426), Workers ❌ | `LockableBackend` — Redis ✅ (LAB-427), SaaS ✅ (`LockableCachekitIO`) | +| TTL inspect / refresh | `TTLInspectableBackend` — Redis ✅, SaaS ✅, File ✅, Memcached ⚠️ `refresh_ttl` only (LAB-446) | `TtlInspectable` — Redis ✅, SaaS ✅, File ✅, Workers ✅ (`backend/workers.rs:322`; LAB-426 — wasm32 build only¹⁹), Memcached ⚠️ `refresh_ttl` only (LAB-429) | `TTLBackend` — Redis ✅, SaaS ✅ (`TTLCachekitIO`), File ✅; Memcached ⚠️ `refreshTTL` only (LAB-430) | +| Distributed locking | `LockableBackend` — Redis ✅ (`redis.lock.Lock`), SaaS ✅ | `LockableBackend` — SaaS ✅, Redis ✅ (`SET NX PX` + Lua compare-and-delete, `:lock` namespace shared with py), Workers ✅ (`backend/workers.rs:262`) — all three LAB-426; the Workers impls are wasm32-only¹⁹ | `LockableBackend` — Redis ✅ (LAB-427), SaaS ✅ (`LockableCachekitIO`) | | Per-operation timeout | `TimeoutConfigurableBackend` — Redis ✅ (SaaS ships a non-protocol `with_timeout` variant) | — no equivalent | — no equivalent | | Zero-copy buffer read | `BufferReadableBackend` / `BufferHandle` — File ✅ (mmap; #171) | — no equivalent | — no equivalent | > [!NOTE] > **Lock API shape divergence:** Python's `acquire_lock` is an async context manager yielding `bool` — the lock token stays internal and release is automatic. Rust and TypeScript return the raw `lock_id` capability token from `acquire_lock`/`acquireLock` and require an explicit `release_lock(key, lock_id)` — a direct mirror of the SaaS lock endpoint. All three pass the **bare cache key** (backends own any `:lock` namespace derivation). Porting a lockable backend across SDKs must bridge this shape difference. > -> **Coverage, not shape, is the parity gap:** all three SDKs use the same required-base + optional-capability pattern, and Python, Rust, and TypeScript all cover Redis locking plus TTL inspection. Rust's Workers backend lacks both despite speaking the same SaaS API (gap tickets under LAB-102). +> **Coverage:** all three SDKs now cover Redis locking plus TTL inspection, and Rust's Workers backend gained both in LAB-426 — the capability gap the LAB-273 audit recorded is closed. Memcached is refresh-only everywhere (the protocol has no command to read a remaining TTL), and per-operation timeout / zero-copy buffer read are Python-only extensions with no cross-SDK contract. --- @@ -127,14 +162,29 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Feature | Python | Rust | TypeScript | PHP | | :--- | :---: | :---: | :---: | :---: | -| Circuit breaker | ✅ | ❌ | ✅ | ❌ | -| Backpressure | ✅ | ✅ Semaphore + bounded queue, decision recorded (LAB-729) | ⚠️ Refresh cap only — deliberate, decision recorded (LAB-519) | ❌ | -| Distributed locking | ✅ Redis + SaaS backends | ✅ Redis + SaaS (`LockableBackend`; LAB-426. Workers lacks an impl) | ✅ Redis + SaaS backends; wired into `wrap()` cold miss (opt-in `stampede.distributedLock`, LAB-519) | ❌ | +| Circuit breaker | ✅ | ✅ `reliability` feature (default-on) — armed in `production`/`encrypted`/`io`, off in `minimal` (LAB-518)¹³ | ✅ | ❌ | +| Retry (backoff + jitter) | ⚠️ Redis-connection retry only — no generic backend-op retry⁷ | ✅ On the `is_retryable` classification, inside the breaker (`reliability.rs`; LAB-518)¹³ | ✅ `reliability/retry.ts` | ❌ | +| Graceful degradation | ⚠️ Fail-open on backend unavailability — and the encrypted read path is **also** fail-open by default¹⁸ | ✅ `#[cachekit]` runs the function uncached on outage-class failures (transient / timeout / circuit-open / shed); permanent and auth errors propagate, and `#[cachekit(secure)]` fails closed on everything (`cachekit-macros/src/lib.rs:425-451`; LAB-518)¹³ | ⚠️ `reliability/degradation.ts` — catches **every** error class, decrypt failures included; see **Tamper / wrong-key failure mode**⁸ | ❌ | +| Backpressure | ✅ | ✅ Semaphore + bounded queue, decision recorded (LAB-729)¹³ | ⚠️ Refresh cap only — deliberate, decision recorded (LAB-519) | ❌ | +| Distributed locking | ✅ Redis + SaaS backends | ✅ Redis + SaaS + Workers (`LockableBackend`; LAB-426) | ✅ Redis + SaaS backends; wired into `wrap()` cold miss (opt-in `stampede.distributedLock`, LAB-519) | ❌ | | L1/L2 dual-layer cache | ✅ | ✅ moka (native) / `l1` feature | ✅ | ❌ | -| Cache stampede prevention | ✅ | ❌ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | -| TTL management | ✅ Redis + SaaS + File; Memcached refresh-only (see note) | ✅ Redis + SaaS + File (`TtlInspectable`); Memcached refresh-only (LAB-429) | ✅ Redis + SaaS + File (`TTLBackend`); Memcached refresh-only (LAB-430) | ❌ | +| Cache stampede prevention | ✅ Async decorators via backend lock; sync path none by design | ✅ Cold-miss single-flight wired into every `#[cachekit]` expansion — per-key in-process gate plus a distributed fill lock on lock-capable backends (`cachekit-macros/src/lib.rs:512`, `:544`; `flight.rs:278`; LAB-518)¹³ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | +| Cross-instance L1 invalidation (pub/sub) | ❌ built, never wired, then deleted⁹ | ❌ | ✅ Opt-in `invalidation` config over a Redis pub/sub channel (`invalidation/redis-channel.ts`) | ❌ | +| TTL management | ✅ Redis + SaaS + File; Memcached refresh-only (see note) | ✅ Redis + SaaS + File + Workers (`TtlInspectable`); Memcached refresh-only (LAB-429/426) | ✅ Redis + SaaS + File (`TTLBackend`); Memcached refresh-only (LAB-430) | ❌ | +| Stale-while-revalidate (client L1) | ⚠️ L1-only mode (`backend=None`) **and an explicit `ttl=`** only¹⁰ | ✅ Serve-stale + single-flight background refresh (LAB-728)¹⁰ ¹³ | ✅ `getWithSwr` — version tokens + background refresh, `maxConcurrentRefreshes` cap; on Workers requires a bound `ExecutionContext` (see [Cache Backends](#cache-backends) note ¹) | ❌ | | Stale-while-revalidate (server stale-grace) | 🚧 LAB-381 | ❌ | ❌ | ❌ | +> [!IMPORTANT] +> ¹³ **The Rust reliability tier ships in `cachekit-rs` 0.6.0+ and is on by default.** Verified inside the published artifact, not the branch: the `cachekit-rs` 0.6.0 `.crate` from crates.io (published 2026-08-03T14:58:16Z) contains `src/reliability.rs`, `src/flight.rs`, `tests/reliability_tests.rs`, and `get_with_swr` in `src/l1/mod.rs`, and its `Cargo.toml` declares `default = ["cachekitio", "encryption", "l1", "reliability"]`. So a plain `cargo add cachekit-rs` gets **circuit breaker, retry, backpressure and L1 SWR** with no feature flags. Two of the six cells need an opt-in feature: macro-level graceful degradation and the automatic `#[cachekit]` single-flight wiring are emitted by the proc-macro, and `macros = ["dep:cachekit-macros"]` is **not** in `default` — add `--features macros`. Redis-backed presets likewise need the non-default `redis` feature (see [Developer Experience](#developer-experience) note ¹¹). +> +> These six cells read **🚧 Unreleased** between 2026-07-25 and 0.6.0's release, correctly at the time — the tier had landed on `main` after the 0.5.0 tag. 0.6.0 published 74 minutes before this document's own preceding revision was committed, so the qualifier outlived its truth by one commit. Recorded because it cuts both ways: a ✅ a user cannot install and a ❌ hiding something already shipped are the same trust bug (LAB-388), and a matrix regenerated from `main` against a registry snapshot will drift in whichever direction the last release moved. + +> ⁷ **Python has no generic backend-operation retry** (LAB-522). What exists is Redis-client-level reconnect/timeout retry and lock-acquisition retry. Setting `max_retries` does nothing: the field on `CachekitConfig` (`config/settings.py:117`) has no **operational** consumer — its only read is a validator branch whose body is `pass` (`settings.py:252`) — and the CachekitIO backend's own `max_retries` (`backends/cachekitio/config.py:121`) has no reader at all. Rust and TypeScript both wrap backend ops in a real retry layer. +> +> ⁹ Python shipped a complete but never-wired `invalidation/` package (channel, events, Redis pub/sub with reconnect) and **deleted** it rather than wiring it (LAB-520, [cachekit-py#237](https://github.com/cachekit-io/cachekit-py/pull/237); the package is absent from `src/cachekit/` as of 0.17.1). The reason is worth keeping: without server-side key tracking, broadcasting mass invalidations makes other pods evict L1 and immediately re-read the stale L2 entries the invalidating process could not delete — strictly worse than not invalidating. TypeScript's implementation is the reference if Python re-adds it, paired with a distributed key registry. **Invalidation events are not cross-SDK interoperable:** no protocol spec exists and ts's channel name and payload differ from what py's package used, so the ts channel is not a protocol surface. +> +> ¹⁰ **Client-L1 SWR is a distinct capability from the server stale-grace row below** (which is a SaaS response contract, LAB-381). Rust (LAB-728, [cachekit-rs#47](https://github.com/cachekit-io/cachekit-rs/pull/47)): a stale L1 hit is served immediately and triggers exactly one background re-execution, deduplicated through the same cold-miss `single_flight()` as a cold miss — builder options `swr_enabled` (default **on**) / `swr_threshold_ratio` (default 0.5, ±10% jitter per entry so a hot key does not stampede its own refresh), native only. Python needs **both** L1-only mode and an explicit TTL: `_l1_swr_active` requires `_object_cache is not None and swr_enabled and ttl is not None and ttl > 0` (`decorators/wrapper.py:666`), and since Python presets set no default TTL (see [Intent-preset semantics](#intent-preset-semantics-parity-not-presence)) a caller who never passes `ttl=` gets no SWR at all. The byte-layer `L1Cache.get_with_swr` that backed modes would use has no caller outside benchmarks — the dead code behind the previous backed-mode ✓ (LAB-388). +> > **Lock id transport (CWE-532):** the unlock call carries the lock capability token in the `X-CacheKit-Lock-Id` request header, never the `?lock_id=` query string (which leaks via access/proxy logs and OTel `http.url` spans). **Migration complete in all three SDKs** (verified 2026-07-20, LAB-273): Python (#131, closed), Rust (#24, closed), TypeScript ships the header (ts#63 remains open only for an unrelated NAPI-rebuild item). SaaS dual-reads both during the rollout window. See [spec/saas-api.md](spec/saas-api.md#delete-v1cachekeylock). > > **TypeScript backpressure decision (LAB-519):** general admission control beyond L1's `maxConcurrentRefreshes` was evaluated and declined. On Node's single-threaded event loop concurrent misses don't compete for threads, cold-miss single-flight collapses the per-key herd (the amplification vector metered-misses punishes), and distinct-key miss floods are bounded by backend timeouts plus the circuit breaker — a global miss semaphore would add queueing latency and a tuning knob without a failure mode it prevents. Revisit only with evidence of backend connection exhaustion. Full rationale on `StampedeConfig` in cachekit-ts. @@ -156,19 +206,58 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria --- +## Observability + +"Wired" means the live cache path feeds it with zero user plumbing. Audited against code 2026-08-04 (LAB-1400). + +| Feature | Python | Rust | TypeScript | PHP | +| :--- | :--- | :--- | :--- | :--- | +| Metrics | ✅ Live `prometheus_client` counters / gauges / histograms on the default registry; no HTTP exposition helper | ⚠️ Not wired — `MetricsProvider` is a user-supplied `Arc Option>` closure (`metrics.rs:16`); nothing populates it by default (LAB-521) | ✅ Node — live Prometheus metrics via the optional `prom-client` peer dep, custom registry supported, warns once and no-ops if absent (LAB-517) · ❌ Workers — `prom-client` is CI-excluded from the edge bundle¹ | ❌ | +| Structured logging | ✅ JSON ring-buffer logger with sensitive-data masking (`logging.py`) | ❌ No `log` / `tracing` integration — neither crate is a dependency (LAB-521) | ⚠️ Pluggable error-logger hook (`setLogger`, default `console.error`) — a logging seam, not a structured logger (LAB-517) | ❌ | +| Distributed tracing (OTel) | ❌ The span-shaped API accepts spans and discards them (`NoOpSpan`, `decorators/orchestrator.py:260-267`) — instrumentation built against it silently produces nothing | ❌ | ❌ | ❌ | +| SaaS telemetry headers (`X-CacheKit-L1-*`) | ✅ Auto | ⚠️ Reports `disabled` unless the user wires a `MetricsProvider` (LAB-521) | ✅ Auto-wired from the live L1/L2 hit/miss counters; an explicit user `metricsProvider` still wins | ❌ | + +--- + ## Developer Experience +*Audited against code 2026-08-04 (LAB-1400): py `config/decorator.py` + `decorators/intent.py`, rs `intents.rs` + `client.rs` + `cachekit-macros/src/lib.rs`, ts `intents.ts` + `cache.ts`. Records **semantics**, not just presence.* + | Feature | Python | Rust | TypeScript | PHP | | :--- | :---: | :---: | :---: | :---: | -| Decorator API (`@cache`) | ✅ | ✅ `#[cachekit]` proc-macro | N/A (functional `wrap()` API) | ❌ attributes | -| Intent-based presets | ✅ `.minimal` `.production` `.secure` `.io` | ✅ `CacheKit::minimal` `::production` `::secure` `::io` | ✅ `createCache.minimal()` etc. | ❌ | -| Builder API | ✅ | ✅ `CacheKit::builder()` / `from_env()` | ✅ | ❌ | +| Decorator API | ✅ `@cache` — wraps sync and async functions | ✅ `#[cachekit]` proc-macro — async fns returning `Result` only | N/A (functional `wrap()` API) | ❌ attributes | +| Intent-based presets | ✅ `.minimal` `.production` `.secure` `.io` (+ Python-only `.dev` `.test` `.local`) | ⚠️ `::minimal` `::production` `::encrypted` `::io` — **there is no `::secure` preset**, and only `::io` compiles on default features¹¹ | ✅ `createCache.minimal()` `.production()` `.secure()` `.io()` | ❌ | +| Builder API | ❌ No builder — `DecoratorConfig` presets + kwargs (frozen dataclass) + pydantic-settings, intentional¹² | ✅ `CacheKit::builder()` / `from_env()` | ❌ Options object on `createCache()` — no builder chain, no `from_env()`¹² | ❌ | | Async support | ✅ | ✅ | ✅ | ❌ | -| Sync support | ✅ | ✅ | ❌ | ✅ | -| WASM / CF Workers | N/A | ✅ `workers` feature (`?Send`, `Rc`) | ✅ `/workers` entrypoint (wasm32 core)¹ | N/A | -| pydantic-settings config | ✅ | N/A | N/A | N/A | +| Sync support | ✅ Same decorator wraps both | ❌ **Async-only** — every cache op is an `async fn` and the macro output only compiles on async fns¹¹ | ❌ | ✅ | +| WASM / CF Workers | N/A | ✅ `workers` feature (`?Send`, `Rc`) — needs `--no-default-features`¹⁹ | ✅ `/workers` entrypoint (wasm32 core — see [Cache Backends](#cache-backends) note ¹) | N/A | +| pydantic-settings config | ✅ `CACHEKIT_` env prefix, `SecretStr` master key | N/A — `from_env()` + `Zeroizing` is the Rust idiom | N/A — per-intent env fallback only, intentional | N/A | | Type hints / strict types | ✅ | ✅ | ✅ | ✅ PHP 8.1+ | +> ¹¹ Corrected 2026-08-04 (LAB-1400, from LAB-274): both cells were factually wrong. **`cachekit-rs` ships no `::secure` preset** — the encrypted preset is `CacheKit::encrypted(url, master_key)` (`intents.rs:158`), and `secure()` is a *post-build accessor* returning `Result` that errors unless encryption was configured (`client.rs:658`). Writing `CacheKit::secure(...)` does not compile. **Three of the four presets also require the non-default `redis` feature** — `::minimal` and `::production` are `#[cfg(feature = "redis")]` (`intents.rs:69`, `:108`) and `::encrypted` needs `redis` + `encryption` (`:157`), while the default feature set is `cachekitio` + `encryption` + `l1` + `reliability`, so on a default `cargo add cachekit-rs` only `::io` exists. **Rust is async-only**: `get`/`set`/`set_with_ttl` are all `async fn` (`client.rs:352`, `:531`, `:538`) and `#[cachekit]` expands only on async functions. +> +> ¹² Neither Python nor TypeScript has a builder; both were previously ✅. Python configures through `DecoratorConfig` intent constructors plus kwargs on a frozen dataclass, with pydantic-settings underneath — there is no `Builder` class or `builder()` method anywhere in `src/cachekit/`. TypeScript takes an options object. Both are deliberate. + +### Intent-preset semantics (parity, not presence) + +The four shared preset names configure **different things per SDK**. Each cell is code-verified; a cross-SDK user hits these as surprises. + +| Semantic | Python | Rust | TypeScript | +| :--- | :--- | :--- | :--- | +| Preset TTL defaults | **None — entries never expire** unless `ttl=` is passed | 300 / 600 / 600 / 3 600 s (`intents.rs:78`, `:118`, `:165`, `:214`) | 300 / 600 / 600 / 3 600 s | +| `minimal`: L1 | **on** (SWR / invalidation off) | **off** (`no_l1()`, `intents.rs:79`) | **on** (SWR / invalidation off) | +| `minimal`: integrity checksums | **off** (`integrity_checking=False`) | n/a — values carry no envelope | **on** (ByteStorage on by default) | +| `minimal`: reliability | circuit breaker / timeout off, backpressure on | **off** — the `reliability` layer is only attached by `production`/`encrypted`/`io` | circuit breaker neutered (∞ threshold), no retry, degradation off | +| Encrypted preset | `.secure(master_key=…)` — hex string, ≥64 hex chars; falls back to `CACHEKIT_MASTER_KEY` | `::encrypted(url, key: &[u8])` — **raw bytes, ≥32**; argument only, no env fallback | `.secure({ masterKey })` — hex; falls back to `CACHEKIT_MASTER_KEY` | +| Tenancy on the encrypted preset | `tenant_extractor` callable, `single_tenant_mode`, `fail_closed` tri-state | fixed `"default"` tenant (override via builder) | `tenantId` string | +| `io`: credentials | `CACHEKIT_API_KEY` env **only** | `api_key` argument **only** (env only via `from_env()`) | `apiKey` option **or** `CACHEKIT_API_KEY` | +| `CACHEKIT_MASTER_KEY` auto-enables encryption | **all presets** | only `CacheKit::from_env()` | only `createCache.secure()` | + +> [!WARNING] +> Preset names promise more parity than they deliver. The sharpest traps: Python preset entries **live forever** where Rust and TypeScript expire in 300–3 600 s; the master key is a **hex string** in py/ts but **raw bytes** in rs, so passing the same value across SDKs is a type error at best and a wrong key at worst; and the same `CACHEKIT_MASTER_KEY` activates encryption everywhere in Python, only in `from_env()` in Rust, and only in `.secure()` in TypeScript. All three do reject a *missing* key on the encrypted preset — Python raises, Rust returns `Err`, and `createCache.secure()` throws `ConfigurationError` (`intents-core.ts:240`). +> +> **`cache.secure.wrap()` guarantees nothing in TypeScript (LAB-513).** It is an unconditional alias for `cache.wrap()` — `secure = { wrap: (fn, options) => this.wrap(fn, options) }` (`cache-core.ts:832`, mirrored at `:873` for `withExecutionContext`) — and every intent, `minimal` and `production` and `io` included, is typed as `SecureCache` (`cache.ts:87`). Encryption applies only where an encryption manager was configured (`if (this.encryption)`, `cache-core.ts:486`), so moving sensitive values behind `cache.secure.wrap` on a cache **not** built by `createCache.secure()` stores them as plaintext with no error, no warning, and no type error. Only `createCache.secure()` turns encryption on. Python and Rust have no equivalent trap: py raises and rs's `secure()` accessor returns `Err` without configured encryption. + --- ## Protocol Compliance @@ -190,23 +279,23 @@ its spec: | Requirement | Python | Rust | TypeScript | PHP | | :--- | :---: | :---: | :---: | :---: | -| Key generation (Blake2b) | ✅ Compliant | N/A auto mode¹ — interop/v1 keygen ✅ merged ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)); `#[cachekit]` mints interop keys ([#35](https://github.com/cachekit-io/cachekit-rs/pull/35)) | ✅ Compliant | ⚠️ Untested | -| Wire format (ByteStorage) | ✅ Compliant² | ✅ Canonical (`cachekit-core`) — unused for stored values² | ✅ Compliant | ⚠️ Untested | -| Storage container (auto mode)² | CK v3 frame (Python-internal) | Plain MessagePack (`rmp` named) — no envelope | Bare ByteStorage envelope (default) | — | +| Key generation (Blake2b) | ✅ Compliant | N/A auto mode¹⁴ — interop/v1 keygen ✅ merged ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)); `#[cachekit]` mints interop keys ([#35](https://github.com/cachekit-io/cachekit-rs/pull/35)) | ✅ Compliant | ⚠️ Untested | +| Wire format (ByteStorage) | ✅ Compliant¹⁵ | ✅ Canonical (`cachekit-core`) — unused for stored values¹⁵ | ✅ Compliant | ⚠️ Untested | +| Storage container (auto mode)¹⁵ | CK v3 frame (Python-internal) | Plain MessagePack (`rmp` named) — no envelope | Bare ByteStorage envelope (default) | — | | Encryption (AES-256-GCM) | ✅ Compliant | ✅ Canonical (cachekit-core) | ✅ Compliant | ⚠️ Untested | | AAD v0x03 | ✅ Compliant (5 components — every auto serializer appends `original_type`; interop mode is the sole 4-component path) | ✅ Compliant (4 components) | ✅ Compliant (4 components) | ❌ Not implemented | | SaaS API | ✅ Compliant | ✅ Compliant (CachekitIO backend) | ✅ Compliant | ❌ Not implemented | -| Test vectors in CI³ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors | ⚠️ Pending | -| Interop mode ([spec](spec/interop-mode.md), opt-in) | ✅ Released — PyPI 0.14.0+⁴ ([#220](https://github.com/cachekit-io/cachekit-py/pull/220)) | ✅ Released — crates.io 0.4.0+ ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)) | ✅ Released — npm 0.1.3+ ([#71](https://github.com/cachekit-io/cachekit-ts/pull/71)) | ❌ Not implemented | +| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors | ⚠️ Pending | +| Interop mode ([spec](spec/interop-mode.md), opt-in) | ✅ Released — PyPI 0.14.0+¹⁷ ([#220](https://github.com/cachekit-io/cachekit-py/pull/220)) | ✅ Released — crates.io 0.4.0+ ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)) | ✅ Released — npm 0.1.3+ ([#71](https://github.com/cachekit-io/cachekit-ts/pull/71)) | ❌ Not implemented | > [!NOTE] -> ¹ "N/A" for Rust *auto-mode* key generation means `cachekit-rs` implements no auto-mode key format: `get`/`set` take caller-supplied keys. The `#[cachekit]` macro mints **interop/v1** keys via `interop_key` — required, compile-time-validated `interop = "operation"` and `namespace` attributes, byte-identical across SDKs ([cachekit-rs#35](https://github.com/cachekit-io/cachekit-rs/pull/35) / LAB-424; keygen itself merged in [#33](https://github.com/cachekit-io/cachekit-rs/pull/33)). The legacy RFC §3.1.5 keygen (`key::generate_cache_key`, `{namespace}:{blake2b256-hex}` — matched no protocol format, and WAS live in every `#[cachekit]` expansion despite the audit's "unused" premise, a proc-macro grep miss) is deleted outright in #35; upgrading is a full cache invalidation for `#[cachekit]` users. `cachekit-core` is a protocol primitive library with no keygen. +> ¹⁴ "N/A" for Rust *auto-mode* key generation means `cachekit-rs` implements no auto-mode key format: `get`/`set` take caller-supplied keys. The `#[cachekit]` macro mints **interop/v1** keys via `interop_key` — required, compile-time-validated `interop = "operation"` and `namespace` attributes, byte-identical across SDKs ([cachekit-rs#35](https://github.com/cachekit-io/cachekit-rs/pull/35) / LAB-424; keygen itself merged in [#33](https://github.com/cachekit-io/cachekit-rs/pull/33)). The legacy RFC §3.1.5 keygen (`key::generate_cache_key`, `{namespace}:{blake2b256-hex}` — matched no protocol format, and WAS live in every `#[cachekit]` expansion despite the audit's "unused" premise, a proc-macro grep miss) is deleted outright in #35; upgrading is a full cache invalidation for `#[cachekit]` users. `cachekit-core` is a protocol primitive library with no keygen. > -> ² Auto-mode **stored bytes** are SDK-internal and differ per SDK — see [wire-format.md → SDK Storage Containers](spec/wire-format.md#sdk-storage-containers-auto-mode). Python stores the ByteStorage envelope *inside* its CK v3 frame; `cachekit-rs` does not use the envelope for values at all (it uses `cachekit-core` only for encryption). Cross-SDK value compatibility is exclusively an [interop-mode](spec/interop-mode.md) property (protocol#11). +> ¹⁵ Auto-mode **stored bytes** are SDK-internal and differ per SDK — see [wire-format.md → SDK Storage Containers](spec/wire-format.md#sdk-storage-containers-auto-mode). Python stores the ByteStorage envelope *inside* its CK v3 frame; `cachekit-rs` does not use the envelope for values at all (it uses `cachekit-core` only for encryption). Cross-SDK value compatibility is exclusively an [interop-mode](spec/interop-mode.md) property (protocol#11). > -> ³ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, and — since LAB-423 — `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py)) against reference implementations. `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425). +> ¹⁶ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, `file-backend.json` ([`tools/file-backend-reference.py`](tools/file-backend-reference.py)), and — since LAB-423 — `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py)) against reference implementations. `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425). > -> ⁴ Version cells are **floors** (`X+`), not snapshots — they stay true as new versions publish; check the registry for the current release. Python's floor is the first *installable* one: interop merged under the `v0.13.0` tag, but neither `0.12.0` nor `0.13.0` was ever published to PyPI, so `0.14.0` is the earliest PyPI release containing interop mode. Do not "correct" this to 0.13.0 from the cachekit-py changelog alone. +> ¹⁷ Version cells are **floors** (`X+`), not snapshots — they stay true as new versions publish; check the registry for the current release. Python's floor is the first *installable* one: interop merged under the `v0.13.0` tag, but neither `0.12.0` nor `0.13.0` was ever published to PyPI, so `0.14.0` is the earliest PyPI release containing interop mode. Do not "correct" this to 0.13.0 from the cachekit-py changelog alone. --- @@ -217,7 +306,8 @@ its spec: - Hybrid Python-Rust architecture: decorators and orchestration in Python, ByteStorage and encryption in Rust (via PyO3) - The `cachekit-core` Rust crate is the canonical implementation for compression, checksums, and encryption -- 4 serializers: Standard (cross-language), Auto (Python-optimized), Orjson (JSON), Arrow (columnar) +- 4 registry serializers: Standard (cross-language), Auto (Python-optimized), Orjson (JSON, `[json]` extra), Arrow (columnar, `[data]` extra) — plus the non-registry interop/v1 serializer for cross-SDK mode +- Live Prometheus metrics (`prometheus_client`, default registry) + structured JSON logging with sensitive-data masking - Backends: Redis, Memcached, File (local), CacheKit SaaS — backend auto-detected from the environment: a single unambiguous selector, with `REDIS_URL`/localhost fallback, and ambiguous (multiple) selectors raising `ConfigurationError` (see [Cache Backends](#cache-backends)) - Config via pydantic-settings; secrets via `SecretStr` @@ -226,22 +316,36 @@ its spec:
Rust SDK (cachekit-rs) -- Published on crates.io as `cachekit-rs` v0.5.0 + `cachekit-macros` v0.5.0 -- Feature flags: `redis`, `cachekitio`, `encryption`, `l1`, `macros`, `workers` -- Backends: `RedisBackend` (fred), `CachekitIO` (reqwest), `WorkersCachekitIO` (CF Workers fetch) -- L1 cache via moka (native only, `l1` feature) -- `#[cachekit]` proc-macro for decorator-style caching +- Published on crates.io as `cachekit-rs` v0.6.0+ + `cachekit-macros` v0.6.0+; MSRV 1.85 +- Feature flags: `cachekitio`, `redis`, `memcached`, `file`, `encryption`, `l1`, `macros`, `workers`, `reliability`, `unsync` — default = `cachekitio` + `encryption` + `l1` + `reliability` (verified in the published 0.6.0 `.crate`) +- Backends: `RedisBackend` (fred), `CachekitIO` (reqwest), `WorkersCachekitIO` (CF Workers fetch), `MemcachedBackend`, `FileBackend` (both native-only cargo features) +- L1 cache via moka (native only, `l1` feature), with serve-stale + single-flight background refresh (LAB-728) +- Reliability tier (`reliability` feature, native, **on by default since 0.6.0**): retry with backoff + jitter on the `is_retryable` classification, circuit breaker, backpressure (semaphore + bounded queue), cold-miss single-flight with distributed fill locks, and macro-level graceful degradation (fail-open; `SecureCache` fail-closed) +- `#[cachekit]` proc-macro for decorator-style caching (async fns only) - `SecureCache` for zero-knowledge encrypted caching - SSRF protection, credential redaction, `Zeroizing` key material - WASM/Workers support: `?Send` + `Rc` paths via `cfg(target_arch = "wasm32")` -- Depends on `cachekit-core` for encryption primitives only (the envelope is unused for stored values — see [Compliance Status](#compliance-status) note ²); core-version rollout state is recorded in the cachekit-core note below +- Depends on `cachekit-core` for encryption primitives only (the envelope is unused for stored values — see [Compliance Status](#compliance-status) note ¹⁵); core-version rollout state is recorded in the cachekit-core note below
Rust Core (cachekit-core) -- Published on crates.io as `cachekit-core` v0.4.0 — the protocol 1.1 writer flip: `StorageEnvelope.compressed_data` now *emits* msgpack `bin` (`serde_bytes`); readers dual-decode both `bin` and the legacy array-of-ints ([spec/wire-format.md](spec/wire-format.md), [decisions/envelope-bin-encoding.md](decisions/envelope-bin-encoding.md)). Consumers: cachekit-py ≥ 0.17.0 ships it ([cachekit-py#249](https://github.com/cachekit-io/cachekit-py/pull/249)); released cachekit-rs 0.5.0 and cachekit-ts 0.1.4 still pin the 0.3 line — their 0.4.0 bumps ([cachekit-rs#53](https://github.com/cachekit-io/cachekit-rs/pull/53), [cachekit-ts#91](https://github.com/cachekit-io/cachekit-ts/pull/91)) are merged on main, unreleased +- Published on crates.io as `cachekit-core` v0.4.0+ — the protocol 1.1 writer flip: `StorageEnvelope.compressed_data` now *emits* msgpack `bin` (`serde_bytes`); readers dual-decode both `bin` and the legacy array-of-ints ([spec/wire-format.md](spec/wire-format.md), [decisions/envelope-bin-encoding.md](decisions/envelope-bin-encoding.md)). **Dual-read is mutual, so the flip is not a breaking change and needs no rollout ordering** — a pre-flip reader shape (plain `Vec`, no `serde_bytes`) deserializes `bin` wire, and a 1.1 reader deserializes legacy wire. That is CI-asserted in the canonical implementation, not inferred: `cachekit-core/tests/dual_decode.rs` decodes every byte-pinned vector and its `*_bin` twin through **both** reader shapes (`assert_all_readers_decode`), with `bin8`/`bin16`/`bin32` width tiers covered. Per-SDK rollout of the *writer*, each row verified inside the published artifact (2026-08-04) rather than from a repo branch: + +| SDK | Published release | Embedded / resolved `cachekit-core` | Writes `bin`? | +| :--- | :--- | :--- | :--- | +| cachekit-py | 0.17.1 (PyPI) | 0.4.0 ([#249](https://github.com/cachekit-io/cachekit-py/pull/249)) | ✅ since 0.17.0 | +| cachekit-rs | 0.6.0 (crates.io, 2026-08-03T14:58Z) | `0.4` per the published `Cargo.toml` ([#53](https://github.com/cachekit-io/cachekit-rs/pull/53)) | ✅ since 0.6.0 | +| cachekit-ts — NAPI path | 0.1.5 (npm) → exact pin `@cachekit-io/cachekit-core-ts@0.1.2` | **0.2.0** (all five platform `.node` binaries) | ❌ legacy only | +| cachekit-ts — Workers path | 0.1.5 (npm) → exact pin `@cachekit-io/cachekit-core-wasm@0.1.1` | **0.3.0** | ❌ legacy only | + + **TypeScript has not shipped the flip on either path.** `@cachekit-io/cachekit@0.1.5` (published 2026-08-03T11:15Z) carries dependency pins byte-identical to 0.1.4's — exact, caret-free pins on `@cachekit-io/cachekit-core-ts@0.1.2` (published 2026-05-17, before core 0.3.0 existed) and `@cachekit-io/cachekit-core-wasm@0.1.1`. `@cachekit-io/cachekit-core-wasm@0.1.2` does embed core 0.4.0, but it published at 14:28Z — **3h13m after** ts 0.1.5 — so no published `cachekit` release pins it. [cachekit-ts#91](https://github.com/cachekit-io/cachekit-ts/pull/91) bumped the source pin and is in 0.1.5's tree, but the NAPI addon it consumes was never republished, so the shipped binary is unchanged. + + **This lag is a missing optimisation, not a hazard — there is nothing to sequence.** Because dual-read is mutual (above), a ts instance on core 0.2.0 reads `bin` written by any newer instance, and the legacy envelopes it writes stay readable by everything forever. What TypeScript forgoes until `cachekit-core-ts` is republished is the `bin` encoding's size saving on its own writes (~35% on incompressible payloads, ≤ +1 B on tiny ones). Nor is it a cross-SDK concern: auto-mode stored bytes are SDK-internal and no SDK reads another's ([protocol#11](https://github.com/cachekit-io/protocol/issues/11)). + + *An earlier revision of this note claimed core ≤ 0.3.0 readers **reject** `bin` and told operators to sequence the ts republish ahead of any fleet upgrade. That was false in the dangerous direction — it manufactured a migration risk that does not exist — and it contradicted this repo's own `CHANGELOG.md` ("**Not a breaking change** — dual-read is mutual in both directions") and the verified compatibility table in [spec/wire-format.md](spec/wire-format.md#encoding-compatibility-dual-read). Recorded because the operational advice was the visible part.* - Provides: `ByteStorage`, `ZeroKnowledgeEncryptor`, `derive_domain_key`, `derive_tenant_keys` - Dependencies: `lz4_flex`, `xxhash-rust`, `ring` (native) / `aes-gcm` (wasm32), `hkdf`, `sha2`, `rmp-serde` - Formally verified security properties via Kani @@ -262,8 +366,8 @@ its spec: - Circuit breaker (rolling window), retry (exponential backoff + jitter), graceful degradation - Distributed locking via Redis and CacheKit SaaS backends (`LockableBackend`) - Intent-based API: `createCache.minimal()`, `.production()`, `.secure()`, `.io()` -- 567 tests, 94.79% statement coverage (measured on the LAB-519 branch, cachekit-ts#77) -- Dual output: ESM + CJS, Node 20+ +- Live Prometheus metrics via the optional `prom-client` peer dep; pluggable error-logger hook (`setLogger`) +- Dual output: ESM + CJS, **Node 22+** (`packages/cachekit/package.json` `engines.node: ">=22.0.0"`)
diff --git a/spec/wire-format.md b/spec/wire-format.md index 8dfdbc2..905026e 100644 --- a/spec/wire-format.md +++ b/spec/wire-format.md @@ -32,7 +32,15 @@ This document specifies two layers: 1. **The ByteStorage envelope** — the LZ4 + xxHash3-64 container implemented by `cachekit-core` and exposed to SDKs. This layer is byte-canonical and pinned by - [`test-vectors/wire-format.json`](../test-vectors/wire-format.json). + [`test-vectors/wire-format.json`](../test-vectors/wire-format.json), which is + enforced in CI in two independent places (LAB-423): this repo's `verify.yml` + runs [`tools/wire-format-reference.py verify`](../tools/wire-format-reference.py) + against the stdlib-only reference implementation, and the canonical + implementation [`cachekit-core`](https://github.com/cachekit-io/cachekit-core) + vendors the file sha256-pinned in `tests/wire_format_vectors.rs`, asserting + decode byte-identity for every vector and re-encode byte-identity for the + canonical `*_bin` vectors only — legacy array-of-integers vectors are + decode-only, retained as legacy-read proof. 2. **[SDK storage containers](#sdk-storage-containers-auto-mode)** — what each SDK *actually stores* in a backend in default (auto) mode. These differ per SDK, are **SDK-internal**, and are documented here so their bytes are identifiable — not so @@ -85,11 +93,22 @@ accept the legacy encoding below — a stored envelope never expires on a schedu so legacy-read support is permanent. > [!NOTE] -> **Implementation status:** the canonical `bin` encoding **is** shipped — -> `cachekit-core` v0.4.0 carries the writer flip, and `cachekit` (Python) -> ≥ 0.17.0 emits it. The released `cachekit-rs` 0.5.0 and `cachekit-ts` 0.1.4 -> lines still pin core 0.3 and therefore still write the legacy encoding, so -> readers encounter both on the wire today. Per-SDK rollout state is tracked in +> **Implementation status** (verified against published artifacts, 2026-08-04): +> the canonical `bin` encoding **is** shipped — `cachekit-core` v0.4.0 carries the +> writer flip, `cachekit` (Python) ≥ 0.17.0 emits it, and `cachekit-rs` ≥ 0.6.0 +> resolves core `0.4` and emits it too. **TypeScript does not yet emit it on +> either path:** published `@cachekit-io/cachekit` 0.1.5 pins +> `@cachekit-io/cachekit-core-ts@0.1.2` (whose native addons embed core **0.2.0**) and +> `@cachekit-io/cachekit-core-wasm@0.1.1` (core **0.3.0**), so it writes and reads legacy only. +> Readers therefore encounter both encodings on the wire today — which is fine, +> and why the flip is **not** a breaking change: dual-read is mutual, so a +> pre-flip reader shape accepts `bin` and a 1.1 reader accepts legacy, per the +> toolchain-verified table under +> [Encoding compatibility](#encoding-compatibility-dual-read) and the permanent +> CI proof in `cachekit-core/tests/dual_decode.rs`. A lagging SDK therefore +> needs no rollout ordering; it simply forgoes the size saving on its own writes +> until it picks up core ≥ 0.4.0. Per-SDK rollout state, with the embedded-core +> evidence per artifact, is tabulated in > [sdk-feature-matrix.md](../sdk-feature-matrix.md#architecture-notes). `checksum` (element `[1]`) is **deliberately excluded** from the `bin` encoding: it diff --git a/tools/check-version-floors.py b/tools/check-version-floors.py new file mode 100644 index 0000000..c913de4 --- /dev/null +++ b/tools/check-version-floors.py @@ -0,0 +1,174 @@ +#!/usr/bin/env python3 +"""Fail if the SDK Overview table states a package version as a snapshot. + +Versions in that table answer "which release do I need?", so they must be floors +(`0.6.0+`) — a floor stays true as new releases publish, while a snapshot is wrong +the moment the next one lands and misleads silently until someone notices. That is +the one failure this guard exists for, and the only one it can see. + +**Scope, and what this does NOT catch.** It checks the SDK Overview table only. It +cannot tell whether a floor is *accurate* (that means opening the published +artifact — rules 1-3 of the decision record), and it does not police versions +elsewhere in the matrix, because those are a mix it cannot safely tell apart: some +are floors too (Compliance Status, the Architecture Notes release bullets), others +are exact facts about one specific artifact (an embedded `cachekit-core-0.2.0`, a +caret-free npm pin) that appending `+` to would make false. Telling those apart +needs the claim's intent, which a regex cannot read. + +So of the six matrix incidents behind this work it catches one facet of one: the +`cachekit-rs` 0.5.0 snapshot that sat in this table against a published 0.6.0 — +not even the six mis-marked Reliability cells from that same event. The other five +incidents are invisible to it too. Rationale and the full incident list: +decisions/matrix-version-verification.md + +Fails closed: if the table cannot be located or parsed, that is an error, not a +pass — a guard that silently checks nothing is worse than no guard. + +Usage: python3 tools/check-version-floors.py [path] (exit 1 on violations) +""" + +from __future__ import annotations + +import re +import sys +from pathlib import Path +from typing import NoReturn + +DEFAULT_TARGET = "sdk-feature-matrix.md" +HEADING = "## SDK Overview" +VERSION_HEADER = "version" + +# A GFM separator cell: ---, :---, ---:, :---: +SEPARATOR_CELL = re.compile(r"^:?-{3,}:?$") +# A floor: 0.6.0+, 1.2.3-rc.1+ +FLOOR = re.compile(r"^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?\+$") +# Placeholder for an unreleased SDK. Escaped rather than literal because the three +# characters are visually indistinguishable inside a character class: U+2014 EM DASH, +# U+2013 EN DASH, U+002D HYPHEN-MINUS. A cell of 1-3 of them means "no release yet"; +# no version can hide in it. +PLACEHOLDER = re.compile("^[\u2014\u2013\u002d]{1,3}$") +SEMVER = re.compile(r"^\d+\.\d+\.\d+") +# Markdown emphasis and footnote markers a version cell may legitimately carry. +DECORATION = re.compile(r"[`*_\u2070\u00b9\u00b2\u00b3\u2074-\u2079]") + + +def fail(msg: str) -> NoReturn: + print(f"check-version-floors: {msg}", file=sys.stderr) + sys.exit(1) + + +def split_row(line: str) -> list[str]: + """Cells of a GFM table row. Tolerates a missing leading/trailing pipe.""" + return [c.strip() for c in line.strip().strip("|").split("|")] + + +def overview_table(text: str) -> tuple[int, list[tuple[int, list[str]]]]: + """(version column index, [(line number, cells)]) for the SDK Overview table.""" + lines = text.splitlines() + try: + start = next(i for i, raw in enumerate(lines) if raw.strip() == HEADING) + except StopIteration: + fail(f"{HEADING!r} section not found; cannot verify anything") + + header: list[str] | None = None + version_idx = -1 + table_ended = False + rows: list[tuple[int, list[str]]] = [] + + for offset, line in enumerate(lines[start + 1 :], start=start + 2): + stripped = line.strip() + if stripped.startswith("## "): + break + if "|" not in stripped: + # Blank lines and the `---` section rule before the table. Once the + # table has started, a non-pipe line ends it — but keep scanning so a + # SECOND table under this heading is caught rather than ignored. + if header is not None and rows: + table_ended = True + continue + + if table_ended: + fail( + f"a second table appears under {HEADING!r} (line {offset}) — this " + "checker verifies the first one only, so refusing to report OK on " + "a section it cannot fully account for" + ) + + cells = split_row(stripped) + + if header is None: + header = cells + lowered = [c.lower() for c in cells] + if VERSION_HEADER not in lowered: + fail( + f"{HEADING!r} table header has no {VERSION_HEADER!r} column " + f"(saw {cells}) — refusing to guess which column holds versions" + ) + version_idx = lowered.index(VERSION_HEADER) + continue + + # The separator row: every cell is a GFM alignment marker. + if cells and all(SEPARATOR_CELL.match(c) for c in cells): + continue + + rows.append((offset, cells)) + + if header is None: + fail(f"no table found under {HEADING!r}") + if not rows: + fail(f"{HEADING!r} table has no data rows") + return version_idx, rows + + +def main() -> int: + target = Path(sys.argv[1] if len(sys.argv) > 1 else DEFAULT_TARGET) + if not target.is_file(): + fail(f"{target} not found") + + version_idx, rows = overview_table(target.read_text(encoding="utf-8")) + + violations: list[tuple[int, str, str]] = [] + for lineno, cells in rows: + sdk = cells[0] if cells else "?" + if len(cells) <= version_idx: + violations.append( + (lineno, sdk, f"row has {len(cells)} cells, no version column") + ) + continue + + raw = cells[version_idx] + value = DECORATION.sub("", raw).strip() + + if PLACEHOLDER.match(value) or FLOOR.match(value): + continue + if not value: + violations.append((lineno, sdk, "version cell is empty")) + elif SEMVER.match(value): + violations.append( + (lineno, sdk, f"{raw!r} is a bare snapshot — write it as a floor") + ) + else: + violations.append((lineno, sdk, f"{raw!r} is not a floor or placeholder")) + + if not violations: + print( + f"check-version-floors: OK — {len(rows)} SDK Overview rows, " + "every version a floor or placeholder" + ) + return 0 + + print( + f"check-version-floors: {len(violations)} non-floor version(s) in the " + f"{target} SDK Overview table.\n" + "Versions there must be floors ('0.6.0+'), not snapshots ('0.6.0'), so they\n" + "stay true as new releases publish. See\n" + "decisions/matrix-version-verification.md.\n", + file=sys.stderr, + ) + for lineno, sdk, detail in violations: + print(f" {target}:{lineno}: {sdk} — {detail}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/test_check_version_floors.py b/tools/test_check_version_floors.py new file mode 100644 index 0000000..cf9ccdd --- /dev/null +++ b/tools/test_check_version_floors.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python3 +"""Mutation tests for check-version-floors.py. + +The guard's own history is why this exists: the first version passed a snapshot +hidden behind an ASCII-hyphen placeholder, passed an empty version cell, and +rejected a perfectly valid backticked floor. A guard nobody tested against +mutations is a guard that reports OK. + +Run: python3 tools/test_check_version_floors.py (exit 1 on any failure) +""" + +from __future__ import annotations + +import subprocess +import sys +import tempfile +from collections.abc import Callable +from pathlib import Path + +HERE = Path(__file__).resolve().parent +CHECKER = HERE / "check-version-floors.py" +MATRIX = HERE.parent / "sdk-feature-matrix.md" + +RS_FLOOR = "| cachekit-rs | `cachekit-rs` (crates.io) | 0.6.0+ | Rust 1.85+ | ✅ Production |" +PHP_ROW = "| cachekit-php | \u2014 | \u2014 | PHP 8.1+ | \U0001f51c Development |" + +# (name, mutate(text) -> text, expected_exit) +CASES: list[tuple[str, Callable[[str], str], int]] = [ + ("unmodified matrix", lambda t: t, 0), + # --- must be CAUGHT (exit 1) --- + ("bare snapshot", lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "0.6.0")), 1), + ( + "snapshot + ASCII-hyphen placeholders in another row", + lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "0.6.0")).replace( + PHP_ROW, "| cachekit-php | - | - | PHP 8.1+ | 🔜 Development |" + ), + 1, + ), + ( + "snapshot in a row that itself holds a hyphen cell", + lambda t: t.replace(RS_FLOOR, "| cachekit-rs | - | 0.6.0 | Rust 1.85+ | - |"), + 1, + ), + ("empty version cell", lambda t: t.replace(RS_FLOOR, "| cachekit-rs | `cachekit-rs` | | Rust 1.85+ | ✅ |"), 1), + ("row trimmed below the version column", lambda t: t.replace(RS_FLOOR, "| cachekit-rs | `cachekit-rs` |"), 1), + ( + "snapshot on a row with no leading pipe", + lambda t: t.replace(RS_FLOOR, "cachekit-rs | `cachekit-rs` (crates.io) | 0.6.0 | Rust 1.85+ | ✅ |"), + 1, + ), + ("prerelease snapshot", lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "0.7.0-rc.1")), 1), + ("garbage version", lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "latest")), 1), + # --- must PASS (exit 0): legitimate forms that are still floors --- + ("backticked floor", lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "`0.6.0+`")), 0), + ("bold floor", lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "**0.6.0+**")), 0), + ("prerelease floor", lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "0.7.0-rc.1+")), 0), + ( + "en-dash placeholder", + lambda t: t.replace(PHP_ROW, "| cachekit-php | \u2013 | \u2013 | PHP 8.1+ | \U0001f51c |"), + 0, + ), + ( + "floor carrying a footnote marker", + lambda t: t.replace(RS_FLOOR, RS_FLOOR.replace("0.6.0+", "0.6.0+¹⁷")), + 0, + ), + ( + "snapshot hidden in a second table under the same heading", + lambda t: t.replace( + PHP_ROW, + PHP_ROW + + "\n\nAnd a second table:\n\n| SDK | Package | Version |\n| :--- | :--- | :--- |\n" + "| cachekit-rs | `cachekit-rs` | 0.5.0 |", + ), + 1, + ), + # --- must ERROR rather than silently pass --- + ("heading removed", lambda t: t.replace("## SDK Overview", "## Overview Of SDKs"), 1), + ( + "version column renamed away", + lambda t: t.replace("| SDK | Package | Version | Language | Status |", "| SDK | Package | Rev | Language | Status |"), + 1, + ), +] + + +def run(text: str) -> int: + with tempfile.NamedTemporaryFile("w", suffix=".md", delete=False, encoding="utf-8") as fh: + fh.write(text) + path = fh.name + try: + return subprocess.run( + [sys.executable, str(CHECKER), path], capture_output=True, text=True + ).returncode + finally: + Path(path).unlink(missing_ok=True) + + +def main() -> int: + base = MATRIX.read_text(encoding="utf-8") + failures = [] + for name, mutate, expected in CASES: + mutated = mutate(base) + if name != "unmodified matrix" and mutated == base: + failures.append(f"{name}: mutation was a no-op; the anchor text moved, fix this test") + continue + got = run(mutated) + verdict = "ok" if got == expected else "FAIL" + print(f" [{verdict}] {name}: expected exit {expected}, got {got}") + if got != expected: + failures.append(f"{name}: expected {expected}, got {got}") + + if failures: + print(f"\n{len(failures)} case(s) failed:", file=sys.stderr) + for f in failures: + print(f" - {f}", file=sys.stderr) + return 1 + print(f"\nall {len(CASES)} cases passed") + return 0 + + +if __name__ == "__main__": + sys.exit(main())