docs: document the two-tier storage and pinning model

README.md

@@ -445,6 +445,7 @@ Read:
 
 - [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md)
 - [`docs/ECONOMICS.md`](docs/ECONOMICS.md)
+- [`docs/STORAGE-AND-PINNING.md`](docs/STORAGE-AND-PINNING.md)
 
 Use a dedicated low-balance operator wallet. Do not use a treasury wallet as the heartbeat key.
 

docs/STORAGE-AND-PINNING.md

@@ -0,0 +1,89 @@
+# Storage and pinning
+
+How a gitlawb node stores git objects and keeps them available. This documents
+the behavior implemented in `crates/gitlawb-node/src/ipfs_pin.rs` and
+`crates/gitlawb-node/src/pinata.rs`; it does not change any behavior.
+
+## Two-tier model
+
+After a push lands, new git objects are pinned to up to two independent sinks.
+Both are **opt-in and independent** — a node runs fine with neither, either, or
+both configured.
+
+| Tier | Sink | Module | Enabled by | Purpose |
+|------|------|--------|-----------|---------|
+| Hot  | Local Kubo (IPFS) | `ipfs_pin.rs` | `GITLAWB_IPFS_API` set | Node-local availability; the node is an HTTP client of a co-located Kubo daemon |
+| Warm | Pinata (Filecoin-backed) | `pinata.rs` | `GITLAWB_PINATA_JWT` set | Off-node durability + public IPFS gateway reachability |
+
+If a sink's config value is empty, its **pin** paths are no-ops — so leaving
+`GITLAWB_PINATA_JWT` unset disables warm-tier *uploads*, though not the tail's
+cost: the post-push Pinata tail still takes a global pin permit and runs the full
+object-list re-derivation walk before the upload step returns empty, since its
+gate is the announce/walk state, not the JWT. Note the no-op applies to pinning
+only, not reads: the hot-tier read path (`ipfs_pin::cat`) returns an error rather
+than a no-op when `GITLAWB_IPFS_API` is unset, so a node that serves the
+encrypted-blob read endpoint needs Kubo configured.
+
+## Configuration
+
+| Env var | Default | Meaning |
+|---------|---------|---------|
+| `GITLAWB_IPFS_API` | `""` (disabled) | Base URL of the local Kubo HTTP API, e.g. `http://127.0.0.1:5001` |
+| `GITLAWB_PINATA_JWT` | `""` (disabled) | Pinata bearer JWT enabling the warm tier |
+| `GITLAWB_PINATA_UPLOAD_URL` | `https://uploads.pinata.cloud/v3/files` | Pinata v3 upload endpoint |
+| `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | `8` | Cap on concurrent post-push pin loops across all repos |
+
+## How pinning runs
+
+Pinning happens **after** a push is accepted, not on the push's critical path:
+
+- The **hot** tier pins inline in the post-push encrypt/pin task.
+- The **warm** (Pinata) tier runs in a spawned replication tail, so a slow or
+  unreachable Pinata never blocks the pusher.
+
+Both tiers share a single global **pin admission semaphore**
+(`max_concurrent_pin_tasks`). The pool **defers rather than sheds**: when it is
+saturated, a pin loop waits for a slot instead of dropping the pin. Each batch is
+bounded by `PIN_BATCH_BUDGET` (120s) so the pin batch itself cannot hold a slot
+indefinitely. The Pinata tail re-derives its object list only *after* acquiring a
+slot, which bounds *its* outstanding memory to O(refs) rather than
+O(pushes × objects) — but that bound is the warm tail's alone. A **hot-tier** pin
+loop takes an owned object list materialized by its caller *before* it waits for
+a slot, so every hot loop parked on a saturated pool is holding its full list:
+the hot side of the parked-list memory is O(pushes × objects), capped per repo by
+`EncryptInflight` but not across repos. Do not size a node's memory from the
+semaphore knob (README says the same).
+Note the budget covers the pin batch, not the preceding object-list re-derivation
+walk: that walk holds the slot too, and bounds only each child git process
+individually (no aggregate deadline).
+
+De-duplication is per sink and best-effort, backed by the single `pinned_cids`
+table: the hot tier keys on its `cid`/`sha256_hex` rows and the warm tier on the
+nullable `pinata_cid` column, so later pushes normally skip objects whose
+successful pin is already recorded. The isolation is one-directional: a hot-only
+pin does not suppress the warm tier (`pinata_cid` stays null), but the hot tier's
+skip check is row *presence*, and a warm-tier pin inserts the row — so an object
+warm-pinned while `GITLAWB_IPFS_API` was unset is skipped by the hot tier after
+Kubo is configured, until something re-pins it. The check-upload-record sequence is not atomic,
+so concurrent post-push tasks for the same object, or a failure to record after a
+successful upload, can still cause a repeat upload attempt.
+
+## Durability notes
+
+- With only the **hot** tier, availability depends on the node (and any IPFS
+  peers that have fetched the CIDs). If the node is down and no peer holds the
+  objects, they are unreachable until it returns.
+- The **warm** tier adds off-node durability via Pinata's Filecoin-backed
+  storage and makes objects reachable through the public IPFS gateway.
+- Running **both** gives a node-local hot copy plus an off-node warm copy.
+
+> The two sinks are currently invoked as separate call paths. Unifying them
+> behind a single pluggable backend interface (to add providers such as direct
+> Filecoin deals or self-hosted clusters without touching push logic) is tracked
+> separately.
+
+## See also
+
+- [RUN-A-NODE.md](RUN-A-NODE.md) — provisioning and running a node
+- `crates/gitlawb-node/src/ipfs_pin.rs` — hot-tier implementation
+- `crates/gitlawb-node/src/pinata.rs` — warm-tier implementation