docs: registry-first README for @corbits/embedding (CL-8775)

ARCHITECTURE.md

@@ -0,0 +1,108 @@
+# Architecture
+
+The package is two layers: a one-shot JSON transport, and an embedding client
+built on it. There is no provider registry. Every call is HTTP POST to
+`{baseURL}/embeddings`.
+
+## Surfaces
+
+Callers import from `@corbits/embedding`. The barrel is the public surface:
+
+| Export | Role |
+| --- | --- |
+| `embedTexts` | Batch texts, post sequentially, return `number[][]` in input order |
+| `probeEmbedDims` | Embed one probe string; report the vector length that came back |
+| `EmbedConfig` / `EmbedConfigSchema` | Endpoint, model, and optional knobs |
+| `EmbedOptions` | Harness deps plus optional retry, `Retry-After` override, abort |
+| `runJSONRequest` | Classified, retried JSON POST (sibling clients reuse this) |
+| `extractRetryAfterMs` | Default `Retry-After` reader (seconds or HTTP-date) |
+| `ModelRequestError` | Transport, HTTP, or non-JSON 200 — `reason` is the classified `InferenceError` |
+
+Quickstart shape (matches the README):
+
+```ts
+import { createDefaultScheduler } from "@intx/inference";
+import { embedTexts, probeEmbedDims } from "@corbits/embedding";
+
+const deps = { fetch, scheduler: createDefaultScheduler() };
+const [vector] = await embedTexts(
+  ["hello world"],
+  { baseURL: "http://localhost:11434/v1", model: "nomic-embed-text" },
+  { deps },
+);
+```
+
+`probeEmbedDims(config, { deps })` is the same config and deps; it is not a
+separate protocol.
+
+## Request dependencies
+
+The transport reads only `fetch` and `scheduler` from the harness
+(`RequestDependencies`). That is deliberately narrower than Interchange
+`Dependencies`, whose `adapters` registry a caller should not have to
+assemble in order to embed a string. A real `Dependencies` still satisfies
+the pick structurally.
+
+Time for retry backoff comes from `deps.scheduler`, not wall-clock globals,
+so a virtual-clock test scheduler drives the loop deterministically. The
+HTTP-date form of `Retry-After` is the exception: it names an absolute
+instant and is resolved against wall time.
+
+## Control flow
+
+```
+embedTexts(texts, config, options)
+  → split on batchSize (default 32)
+  → for each batch, sequentially:
+        build POST {baseURL}/embeddings
+        runJSONRequest (classify + retry)
+        parse body: place by echoed index, reject short / duplicate / OOB
+  → concatenate batch results in input order
+```
+
+`probeEmbedDims` is `embedTexts` of one fixed probe string, then
+`vector.length`.
+
+Batches are sequential, not concurrent: the retry policy is per request, not
+per burst, and a rate-limited provider must not see the whole corpus at once.
+
+## Reply integrity
+
+The OpenAI-compatible reply is not promised in request order. Placement is
+by the echoed `index`, never by array position. A reordered reply is
+reordered back; a short reply, a duplicate index, or a missing slot is
+rejected. Callers pair returned vectors with their own texts positionally —
+a silent mispair would persist the wrong vector.
+
+`embedding` on the wire is either `number[]` (`float`) or a base64-packed
+little-endian float32 buffer (`base64`). The client always returns
+`number[]`.
+
+## Transport vs `runInference`
+
+`runInference` is turn-shaped and streams SSE. An embedding is one-shot JSON.
+What carries over is everything around the body: the single `deps.fetch`
+path, the `InferenceError` classifiers (`classifyHTTPError`,
+`classifyNetworkError`, `classifyAbortError`, `classifyProtocolMismatch`),
+and `createDefaultRetryPolicy` (back off retryables, abort the rest). A 429
+is therefore classified and backed off as a chat 429 would be; a
+`credential_failure` aborts immediately.
+
+`runJSONRequest` is duplicated, modulo comments, in `@corbits/reranking`. It
+is not shared infrastructure yet — the intended home is `@intx/inference` as
+a non-streaming sibling of `runInference`. Until then, `ModelRequestError` is
+a distinct class in each package: `instanceof` does not hold across the two.
+Catch on `error.name === "ModelRequestError"`.
+
+## Failure modes
+
+- Transport, HTTP status, or a 200 whose body is not JSON →
+  `ModelRequestError` with classified `reason` and the URL.
+- Malformed embeddings JSON, wrong count, bad index → thrown `Error` (not
+  `ModelRequestError`); the protocol succeeded, the payload did not.
+- `batchSize` not an integer `>= 1` → `RangeError` before any request
+  (`i += size` of 0 would never advance).
+- Empty `texts` → `[]`, no request.
+- Caller `signal` or per-attempt timeout (default 30s) abort the attempt;
+  aborting mid-delay wakes immediately and the next attempt fails its
+  entry-time signal check.

IMPLEMENTATION.md

@@ -0,0 +1,113 @@
+# Implementation
+
+## Package
+
+`@corbits/embedding` is an ESM TypeScript package. Development runtime is
+Bun `>= 1.2.0`. Node `>= 24` consumes built `dist/`; native Node does not
+load this package's TypeScript source.
+
+```json
+"exports": {
+  ".": {
+    "intx-src": "./src/index.ts",
+    "types": "./dist/index.d.ts",
+    "default": "./dist/index.js"
+  }
+}
+```
+
+`bun run build` is `tsc -p tsconfig.build.json` (`outDir: dist`,
+`rootDir: src`, tests excluded). Published files are `dist`, `README.md`,
+and `LICENSE`. Install from the registry:
+
+```bash
+npm add @corbits/embedding
+pnpm add @corbits/embedding
+yarn add @corbits/embedding
+bun add @corbits/embedding
+```
+
+## Dependencies
+
+| Package | Use |
+| --- | --- |
+| `@intx/inference` | `BuiltRequest`, `createDefaultRetryPolicy`, error classifiers, `createDefaultScheduler` (caller-side) |
+| `@intx/types` | `RetryPolicy`, `InferenceError` |
+| `@intx/log` | Transitive Interchange surface (not imported by this package's sources) |
+| `arktype` | `EmbedConfigSchema` and the embeddings reply shape |
+
+Dev: `@intx/inference-testing`, `@types/bun`, `prettier`, `typescript` 5.9.
+
+## HTTP
+
+Each batch is:
+
+- `POST {baseURL}/embeddings` (`baseURL` already includes the version
+  prefix, e.g. `http://localhost:11434/v1` or `https://api.openai.com/v1`)
+- `content-type: application/json`
+- `authorization: Bearer {apiKey}` only when `apiKey` is set
+- `redirect: "manual"`
+- body:
+
+```json
+{
+  "model": "<config.model>",
+  "input": ["…"],
+  "dimensions": 256,
+  "encoding_format": "float"
+}
+```
+
+`dimensions` and `encoding_format` are omitted unless configured. Models
+without Matryoshka reject `dimensions` outright, so an unset knob must not
+appear. Client field is `encodingFormat`; wire field is `encoding_format`
+(`float` | `base64`). Default `batchSize` is 32 (OpenAI rejects arrays over
+2048). Default per-attempt timeout is 30s (`AbortSignal.timeout`, combined
+with a caller `signal` via `AbortSignal.any`).
+
+A 200 is read as text then `JSON.parse`. `res.json()` is not used: an HTML
+interstitial or truncated body in front of a local model server must classify
+as protocol mismatch, not escape as `SyntaxError`.
+
+## arktype
+
+`EmbedConfigSchema` (`src/embed.ts`):
+
+- required: `baseURL: string`, `model: string`
+- optional: `apiKey`, `dimensions` (`number > 0`), `encodingFormat`
+  (`'float'|'base64'`), `batchSize` (`number.integer >= 1`), `timeoutMs`
+  (`number > 0`)
+
+Reply: `{ data: { index: number, embedding: number[] | string }[] }`.
+Base64 strings decode with `atob` → `Uint8Array` → little-endian
+`Float32Array` → `number[]`.
+
+`embedTexts` also range-checks `batchSize` at runtime so a value that
+bypassed the schema cannot stall `batches` (`i += size`).
+
+## Errors and retry
+
+`runJSONRequest` (`src/request.ts`) loops:
+
+1. `deps.fetch` POST
+2. non-OK → `classifyHTTPError(status, detail, detail, retryAfterMs)`
+3. network throw → `classifyNetworkError` or `classifyAbortError`
+4. JSON parse fail → `classifyProtocolMismatch`
+5. `retryPolicy` (default `createDefaultRetryPolicy`) → `abort` throws
+   `ModelRequestError`, else sleep `delayMs` on `deps.scheduler`
+
+`extractRetryAfterMs` reads `Retry-After` as seconds or HTTP-date, clamped
+at zero. Callers override via `EmbedOptions.extractRetryAfterMs` (same
+signature as Interchange's unexported extractor; re-declared so this
+package depends only on the published surface).
+
+`ModelRequestError.name` is `"ModelRequestError"`. Discriminate on the
+name when catching both this package and `@corbits/reranking`.
+
+## Tests
+
+`bun run test` is `bun test ./src`. Coverage is the client contract: empty
+input, URL and body knobs, auth header presence, batch split and order,
+`batchSize` rejection, malformed / short / duplicate-index replies,
+index-not-position placement, 429 retry, 401 abort, `probeEmbedDims`,
+overridable `Retry-After`.

PRODUCT.md

@@ -0,0 +1,48 @@
+# Product
+
+`@corbits/embedding` is the client you call when you need vectors from an
+OpenAI-compatible `/v1/embeddings` endpoint. Given a `baseURL`, a model, and
+some texts, `embedTexts` batches them, posts each batch, validates the reply,
+and returns vectors in input order.
+
+## Why it exists
+
+Embedding is not a chat turn. Callers persist vectors next to text and later
+retrieve by similarity. A wrong pairing, an assumed dimension, or a one-off
+HTTP client that does not retry a 429 the way a chat call does all corrupt
+that store. This package is the one place that does the boring work: batch,
+validate, place by echoed `index`, and fail loudly instead of silently
+mispairing.
+
+One wire format, not a provider switch. OpenAI, Ollama, TEI, vLLM, and Jina
+all serve `/v1/embeddings`. What differs between them is parameters
+(`dimensions`, `encodingFormat`, `batchSize`), not shape. Swap models without
+rewriting the call site — but treat a swap as a migration, because output
+dimensionality is not knowable from a model name.
+
+## Who it is for
+
+- Hosts that persist embeddings (memory, search, recall) and must discover
+  dimensionality before writing a row.
+- Sibling one-shot JSON clients that want the same classified, retried request
+  path without assembling an inference `AdapterRegistry`.
+- Anyone already holding an Interchange harness `Dependencies` — only `fetch`
+  and `scheduler` are required; a full harness object satisfies that.
+
+## Goals
+
+- Callers can `embedTexts` and `probeEmbedDims` from `@corbits/embedding`
+  with `{ fetch, scheduler }` and an `EmbedConfig`.
+- A 429 from an embedding endpoint behaves as one from a chat endpoint; a
+  credential failure aborts immediately.
+- Dimensionality is discovered, never assumed. `probeEmbedDims` embeds one
+  probe string and reports the length it got back. `dimensions` (Matryoshka
+  truncation) changes that length when set.
+- Empty input returns no vectors and issues no request.
+
+## Non-goals
+
+- A provider enum or adapter per vendor. There is one `/v1/embeddings` path.
+- Inferring vector width from a model name.
+- Concurrent batches. Sequential posting is how a rate-limited provider is
+  not hit with the whole corpus at once.

README.md

@@ -1,92 +1,66 @@
 # @corbits/embedding
 
-Embedding client for OpenAI-compatible `/v1/embeddings` endpoints. Given a
-`baseURL`, a model, and some texts, `embedTexts` batches them, posts each batch,
-validates the response, and returns vectors in input order.
+One API for every embedding provider. Point at any OpenAI-compatible `/v1/embeddings` endpoint, swap models with a config change, and get ordered vectors back.
 
-One wire format, not a provider switch. OpenAI, Ollama, TEI, vLLM and Jina all
-serve `/v1/embeddings` — what differs between them is parameters:
+OpenAI, Ollama, TEI, vLLM, and Jina all serve the same wire shape. This package is one code path over that shape, with parameters for what varies between providers:
 
-- `dimensions` — Matryoshka truncation, supported by OpenAI text-embedding-3
-  (down to 256), Jina v3 (32) and v4 (128). Models without MRL reject it, so it
-  is only sent when set.
-- `encodingFormat` — `float` or `base64`.
-- `batchSize` — providers cap inputs per request; OpenAI rejects arrays over
-  2048, and a smaller batch is also how you stay under a token ceiling.
+- `dimensions` — Matryoshka truncation for models that support it, including OpenAI `text-embedding-3` (down to 256), Jina v3 (32), and Jina v4 (128). Sent only when set, so models without MRL stay happy.
+- `encodingFormat` — `float` (default) or compact `base64`.
+- `batchSize` — inputs per request (default 32). Tune it to provider caps and per-request token ceilings.
 
-Transport, error classification and retry come from `@intx/inference` rather
-than being reimplemented here: `deps.fetch` is the single request path,
-failures become an `InferenceError` through the same classifiers a chat call
-uses, and `createDefaultRetryPolicy` decides whether to back off or abort. A 429
-from an embedding endpoint therefore behaves exactly as one from a chat
-endpoint, and a `credential_failure` aborts immediately rather than retrying a
-bad key.
+Transport, error classification, and retry build on `@intx/inference`: requests go through the shared `deps.fetch` path, failures classify into `InferenceError` just like chat calls, and `createDefaultRetryPolicy` guides backoff. A 429 from an embeddings endpoint behaves like one from a chat endpoint, and credential failures short-circuit.
 
-## Installation
-
-Not published to npm yet. Until a registry publish, `npm install @corbits/embedding`
-404s. Git is the install path.
+## Install
 
 ```bash
-bun add github:corbitsdev/corbits-embedding
+bun add @corbits/embedding
 ```
 
-Requires Node >= 24 or Bun >= 1.2. The default export is built `dist/`; native Node
-does not load this package's extensionless TypeScript source.
+Runs on Bun >= 1.2 or Node >= 24. The built `dist/` entry is the default import.
+
+## Quickstart
 
 ```ts
 import { createDefaultScheduler } from "@intx/inference";
 import { embedTexts, probeEmbedDims } from "@corbits/embedding";
 
-// Only `fetch` and `scheduler` are needed — a full harness `Dependencies`
-// also satisfies this if you already have one.
 const deps = { fetch, scheduler: createDefaultScheduler() };
 
 const config = {
-  baseURL: "http://localhost:11434/v1", // include the provider's version prefix
+  baseURL: "http://localhost:11434/v1", // provider root, version prefix included
   model: "nomic-embed-text",
 };
 
 const dims = await probeEmbedDims(config, { deps });
 const [vector] = await embedTexts(["hello world"], config, { deps });
 ```
 
-## Dimensionality is discovered, never assumed
+`embedTexts(texts, config, options)` batches sequentially, posts each batch to `{baseURL}/embeddings`, and returns vectors in input order. Options carry `deps`, with optional `retryPolicy`, `extractRetryAfterMs`, and `signal`.
 
-`probeEmbedDims` embeds one probe string and reports the length it got back.
-Dimensionality is not knowable from a model name — it varies by provider and is
-changed by `dimensions` — so a caller that persists vectors must discover it.
-Defaults differ enough to matter: 1536 for OpenAI `text-embedding-3-small`, 768
-for `nomic-embed-text`, 384 for `bge-small-en-v1.5`. Swapping models is a
-migration, not a config change.
+## Dimensionality
 
-## Errors
+`probeEmbedDims(config, { deps })` embeds one probe string and reports the length received. Dimensionality follows the provider and the `dimensions` setting, so callers that persist vectors discover it at startup and treat a model swap as a migration. Representative widths: 1536 for OpenAI `text-embedding-3-small`, 768 for `nomic-embed-text`, 384 for `bge-small-en-v1.5`.
 
-Every failure — transport, HTTP status, or a 200 whose body is not JSON —
-raises `ModelRequestError`, carrying the classified `InferenceError` as `reason`
-and the URL.
+## Errors
 
-Note that `@corbits/embedding` and `@corbits/reranking` each carry their own
-copy of this class until the shared transport is upstreamed, so `instanceof`
-does not hold across the two. Catching both? Discriminate on
-`error.name === "ModelRequestError"`.
+Every failure mode — transport, HTTP status, or a 200 with an unexpected body — raises `ModelRequestError`, carrying the classified `InferenceError` as `reason` plus the request URL. The embedding and reranking packages each carry their own copy of this class while the shared transport is upstreamed, so code catching both discriminates on `error.name === "ModelRequestError"`.
 
 ## Transport exports
 
-The barrel re-exports the transport `embedTexts` is built on, for sibling
-one-shot JSON clients that want the same classified, retried request path
-without touching `@intx/inference` internals:
+The barrel also re-exports the one-shot JSON transport `embedTexts` is built on, for sibling clients that want the same classified, retried request path:
 
 - `runJSONRequest` — one JSON POST with error classification and retry.
-- `extractRetryAfterMs` — the default `Retry-After` reader, overridable per
-  call via `extractRetryAfterMs`.
-- `ModelRequestError` — the error every failure mode raises, carrying the
-  classified `InferenceError` as `reason` and the URL.
+- `extractRetryAfterMs` — default `Retry-After` reader, overridable per call.
+- `ModelRequestError` — error for every failure mode, with `reason` and URL.
 - Types `RunRequestOptions` and `RetryAfterExtractor`.
 
+## Interchange
+
+Interchange hubs and ingestion workers use this package for embeddings behind the shared `@intx/inference` transport: same fetch path, same error taxonomy, and same retry behavior as chat. Memory and retrieval pipelines pair `probeEmbedDims` at startup with `embedTexts` batches at ingest and query time.
+
 ## Versioning
 
-Semver. Released manually with `bun run build && npm publish` once CI is green.
+Semver. Releases run `bun run build && npm publish` with green CI.
 
 ## Development