From e471006d73cee156264ac6155a820ec5674c77c3 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Sun, 20 Sep 2026 22:13:18 -0700 Subject: [PATCH 1/4] docs: registry-first README for @corbits/embedding Replace the git-install README with the Interchange section shape: Install / Use / Full example / How it works / Contributing. --- README.md | 132 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 68 insertions(+), 64 deletions(-) diff --git a/README.md b/README.md index f9d47ca..2dc613d 100644 --- a/README.md +++ b/README.md @@ -1,101 +1,105 @@ # @corbits/embedding Embedding client for OpenAI-compatible `/v1/embeddings` endpoints. Given a -`baseURL`, a model, and some texts, `embedTexts` batches them, posts each batch, -validates the response, and returns vectors in input order. +`baseURL`, a model, and some texts, `embedTexts` batches them, posts each +batch, validates the reply, and returns vectors in input order. -One wire format, not a provider switch. OpenAI, Ollama, TEI, vLLM and Jina all -serve `/v1/embeddings` — what differs between them is parameters: +One wire format, not a provider switch — OpenAI, Ollama, TEI, vLLM, and Jina +all serve `/v1/embeddings`. -- `dimensions` — Matryoshka truncation, supported by OpenAI text-embedding-3 - (down to 256), Jina v3 (32) and v4 (128). Models without MRL reject it, so it - is only sent when set. -- `encodingFormat` — `float` or `base64`. -- `batchSize` — providers cap inputs per request; OpenAI rejects arrays over - 2048, and a smaller batch is also how you stay under a token ceiling. +## Install -Transport, error classification and retry come from `@intx/inference` rather -than being reimplemented here: `deps.fetch` is the single request path, -failures become an `InferenceError` through the same classifiers a chat call -uses, and `createDefaultRetryPolicy` decides whether to back off or abort. A 429 -from an embedding endpoint therefore behaves exactly as one from a chat -endpoint, and a `credential_failure` aborts immediately rather than retrying a -bad key. +```bash +npm install @corbits/embedding +pnpm add @corbits/embedding +yarn add @corbits/embedding +bun add @corbits/embedding +``` -## Installation +Requires Node >= 24 or Bun >= 1.2. The published export is built `dist/`; +native Node does not load this package's TypeScript source. -Not published to npm yet. Until a registry publish, `npm install @corbits/embedding` -404s. Git is the install path. +## Use -```bash -bun add github:corbitsdev/corbits-embedding +```ts +import { createDefaultScheduler } from "@intx/inference"; +import { embedTexts } from "@corbits/embedding"; + +const deps = { fetch, scheduler: createDefaultScheduler() }; +const [vector] = await embedTexts( + ["hello world"], + { baseURL: "http://localhost:11434/v1", model: "nomic-embed-text" }, + { deps }, +); ``` -Requires Node >= 24 or Bun >= 1.2. The default export is built `dist/`; native Node -does not load this package's extensionless TypeScript source. +Only `fetch` and `scheduler` are required — a full harness `Dependencies` +satisfies this if you already have one. + +## Full example ```ts import { createDefaultScheduler } from "@intx/inference"; import { embedTexts, probeEmbedDims } from "@corbits/embedding"; -// Only `fetch` and `scheduler` are needed — a full harness `Dependencies` -// also satisfies this if you already have one. const deps = { fetch, scheduler: createDefaultScheduler() }; - const config = { - baseURL: "http://localhost:11434/v1", // include the provider's version prefix - model: "nomic-embed-text", + baseURL: "https://api.openai.com/v1", + model: "text-embedding-3-small", + apiKey: process.env.OPENAI_API_KEY, + // dimensions: 256, // Matryoshka truncation; sent only when set + // encodingFormat: "float", // or "base64" + // batchSize: 32, // OpenAI rejects arrays over 2048 }; const dims = await probeEmbedDims(config, { deps }); -const [vector] = await embedTexts(["hello world"], config, { deps }); -``` +const vectors = await embedTexts( + ["staging deploys run from main", "on-call rotation is weekly"], + config, + { deps }, +); -## Dimensionality is discovered, never assumed +console.log(dims, vectors[0]?.length); +``` `probeEmbedDims` embeds one probe string and reports the length it got back. -Dimensionality is not knowable from a model name — it varies by provider and is -changed by `dimensions` — so a caller that persists vectors must discover it. -Defaults differ enough to matter: 1536 for OpenAI `text-embedding-3-small`, 768 -for `nomic-embed-text`, 384 for `bge-small-en-v1.5`. Swapping models is a -migration, not a config change. - -## Errors - -Every failure — transport, HTTP status, or a 200 whose body is not JSON — -raises `ModelRequestError`, carrying the classified `InferenceError` as `reason` -and the URL. - -Note that `@corbits/embedding` and `@corbits/reranking` each carry their own -copy of this class until the shared transport is upstreamed, so `instanceof` -does not hold across the two. Catching both? Discriminate on +Dimensionality is not knowable from a model name — it varies by provider and +is changed by `dimensions` — so a caller that persists vectors must discover +it. Swapping models is a migration, not a config change. + +Failures raise `ModelRequestError` (transport, HTTP status, or a 200 whose +body is not JSON), carrying the classified `InferenceError` as `reason` and +the URL. `@corbits/embedding` and `@corbits/reranking` each carry their own +copy of this class, so `instanceof` does not hold across the two — catch on `error.name === "ModelRequestError"`. -## Transport exports - -The barrel re-exports the transport `embedTexts` is built on, for sibling -one-shot JSON clients that want the same classified, retried request path -without touching `@intx/inference` internals: +## How it works -- `runJSONRequest` — one JSON POST with error classification and retry. -- `extractRetryAfterMs` — the default `Retry-After` reader, overridable per - call via `extractRetryAfterMs`. -- `ModelRequestError` — the error every failure mode raises, carrying the - classified `InferenceError` as `reason` and the URL. -- Types `RunRequestOptions` and `RetryAfterExtractor`. +`embedTexts` splits the input on `batchSize` (default 32) and posts each +batch sequentially to `{baseURL}/embeddings`. Placement is by the echoed +`index`, never by reply position; a reordered or short reply is rejected +rather than silently pairing the wrong vector with a text. -## Versioning +Transport, error classification, and retry come from `@intx/inference`: +`deps.fetch` is the single request path, failures become an `InferenceError` +through the same classifiers a chat call uses, and `createDefaultRetryPolicy` +decides whether to back off or abort. A 429 from an embedding endpoint +therefore behaves as one from a chat endpoint; a `credential_failure` aborts +immediately. -Semver. Released manually with `bun run build && npm publish` once CI is green. +The barrel also re-exports the one-shot JSON transport (`runJSONRequest`, +`extractRetryAfterMs`, `ModelRequestError`) for sibling clients that want +the same classified, retried request path. -## Development +## Contributing ```bash bun install -bun test ./src -bunx tsc --noEmit +bun run build # tsc -p tsconfig.build.json +bun run test # bun test ./src +bun run typecheck # tsc --noEmit ``` -## License +Node >= 24, Bun >= 1.2.0. LGPL-2.1-only — see [`LICENSE`](LICENSE). From cb573e0f1347f3a119e7c3fd321941444a91fb7c Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Sun, 20 Sep 2026 22:43:19 -0700 Subject: [PATCH 2/4] docs: align README headings with Interchange (CL-8775) Use Quickstart / How it works / Development / License instead of Install / Use / Full example / Contributing. --- README.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 2dc613d..16a8c5c 100644 --- a/README.md +++ b/README.md @@ -7,20 +7,20 @@ batch, validates the reply, and returns vectors in input order. One wire format, not a provider switch — OpenAI, Ollama, TEI, vLLM, and Jina all serve `/v1/embeddings`. -## Install +## Runtime support + +Bun >= 1.2 is the development runtime. Node >= 24 consumes built `dist/`; +native Node does not load this package's TypeScript source. + +## Quickstart ```bash -npm install @corbits/embedding +npm add @corbits/embedding pnpm add @corbits/embedding yarn add @corbits/embedding bun add @corbits/embedding ``` -Requires Node >= 24 or Bun >= 1.2. The published export is built `dist/`; -native Node does not load this package's TypeScript source. - -## Use - ```ts import { createDefaultScheduler } from "@intx/inference"; import { embedTexts } from "@corbits/embedding"; @@ -36,8 +36,6 @@ const [vector] = await embedTexts( Only `fetch` and `scheduler` are required — a full harness `Dependencies` satisfies this if you already have one. -## Full example - ```ts import { createDefaultScheduler } from "@intx/inference"; import { embedTexts, probeEmbedDims } from "@corbits/embedding"; @@ -91,15 +89,17 @@ The barrel also re-exports the one-shot JSON transport (`runJSONRequest`, `extractRetryAfterMs`, `ModelRequestError`) for sibling clients that want the same classified, retried request path. -## Contributing +## Development ```bash +git clone https://github.com/corbitsdev/corbits-embedding.git +cd corbits-embedding bun install bun run build # tsc -p tsconfig.build.json bun run test # bun test ./src bun run typecheck # tsc --noEmit ``` -Node >= 24, Bun >= 1.2.0. +## License LGPL-2.1-only — see [`LICENSE`](LICENSE). From 79a7ecef8626df528e9ccc27201ac497113002f8 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Mon, 21 Sep 2026 18:53:26 -0700 Subject: [PATCH 3/4] docs: product-voice README (CL-8775) --- README.md | 102 +++++++++++++++++++----------------------------------- 1 file changed, 36 insertions(+), 66 deletions(-) diff --git a/README.md b/README.md index 16a8c5c..d269fa3 100644 --- a/README.md +++ b/README.md @@ -1,103 +1,73 @@ # @corbits/embedding -Embedding client for OpenAI-compatible `/v1/embeddings` endpoints. Given a -`baseURL`, a model, and some texts, `embedTexts` batches them, posts each -batch, validates the reply, and returns vectors in input order. +One API for every embedding provider. Point at any OpenAI-compatible `/v1/embeddings` endpoint, swap models with a config change, and get ordered vectors back. -One wire format, not a provider switch — OpenAI, Ollama, TEI, vLLM, and Jina -all serve `/v1/embeddings`. +OpenAI, Ollama, TEI, vLLM, and Jina all serve the same wire shape. This package is one code path over that shape, with parameters for what varies between providers: -## Runtime support +- `dimensions` — Matryoshka truncation for models that support it, including OpenAI `text-embedding-3` (down to 256), Jina v3 (32), and Jina v4 (128). Sent only when set, so models without MRL stay happy. +- `encodingFormat` — `float` (default) or compact `base64`. +- `batchSize` — inputs per request (default 32). Tune it to provider caps and per-request token ceilings. -Bun >= 1.2 is the development runtime. Node >= 24 consumes built `dist/`; -native Node does not load this package's TypeScript source. +Transport, error classification, and retry build on `@intx/inference`: requests go through the shared `deps.fetch` path, failures classify into `InferenceError` just like chat calls, and `createDefaultRetryPolicy` guides backoff. A 429 from an embeddings endpoint behaves like one from a chat endpoint, and credential failures short-circuit. -## Quickstart +## Install ```bash -npm add @corbits/embedding -pnpm add @corbits/embedding -yarn add @corbits/embedding bun add @corbits/embedding ``` -```ts -import { createDefaultScheduler } from "@intx/inference"; -import { embedTexts } from "@corbits/embedding"; - -const deps = { fetch, scheduler: createDefaultScheduler() }; -const [vector] = await embedTexts( - ["hello world"], - { baseURL: "http://localhost:11434/v1", model: "nomic-embed-text" }, - { deps }, -); -``` +Runs on Bun >= 1.2 or Node >= 24. The built `dist/` entry is the default import. -Only `fetch` and `scheduler` are required — a full harness `Dependencies` -satisfies this if you already have one. +## Quickstart ```ts import { createDefaultScheduler } from "@intx/inference"; import { embedTexts, probeEmbedDims } from "@corbits/embedding"; const deps = { fetch, scheduler: createDefaultScheduler() }; + const config = { - baseURL: "https://api.openai.com/v1", - model: "text-embedding-3-small", - apiKey: process.env.OPENAI_API_KEY, - // dimensions: 256, // Matryoshka truncation; sent only when set - // encodingFormat: "float", // or "base64" - // batchSize: 32, // OpenAI rejects arrays over 2048 + baseURL: "http://localhost:11434/v1", // provider root, version prefix included + model: "nomic-embed-text", }; const dims = await probeEmbedDims(config, { deps }); -const vectors = await embedTexts( - ["staging deploys run from main", "on-call rotation is weekly"], - config, - { deps }, -); - -console.log(dims, vectors[0]?.length); +const [vector] = await embedTexts(["hello world"], config, { deps }); ``` -`probeEmbedDims` embeds one probe string and reports the length it got back. -Dimensionality is not knowable from a model name — it varies by provider and -is changed by `dimensions` — so a caller that persists vectors must discover -it. Swapping models is a migration, not a config change. +`embedTexts(texts, config, options)` batches sequentially, posts each batch to `{baseURL}/embeddings`, and returns vectors in input order. Options carry `deps`, with optional `retryPolicy`, `extractRetryAfterMs`, and `signal`. + +## Dimensionality + +`probeEmbedDims(config, { deps })` embeds one probe string and reports the length received. Dimensionality follows the provider and the `dimensions` setting, so callers that persist vectors discover it at startup and treat a model swap as a migration. Representative widths: 1536 for OpenAI `text-embedding-3-small`, 768 for `nomic-embed-text`, 384 for `bge-small-en-v1.5`. + +## Errors + +Every failure mode — transport, HTTP status, or a 200 with an unexpected body — raises `ModelRequestError`, carrying the classified `InferenceError` as `reason` plus the request URL. The embedding and reranking packages each carry their own copy of this class while the shared transport is upstreamed, so code catching both discriminates on `error.name === "ModelRequestError"`. + +## Transport exports + +The barrel also re-exports the one-shot JSON transport `embedTexts` is built on, for sibling clients that want the same classified, retried request path: -Failures raise `ModelRequestError` (transport, HTTP status, or a 200 whose -body is not JSON), carrying the classified `InferenceError` as `reason` and -the URL. `@corbits/embedding` and `@corbits/reranking` each carry their own -copy of this class, so `instanceof` does not hold across the two — catch on -`error.name === "ModelRequestError"`. +- `runJSONRequest` — one JSON POST with error classification and retry. +- `extractRetryAfterMs` — default `Retry-After` reader, overridable per call. +- `ModelRequestError` — error for every failure mode, with `reason` and URL. +- Types `RunRequestOptions` and `RetryAfterExtractor`. -## How it works +## Interchange -`embedTexts` splits the input on `batchSize` (default 32) and posts each -batch sequentially to `{baseURL}/embeddings`. Placement is by the echoed -`index`, never by reply position; a reordered or short reply is rejected -rather than silently pairing the wrong vector with a text. +Interchange hubs and ingestion workers use this package for embeddings behind the shared `@intx/inference` transport: same fetch path, same error taxonomy, and same retry behavior as chat. Memory and retrieval pipelines pair `probeEmbedDims` at startup with `embedTexts` batches at ingest and query time. -Transport, error classification, and retry come from `@intx/inference`: -`deps.fetch` is the single request path, failures become an `InferenceError` -through the same classifiers a chat call uses, and `createDefaultRetryPolicy` -decides whether to back off or abort. A 429 from an embedding endpoint -therefore behaves as one from a chat endpoint; a `credential_failure` aborts -immediately. +## Versioning -The barrel also re-exports the one-shot JSON transport (`runJSONRequest`, -`extractRetryAfterMs`, `ModelRequestError`) for sibling clients that want -the same classified, retried request path. +Semver. Releases run `bun run build && npm publish` with green CI. ## Development ```bash -git clone https://github.com/corbitsdev/corbits-embedding.git -cd corbits-embedding bun install -bun run build # tsc -p tsconfig.build.json -bun run test # bun test ./src -bun run typecheck # tsc --noEmit +bun test ./src +bunx tsc --noEmit ``` ## License From 7feb85731846988d6c9f8ce1e5b384538e0baba6 Mon Sep 17 00:00:00 2001 From: Sawyer Cutler Date: Mon, 21 Sep 2026 19:39:11 -0700 Subject: [PATCH 4/4] docs: add PRODUCT, ARCHITECTURE, IMPLEMENTATION for @corbits/embedding (CL-8775) (#5) --- ARCHITECTURE.md | 108 ++++++++++++++++++++++++++++++++++++++++++++ IMPLEMENTATION.md | 113 ++++++++++++++++++++++++++++++++++++++++++++++ PRODUCT.md | 48 ++++++++++++++++++++ 3 files changed, 269 insertions(+) create mode 100644 ARCHITECTURE.md create mode 100644 IMPLEMENTATION.md create mode 100644 PRODUCT.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..fbb1eff --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,108 @@ +# Architecture + +The package is two layers: a one-shot JSON transport, and an embedding client +built on it. There is no provider registry. Every call is HTTP POST to +`{baseURL}/embeddings`. + +## Surfaces + +Callers import from `@corbits/embedding`. The barrel is the public surface: + +| Export | Role | +| --- | --- | +| `embedTexts` | Batch texts, post sequentially, return `number[][]` in input order | +| `probeEmbedDims` | Embed one probe string; report the vector length that came back | +| `EmbedConfig` / `EmbedConfigSchema` | Endpoint, model, and optional knobs | +| `EmbedOptions` | Harness deps plus optional retry, `Retry-After` override, abort | +| `runJSONRequest` | Classified, retried JSON POST (sibling clients reuse this) | +| `extractRetryAfterMs` | Default `Retry-After` reader (seconds or HTTP-date) | +| `ModelRequestError` | Transport, HTTP, or non-JSON 200 — `reason` is the classified `InferenceError` | + +Quickstart shape (matches the README): + +```ts +import { createDefaultScheduler } from "@intx/inference"; +import { embedTexts, probeEmbedDims } from "@corbits/embedding"; + +const deps = { fetch, scheduler: createDefaultScheduler() }; +const [vector] = await embedTexts( + ["hello world"], + { baseURL: "http://localhost:11434/v1", model: "nomic-embed-text" }, + { deps }, +); +``` + +`probeEmbedDims(config, { deps })` is the same config and deps; it is not a +separate protocol. + +## Request dependencies + +The transport reads only `fetch` and `scheduler` from the harness +(`RequestDependencies`). That is deliberately narrower than Interchange +`Dependencies`, whose `adapters` registry a caller should not have to +assemble in order to embed a string. A real `Dependencies` still satisfies +the pick structurally. + +Time for retry backoff comes from `deps.scheduler`, not wall-clock globals, +so a virtual-clock test scheduler drives the loop deterministically. The +HTTP-date form of `Retry-After` is the exception: it names an absolute +instant and is resolved against wall time. + +## Control flow + +``` +embedTexts(texts, config, options) + → split on batchSize (default 32) + → for each batch, sequentially: + build POST {baseURL}/embeddings + runJSONRequest (classify + retry) + parse body: place by echoed index, reject short / duplicate / OOB + → concatenate batch results in input order +``` + +`probeEmbedDims` is `embedTexts` of one fixed probe string, then +`vector.length`. + +Batches are sequential, not concurrent: the retry policy is per request, not +per burst, and a rate-limited provider must not see the whole corpus at once. + +## Reply integrity + +The OpenAI-compatible reply is not promised in request order. Placement is +by the echoed `index`, never by array position. A reordered reply is +reordered back; a short reply, a duplicate index, or a missing slot is +rejected. Callers pair returned vectors with their own texts positionally — +a silent mispair would persist the wrong vector. + +`embedding` on the wire is either `number[]` (`float`) or a base64-packed +little-endian float32 buffer (`base64`). The client always returns +`number[]`. + +## Transport vs `runInference` + +`runInference` is turn-shaped and streams SSE. An embedding is one-shot JSON. +What carries over is everything around the body: the single `deps.fetch` +path, the `InferenceError` classifiers (`classifyHTTPError`, +`classifyNetworkError`, `classifyAbortError`, `classifyProtocolMismatch`), +and `createDefaultRetryPolicy` (back off retryables, abort the rest). A 429 +is therefore classified and backed off as a chat 429 would be; a +`credential_failure` aborts immediately. + +`runJSONRequest` is duplicated, modulo comments, in `@corbits/reranking`. It +is not shared infrastructure yet — the intended home is `@intx/inference` as +a non-streaming sibling of `runInference`. Until then, `ModelRequestError` is +a distinct class in each package: `instanceof` does not hold across the two. +Catch on `error.name === "ModelRequestError"`. + +## Failure modes + +- Transport, HTTP status, or a 200 whose body is not JSON → + `ModelRequestError` with classified `reason` and the URL. +- Malformed embeddings JSON, wrong count, bad index → thrown `Error` (not + `ModelRequestError`); the protocol succeeded, the payload did not. +- `batchSize` not an integer `>= 1` → `RangeError` before any request + (`i += size` of 0 would never advance). +- Empty `texts` → `[]`, no request. +- Caller `signal` or per-attempt timeout (default 30s) abort the attempt; + aborting mid-delay wakes immediately and the next attempt fails its + entry-time signal check. diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md new file mode 100644 index 0000000..a86cfb4 --- /dev/null +++ b/IMPLEMENTATION.md @@ -0,0 +1,113 @@ +# Implementation + +## Package + +`@corbits/embedding` is an ESM TypeScript package. Development runtime is +Bun `>= 1.2.0`. Node `>= 24` consumes built `dist/`; native Node does not +load this package's TypeScript source. + +```json +"exports": { + ".": { + "intx-src": "./src/index.ts", + "types": "./dist/index.d.ts", + "default": "./dist/index.js" + } +} +``` + +`bun run build` is `tsc -p tsconfig.build.json` (`outDir: dist`, +`rootDir: src`, tests excluded). Published files are `dist`, `README.md`, +and `LICENSE`. Install from the registry: + +```bash +npm add @corbits/embedding +pnpm add @corbits/embedding +yarn add @corbits/embedding +bun add @corbits/embedding +``` + +## Dependencies + +| Package | Use | +| --- | --- | +| `@intx/inference` | `BuiltRequest`, `createDefaultRetryPolicy`, error classifiers, `createDefaultScheduler` (caller-side) | +| `@intx/types` | `RetryPolicy`, `InferenceError` | +| `@intx/log` | Transitive Interchange surface (not imported by this package's sources) | +| `arktype` | `EmbedConfigSchema` and the embeddings reply shape | + +Dev: `@intx/inference-testing`, `@types/bun`, `prettier`, `typescript` 5.9. + +## HTTP + +Each batch is: + +- `POST {baseURL}/embeddings` (`baseURL` already includes the version + prefix, e.g. `http://localhost:11434/v1` or `https://api.openai.com/v1`) +- `content-type: application/json` +- `authorization: Bearer {apiKey}` only when `apiKey` is set +- `redirect: "manual"` +- body: + +```json +{ + "model": "", + "input": ["…"], + "dimensions": 256, + "encoding_format": "float" +} +``` + +`dimensions` and `encoding_format` are omitted unless configured. Models +without Matryoshka reject `dimensions` outright, so an unset knob must not +appear. Client field is `encodingFormat`; wire field is `encoding_format` +(`float` | `base64`). Default `batchSize` is 32 (OpenAI rejects arrays over +2048). Default per-attempt timeout is 30s (`AbortSignal.timeout`, combined +with a caller `signal` via `AbortSignal.any`). + +A 200 is read as text then `JSON.parse`. `res.json()` is not used: an HTML +interstitial or truncated body in front of a local model server must classify +as protocol mismatch, not escape as `SyntaxError`. + +## arktype + +`EmbedConfigSchema` (`src/embed.ts`): + +- required: `baseURL: string`, `model: string` +- optional: `apiKey`, `dimensions` (`number > 0`), `encodingFormat` + (`'float'|'base64'`), `batchSize` (`number.integer >= 1`), `timeoutMs` + (`number > 0`) + +Reply: `{ data: { index: number, embedding: number[] | string }[] }`. +Base64 strings decode with `atob` → `Uint8Array` → little-endian +`Float32Array` → `number[]`. + +`embedTexts` also range-checks `batchSize` at runtime so a value that +bypassed the schema cannot stall `batches` (`i += size`). + +## Errors and retry + +`runJSONRequest` (`src/request.ts`) loops: + +1. `deps.fetch` POST +2. non-OK → `classifyHTTPError(status, detail, detail, retryAfterMs)` +3. network throw → `classifyNetworkError` or `classifyAbortError` +4. JSON parse fail → `classifyProtocolMismatch` +5. `retryPolicy` (default `createDefaultRetryPolicy`) → `abort` throws + `ModelRequestError`, else sleep `delayMs` on `deps.scheduler` + +`extractRetryAfterMs` reads `Retry-After` as seconds or HTTP-date, clamped +at zero. Callers override via `EmbedOptions.extractRetryAfterMs` (same +signature as Interchange's unexported extractor; re-declared so this +package depends only on the published surface). + +`ModelRequestError.name` is `"ModelRequestError"`. Discriminate on the +name when catching both this package and `@corbits/reranking`. + +## Tests + +`bun run test` is `bun test ./src`. Coverage is the client contract: empty +input, URL and body knobs, auth header presence, batch split and order, +`batchSize` rejection, malformed / short / duplicate-index replies, +index-not-position placement, 429 retry, 401 abort, `probeEmbedDims`, +overridable `Retry-After`. diff --git a/PRODUCT.md b/PRODUCT.md new file mode 100644 index 0000000..62f961f --- /dev/null +++ b/PRODUCT.md @@ -0,0 +1,48 @@ +# Product + +`@corbits/embedding` is the client you call when you need vectors from an +OpenAI-compatible `/v1/embeddings` endpoint. Given a `baseURL`, a model, and +some texts, `embedTexts` batches them, posts each batch, validates the reply, +and returns vectors in input order. + +## Why it exists + +Embedding is not a chat turn. Callers persist vectors next to text and later +retrieve by similarity. A wrong pairing, an assumed dimension, or a one-off +HTTP client that does not retry a 429 the way a chat call does all corrupt +that store. This package is the one place that does the boring work: batch, +validate, place by echoed `index`, and fail loudly instead of silently +mispairing. + +One wire format, not a provider switch. OpenAI, Ollama, TEI, vLLM, and Jina +all serve `/v1/embeddings`. What differs between them is parameters +(`dimensions`, `encodingFormat`, `batchSize`), not shape. Swap models without +rewriting the call site — but treat a swap as a migration, because output +dimensionality is not knowable from a model name. + +## Who it is for + +- Hosts that persist embeddings (memory, search, recall) and must discover + dimensionality before writing a row. +- Sibling one-shot JSON clients that want the same classified, retried request + path without assembling an inference `AdapterRegistry`. +- Anyone already holding an Interchange harness `Dependencies` — only `fetch` + and `scheduler` are required; a full harness object satisfies that. + +## Goals + +- Callers can `embedTexts` and `probeEmbedDims` from `@corbits/embedding` + with `{ fetch, scheduler }` and an `EmbedConfig`. +- A 429 from an embedding endpoint behaves as one from a chat endpoint; a + credential failure aborts immediately. +- Dimensionality is discovered, never assumed. `probeEmbedDims` embeds one + probe string and reports the length it got back. `dimensions` (Matryoshka + truncation) changes that length when set. +- Empty input returns no vectors and issues no request. + +## Non-goals + +- A provider enum or adapter per vendor. There is one `/v1/embeddings` path. +- Inferring vector width from a model name. +- Concurrent batches. Sequential posting is how a rate-limited provider is + not hit with the whole corpus at once.