diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 1b477cb..2f85019 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -2,17 +2,22 @@ How `@corbits/artifacts` is put together, and where its boundaries are. For install, the mount snippet, the route table and the response contracts, see the -[package README](./README.md) — this document is about -structure and reasoning, and does not repeat them. +[package README](./README.md). Intent is in [PRODUCT.md](./PRODUCT.md); named +libraries, paths, headers and ceilings are in +[IMPLEMENTATION.md](./IMPLEMENTATION.md). This document is about structure +and reasoning, and does not repeat those. ## The shape of the thing A **library, not a service**. It creates no app, starts no background work, and -opens no pool unless asked. A host calls two functions: +opens no pool unless asked. A host calls three functions: - `runArtifactMigrations(db)` — once at boot, before serving. -- `mountArtifacts(app, opts)` — registers the artifact routes on a Hono app the - host already built. +- `mountArtifacts(app, opts)` — tenant-session routes on a Hono app the host + already built. +- `mountWorkflowArtifacts(app, opts)` — the parallel mount for sidecar and + agent callers that have no browser session. A host that never runs + workflows omits it. ## Where the routes are served @@ -35,6 +40,12 @@ which serves `/api/artifacts`, `/api/artifacts/:id`, `/api/instances/:instanceId/mail-attachments`. Nesting rather than teaching the core a base path keeps the mount free of a configurable base path. +`mountWorkflowArtifacts` is the same idea on a second app: root-relative +`/artifacts*` routes, host-chosen prefix. The sidecar bundle has no options +of its own, so the shared prefix is +`WORKFLOW_ARTIFACTS_BASE_PATH` (`/api/workflow-artifacts`). A host that pins +the bundle must nest the workflow app there, or the tools miss. + Everything else it needs arrives through `opts` or the host's request context. Nothing is reached for. @@ -79,6 +90,37 @@ host's job" is meant to prevent. If a real product need for cross-tenant artifact reads shows up, it belongs in Interchange's grant model, not re-derived per package. +## The workflow mount + +`mountArtifacts` reads `tenant` / `principal` off `TenantEnv`. It has no +bearer-token surface and never will: mixing a browser-session convention and +a sidecar-token convention into one mount would make each harder to reason +about. A workflow run has no browser session. `mountWorkflowArtifacts` is +the parallel mount for that caller. + +The host supplies `resolveRunScope(bearerToken, runAddress)` — exactly how it +already authenticates its sidecar. This package trusts whatever it returns +and never talks to the host's run or sidecar tables. Returning `null` is 401. +Optional `agentToken` is a second authentication path tried first: only the +credential changes; the run is still named by `x-workflow-run-address`. A +token whose tenant is not the resolved run's tenant is the same bare 401 as +a missing run, so a bearer learns nothing from the difference. + +There is no `requireGrant` on this mount. The resolved run scope *is* the +authorization: every route is authenticated, and reads stay inside +`scope.tenantId`. Per-run rate limiting is deliberately not implemented +here — wrap `resolveRunScope` or the mounted app. The two ceilings this +package does own (`maxContentChars`, `maxBinaryBytes`) protect the same +storage `mountArtifacts` protects. + +Provenance is server-stamped from that scope (`source.origin = "workflow"`, +`source.runId`, `generatedBy`). A body field of either name is never read, +so a run cannot mint provenance for itself. Workflow-authored rows have no +human `ownerPrincipalId`; a human enters later as an approver, not as an +owner. The routes exist to cover `ARTIFACT_TOOL_DEFINITIONS`; the sidecar +bundle is the `defineTool` factory an agent pins so no agent owns client +code. + ## Three custom seams Beyond the host's native context and grants, this package exposes **three** @@ -154,6 +196,8 @@ which store is installed. | File | Role | | --- | --- | | `mount.ts` | HTTP surface: parsing, validation, status codes; reads `TenantEnv` principal; wires host `requireGrant`. | +| `workflow-mount.ts` | Parallel HTTP surface for sidecar/agent callers; host `resolveRunScope` (and optional `agentToken`). | +| `sidecar-bundle.ts` | `defineTool` factory; mediated fetch to the run-scoped routes. No db, no secret. | | `artifacts.ts` | The core domain — create, revise, find-or-version, list, get, archive, serialize. | | `uploads.ts` | `createFileArtifact`, the MIME policies, and the size caps. | | `download.ts` | One download path over the three storage conventions. | @@ -328,7 +372,9 @@ download path with its `nosniff`/`attachment` behaviour. Supplied by the host: the `Hono` app and the database handle; the authenticated `tenant`/`principal` on the request context; the host's -`RequireGrant`; display-only provenance decoration; and a `ContentStore`. +`RequireGrant`; display-only provenance decoration; a `ContentStore`; and, +when the workflow mount is wired, `resolveRunScope` (and optional +`agentToken.verify` / `resolveRun`). ## Known limits @@ -350,3 +396,8 @@ authenticated `tenant`/`principal` on the request context; the host's - **One 404 covers four causes** for a resolved caller — never minted, malformed, a `skill-draft`, or another tenant's. Distinguishing them would be an existence oracle. Expect no more detail than that from the API. +- **Workflow 401 is equally opaque.** Missing bearer, unknown run address, and + a token whose tenant is not the run's tenant all read the same. Distinguishing + them would be an existence oracle on runs. +- **No per-run quota in this package.** A host that wants one wraps + `resolveRunScope` or puts middleware in front of the workflow app. diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md new file mode 100644 index 0000000..48f3133 --- /dev/null +++ b/IMPLEMENTATION.md @@ -0,0 +1,160 @@ +# @corbits/artifacts — Implementation + +## Package + +- Name: `@corbits/artifacts` `0.2.1` +- License: LGPL-2.1-only +- Node >= 24 consumes built `dist/` (`main` / default export). +- Bun loads TypeScript source via the `bun` export condition (`./src/index.ts`). +- These design docs live at the repository root. The npm tarball currently + ships `dist/`, `src/` (tests excluded), `README.md`, and `LICENSE`. + +## Runtime and peers + +Minimum `@intx/*` **0.3.0**. Peer stack (must resolve to the host's copy): + +`hono`, `hono-openapi`, `drizzle-orm`, `postgres`, `arktype`, `@intx/types`, +`@intx/hub-api`, `@intx/agent`. + +Direct dependency: `@hono/standard-validator`. Postgres 13+ (`gen_random_uuid()`). + +## Install + +```sh +npm add @corbits/artifacts +pnpm add @corbits/artifacts +yarn add @corbits/artifacts +bun add @corbits/artifacts +``` + +## Public surface + +From `@corbits/artifacts`: + +- `runArtifactMigrations(db)` — idempotent boot DDL into schema `artifacts`. +- `mountArtifacts(app, opts)` — tenant-session routes on `Hono`. +- `mountWorkflowArtifacts(app, opts)` — run-scoped routes on + `Hono`. +- `InlineContentStore`, `DataUrlContentStore`, and the `ContentStore` type. +- Domain helpers (`createArtifact`, `createFileArtifact`, list/get/archive, + tools, web-site, mail-attachment refs) for hosts that call the store without + HTTP. + +From `@corbits/artifacts/sidecar-bundle`: + +- `defineTool` factory pinned by an agent definition. Declares + `requires: ["capabilities", "address"]`. +- `WORKFLOW_ARTIFACTS_BASE_PATH` — `"/api/workflow-artifacts"`. The bundle has + no mount options; a host that wants the bundle to hit this package must + nest `mountWorkflowArtifacts` under that prefix. + +## `mountArtifacts` + +Required opts: `db`, `contentStore`, `requireGrant`. Optional: `decorate` +(display-only, default no-op), `onArtifactCreated` (same transaction as +create, default no-op), `uploadPolicy` (default `ARTIFACT_UPLOAD_POLICY`), +`countSegments` (default `{}`). + +Host middleware must place Interchange `tenant` and `principal` on the +context. No principal: collection reads answer empty 200; detail and +mutations answer 403. + +Convention: nest the app at `/api` so paths match Interchange (`/api/me`, +`/api/tenants`). Routes are registered root-relative; the core takes no base +path. + +| Method | Path | Role | +| --- | --- | --- | +| GET | `/artifacts` | List (keyset) | +| POST | `/artifacts` | Import `link` or `document` | +| POST | `/artifacts/upload` | Eager file artifact | +| GET | `/artifacts/counts` | Segment tallies | +| GET | `/artifacts/:id` | Detail | +| GET | `/artifacts/:id/preview` | Preview headers/body | +| GET | `/artifacts/:id/versions` | Version list | +| GET | `/artifacts/:id/versions/:version` | Pinned version | +| POST | `/artifacts/:id/versions` | Revise | +| POST | `/artifacts/:id/archive` | Soft hide | +| POST | `/artifacts/:id/unarchive` | Restore | +| GET | `/artifacts/:id/download` | Bytes | +| POST | `/instances/:instanceId/mail-attachments` | Save refs | +| GET | `/instances/:instanceId/mail-attachments` | List refs | + +Single-artifact mutations authorize with +`requireGrant(idResource("artifact", "id"), )`. Trust-boundary bodies +are arktype. Text bodies over `MAX_ARTIFACT_CONTENT_BYTES` (15 MiB) are 413. +A file over `MAX_UPLOAD_BYTES` (10 MiB) is 413; a refused MIME is 415. + +## `mountWorkflowArtifacts` + +Required opts: `db`, `contentStore`, `resolveRunScope`. Optional: `agentToken` +(tried before the sidecar path), `uploadPolicy`, `maxBinaryBytes` (default +`MAX_UPLOAD_BYTES`), `maxContentChars` (default `64_000`). + +Every route runs behind bearer middleware. There is no unauthenticated +collection-read case. + +Authentication: + +- `Authorization: Bearer ` +- `x-workflow-run-address: ` + +`resolveRunScope(bearerToken, runAddress)` returns +`{ tenantId, principalId, runId }` or `null` (401). When `agentToken` is +set, `verify` then `resolveRun` run first. A token whose tenant is not the +resolved run's tenant is the same bare 401 as a missing run — no existence +oracle. + +Create/revise stamp `source: { origin: "workflow", runId }` and `generatedBy` +from the resolved scope. A body field of either name is never read. +`ownerPrincipalId` is `null` on workflow creates. + +| Method | Path | Role | +| --- | --- | --- | +| POST | `/artifacts` | Create text | +| POST | `/artifacts/binary` | Base64 file bytes | +| POST | `/artifacts/link-file` | Link a path | +| GET | `/artifacts` | List (`kind` / `limit`) | +| GET | `/artifacts/recent` | Recent (default 10, max 50) | +| GET | `/artifacts/find` | Exact title (`title` required) | +| PATCH | `/artifacts/:id` | Revise | +| GET | `/artifacts/:id` | Read-back | +| GET | `/artifacts/:id/read` | Budgeted tool read | +| GET | `/artifacts/:id/chunk` | Windowed tool read | + +Together these cover `ARTIFACT_TOOL_DEFINITIONS`. A missing row, another +tenant's row, or `skill-draft` is 404 on detail/revise, same as +`mountArtifacts`. Text over `maxContentChars` and binary over +`maxBinaryBytes` are 413. + +Per-run rate limiting is not implemented here. Wrap `resolveRunScope` or the +mounted app. + +## Persistence + +Schema `artifacts`: tables `artifact`, `artifact_version`, `upload`, +`mail_attachment_ref`, plus ledger `artifacts.migrations`. Hard FKs into +`public.tenant` and `public.principal`. `ContentStore.put` runs inside the +caller's transaction. + +## Development + +```sh +git clone https://github.com/corbitsdev/corbits-artifacts.git +cd corbits-artifacts +bun install +docker run -d --name corbits-artifact-pg -p 5457:5432 \ + -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=artifact_core postgres:16 +export ALLOW_DESTRUCTIVE_ARTIFACT_TESTS=1 + +bun run typecheck +bun run test # pretest dependency check, then unit + integration +bun run build # dist/ (JS + .d.ts) +bun run test:acceptance # builds, then examples/reference-host +``` + +Tests expect `postgres://postgres:postgres@localhost:5457/artifact_core` +(override with `ARTIFACT_DATABASE_URL`). Destructive tests require +`ALLOW_DESTRUCTIVE_ARTIFACT_TESTS=1` and an allowlisted database name +(`artifact_core`, or any name ending in `_test`). See +[CONTRIBUTING.md](./CONTRIBUTING.md). diff --git a/PRODUCT.md b/PRODUCT.md new file mode 100644 index 0000000..36f7a43 --- /dev/null +++ b/PRODUCT.md @@ -0,0 +1,74 @@ +# @corbits/artifacts — Product + +## What it is + +Persistent artifacts, versions, and file uploads as a mountable module for any +Interchange host. Backend only — this package ships no UI. + +A host keeps the things agents and people make: text, links, uploaded files, +and the version history of each. `mountArtifacts` is the tenant-session +surface. `mountWorkflowArtifacts` is the parallel surface for a workflow run +that has no browser session. Both write the same rows. + +## Why it exists + +Agents produce files, drafts, and links that disappear unless the host stores +them. Every host that wants that store should not reimplement versions, +uploads, and tenant scoping. This package is the drop-in: mount it, run its +migrations, keep the bytes. + +Two caller kinds exist and must not share one auth convention. A browser +session has Interchange `tenant` / `principal` and grants. A sidecar or +deployed agent has a bearer token and a run address. Mixing those into one +mount would make each harder to reason about, so the product is two mounts +over one store. + +## Who it is for + +- Interchange host operators who already have a Hono app, a Postgres pool, and + `RequireGrant`. +- Workflow hosts that authenticate sidecar or agent callers and need those + runs to create and read artifacts in the same tenant. +- Agent authors who pin `@corbits/artifacts/sidecar-bundle` so no agent owns + artifact client code. + +There is nothing here for an end-user UI. Clients consume the HTTP surface the +host mounted. + +## What users can do + +- Persist an artifact with append-only versions, list and download it, and + soft-archive it without destroying history. +- Upload files through a pluggable `ContentStore` (inline Postgres bytes for a + minimal host; another backend without a rewrite). +- Authorize tenant-session mutations through the host's Interchange grants — + this package invents no owner, membership, or admin policy. +- Let a workflow run create, revise, find, and budget-read artifacts using + the host's sidecar (or agent-token) resolver. +- Grant agents the sidecar bundle so tool calls hit the run-scoped routes + through the hub credential handle. + +## What it is not + +- Not a service. It creates no app, opens no pool, and starts no background + work. +- Not a UI, directory, or session library. +- Not an authorization product. Grants are the host's. The workflow mount + trusts whatever `resolveRunScope` (and optional `agentToken`) returns. +- Not a parser. No PDF, spreadsheet, or text extractor lives here or is + planned. +- Not a rate limiter for runs. Per-run quota stays host-side. + +## Goals + +1. One store for tenant-session and workflow-run callers. +2. Host-owned app, pool, session, and grants. +3. Two mounts, two auth conventions, no mix. +4. Bytes and artifact row atomic on write; tenant-scoped on read. +5. Honest public docs: README for install and the mount snippet; + this file for intent; `ARCHITECTURE.md` for structure; + `IMPLEMENTATION.md` for wire and config. + +## License + +LGPL-2.1-only. diff --git a/README.md b/README.md index 1d73522..70fda96 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,7 @@ mountWorkflowArtifacts(workflowApi, { resolveRunScope: (bearerToken, runAddress) => hub.resolveWorkflowRun(bearerToken, runAddress), }); -app.route("/workflow-artifacts", workflowApi); +app.route("/api/workflow-artifacts", workflowApi); ``` Agent tools live in `@corbits/artifacts/sidecar-bundle`. `examples/reference-host` is a complete `@intx/hub-api` host with this module mounted. @@ -70,7 +70,9 @@ Agent tools live in `@corbits/artifacts/sidecar-bundle`. `examples/reference-hos `mountArtifacts` registers tenant-session routes (list, import, upload, versions, download, archive) and authorizes through the host's `requireGrant`. `mountWorkflowArtifacts` is the parallel mount for sidecar/agent callers: a bearer plus run address, no browser session. Both persist rows in Postgres and blobs through a pluggable `ContentStore` (`InlineContentStore` for a minimal host). The host never shares app creation, pooling, or auth with this package. -See [ARCHITECTURE.md](./ARCHITECTURE.md) for the data model and mount options. +See [PRODUCT.md](./PRODUCT.md) for intent, [ARCHITECTURE.md](./ARCHITECTURE.md) +for the data model and mount options, and +[IMPLEMENTATION.md](./IMPLEMENTATION.md) for routes, headers, and ceilings. ## Development