Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 57 additions & 6 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,22 @@

How `@corbits/artifacts` is put together, and where its boundaries are. For
install, the mount snippet, the route table and the response contracts, see the
[package README](./README.md) — this document is about
structure and reasoning, and does not repeat them.
[package README](./README.md). Intent is in [PRODUCT.md](./PRODUCT.md); named
libraries, paths, headers and ceilings are in
[IMPLEMENTATION.md](./IMPLEMENTATION.md). This document is about structure
and reasoning, and does not repeat those.

## The shape of the thing

A **library, not a service**. It creates no app, starts no background work, and
opens no pool unless asked. A host calls two functions:
opens no pool unless asked. A host calls three functions:

- `runArtifactMigrations(db)` — once at boot, before serving.
- `mountArtifacts(app, opts)` — registers the artifact routes on a Hono app the
host already built.
- `mountArtifacts(app, opts)` — tenant-session routes on a Hono app the host
already built.
- `mountWorkflowArtifacts(app, opts)` — the parallel mount for sidecar and
agent callers that have no browser session. A host that never runs
workflows omits it.

## Where the routes are served

Expand All @@ -35,6 +40,12 @@ which serves `/api/artifacts`, `/api/artifacts/:id`,
`/api/instances/:instanceId/mail-attachments`. Nesting rather than teaching the
core a base path keeps the mount free of a configurable base path.

`mountWorkflowArtifacts` is the same idea on a second app: root-relative
`/artifacts*` routes, host-chosen prefix. The sidecar bundle has no options
of its own, so the shared prefix is
`WORKFLOW_ARTIFACTS_BASE_PATH` (`/api/workflow-artifacts`). A host that pins
the bundle must nest the workflow app there, or the tools miss.

Everything else it needs arrives through `opts` or the host's request context.
Nothing is reached for.

Expand Down Expand Up @@ -79,6 +90,37 @@ host's job" is meant to prevent. If a real product need for cross-tenant
artifact reads shows up, it belongs in Interchange's grant model, not
re-derived per package.

## The workflow mount

`mountArtifacts` reads `tenant` / `principal` off `TenantEnv`. It has no
bearer-token surface and never will: mixing a browser-session convention and
a sidecar-token convention into one mount would make each harder to reason
about. A workflow run has no browser session. `mountWorkflowArtifacts` is
the parallel mount for that caller.

The host supplies `resolveRunScope(bearerToken, runAddress)` — exactly how it
already authenticates its sidecar. This package trusts whatever it returns
and never talks to the host's run or sidecar tables. Returning `null` is 401.
Optional `agentToken` is a second authentication path tried first: only the
credential changes; the run is still named by `x-workflow-run-address`. A
token whose tenant is not the resolved run's tenant is the same bare 401 as
a missing run, so a bearer learns nothing from the difference.

There is no `requireGrant` on this mount. The resolved run scope *is* the
authorization: every route is authenticated, and reads stay inside
`scope.tenantId`. Per-run rate limiting is deliberately not implemented
here — wrap `resolveRunScope` or the mounted app. The two ceilings this
package does own (`maxContentChars`, `maxBinaryBytes`) protect the same
storage `mountArtifacts` protects.

Provenance is server-stamped from that scope (`source.origin = "workflow"`,
`source.runId`, `generatedBy`). A body field of either name is never read,
so a run cannot mint provenance for itself. Workflow-authored rows have no
human `ownerPrincipalId`; a human enters later as an approver, not as an
owner. The routes exist to cover `ARTIFACT_TOOL_DEFINITIONS`; the sidecar
bundle is the `defineTool` factory an agent pins so no agent owns client
code.

## Three custom seams

Beyond the host's native context and grants, this package exposes **three**
Expand Down Expand Up @@ -154,6 +196,8 @@ which store is installed.
| File | Role |
| --- | --- |
| `mount.ts` | HTTP surface: parsing, validation, status codes; reads `TenantEnv` principal; wires host `requireGrant`. |
| `workflow-mount.ts` | Parallel HTTP surface for sidecar/agent callers; host `resolveRunScope` (and optional `agentToken`). |
| `sidecar-bundle.ts` | `defineTool` factory; mediated fetch to the run-scoped routes. No db, no secret. |
| `artifacts.ts` | The core domain — create, revise, find-or-version, list, get, archive, serialize. |
| `uploads.ts` | `createFileArtifact`, the MIME policies, and the size caps. |
| `download.ts` | One download path over the three storage conventions. |
Expand Down Expand Up @@ -328,7 +372,9 @@ download path with its `nosniff`/`attachment` behaviour.

Supplied by the host: the `Hono<TenantEnv>` app and the database handle; the
authenticated `tenant`/`principal` on the request context; the host's
`RequireGrant`; display-only provenance decoration; and a `ContentStore`.
`RequireGrant`; display-only provenance decoration; a `ContentStore`; and,
when the workflow mount is wired, `resolveRunScope` (and optional
`agentToken.verify` / `resolveRun`).

## Known limits

Expand All @@ -350,3 +396,8 @@ authenticated `tenant`/`principal` on the request context; the host's
- **One 404 covers four causes** for a resolved caller — never minted,
malformed, a `skill-draft`, or another tenant's. Distinguishing them would be
an existence oracle. Expect no more detail than that from the API.
- **Workflow 401 is equally opaque.** Missing bearer, unknown run address, and
a token whose tenant is not the run's tenant all read the same. Distinguishing
them would be an existence oracle on runs.
- **No per-run quota in this package.** A host that wants one wraps
`resolveRunScope` or puts middleware in front of the workflow app.
160 changes: 160 additions & 0 deletions IMPLEMENTATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# @corbits/artifacts — Implementation

## Package

- Name: `@corbits/artifacts` `0.2.1`
- License: LGPL-2.1-only
- Node >= 24 consumes built `dist/` (`main` / default export).
- Bun loads TypeScript source via the `bun` export condition (`./src/index.ts`).
- These design docs live at the repository root. The npm tarball currently
ships `dist/`, `src/` (tests excluded), `README.md`, and `LICENSE`.

## Runtime and peers

Minimum `@intx/*` **0.3.0**. Peer stack (must resolve to the host's copy):

`hono`, `hono-openapi`, `drizzle-orm`, `postgres`, `arktype`, `@intx/types`,
`@intx/hub-api`, `@intx/agent`.

Direct dependency: `@hono/standard-validator`. Postgres 13+ (`gen_random_uuid()`).

## Install

```sh
npm add @corbits/artifacts
pnpm add @corbits/artifacts
yarn add @corbits/artifacts
bun add @corbits/artifacts
```

## Public surface

From `@corbits/artifacts`:

- `runArtifactMigrations(db)` — idempotent boot DDL into schema `artifacts`.
- `mountArtifacts(app, opts)` — tenant-session routes on `Hono<TenantEnv>`.
- `mountWorkflowArtifacts(app, opts)` — run-scoped routes on
`Hono<WorkflowArtifactEnv>`.
- `InlineContentStore`, `DataUrlContentStore`, and the `ContentStore` type.
- Domain helpers (`createArtifact`, `createFileArtifact`, list/get/archive,
tools, web-site, mail-attachment refs) for hosts that call the store without
HTTP.

From `@corbits/artifacts/sidecar-bundle`:

- `defineTool` factory pinned by an agent definition. Declares
`requires: ["capabilities", "address"]`.
- `WORKFLOW_ARTIFACTS_BASE_PATH` — `"/api/workflow-artifacts"`. The bundle has
no mount options; a host that wants the bundle to hit this package must
nest `mountWorkflowArtifacts` under that prefix.

## `mountArtifacts`

Required opts: `db`, `contentStore`, `requireGrant`. Optional: `decorate`
(display-only, default no-op), `onArtifactCreated` (same transaction as
create, default no-op), `uploadPolicy` (default `ARTIFACT_UPLOAD_POLICY`),
`countSegments` (default `{}`).

Host middleware must place Interchange `tenant` and `principal` on the
context. No principal: collection reads answer empty 200; detail and
mutations answer 403.

Convention: nest the app at `/api` so paths match Interchange (`/api/me`,
`/api/tenants`). Routes are registered root-relative; the core takes no base
path.

| Method | Path | Role |
| --- | --- | --- |
| GET | `/artifacts` | List (keyset) |
| POST | `/artifacts` | Import `link` or `document` |
| POST | `/artifacts/upload` | Eager file artifact |
| GET | `/artifacts/counts` | Segment tallies |
| GET | `/artifacts/:id` | Detail |
| GET | `/artifacts/:id/preview` | Preview headers/body |
| GET | `/artifacts/:id/versions` | Version list |
| GET | `/artifacts/:id/versions/:version` | Pinned version |
| POST | `/artifacts/:id/versions` | Revise |
| POST | `/artifacts/:id/archive` | Soft hide |
| POST | `/artifacts/:id/unarchive` | Restore |
| GET | `/artifacts/:id/download` | Bytes |
| POST | `/instances/:instanceId/mail-attachments` | Save refs |
| GET | `/instances/:instanceId/mail-attachments` | List refs |

Single-artifact mutations authorize with
`requireGrant(idResource("artifact", "id"), <action>)`. Trust-boundary bodies
are arktype. Text bodies over `MAX_ARTIFACT_CONTENT_BYTES` (15 MiB) are 413.
A file over `MAX_UPLOAD_BYTES` (10 MiB) is 413; a refused MIME is 415.

## `mountWorkflowArtifacts`

Required opts: `db`, `contentStore`, `resolveRunScope`. Optional: `agentToken`
(tried before the sidecar path), `uploadPolicy`, `maxBinaryBytes` (default
`MAX_UPLOAD_BYTES`), `maxContentChars` (default `64_000`).

Every route runs behind bearer middleware. There is no unauthenticated
collection-read case.

Authentication:

- `Authorization: Bearer <token>`
- `x-workflow-run-address: <run address>`

`resolveRunScope(bearerToken, runAddress)` returns
`{ tenantId, principalId, runId }` or `null` (401). When `agentToken` is
set, `verify` then `resolveRun` run first. A token whose tenant is not the
resolved run's tenant is the same bare 401 as a missing run — no existence
oracle.

Create/revise stamp `source: { origin: "workflow", runId }` and `generatedBy`
from the resolved scope. A body field of either name is never read.
`ownerPrincipalId` is `null` on workflow creates.

| Method | Path | Role |
| --- | --- | --- |
| POST | `/artifacts` | Create text |
| POST | `/artifacts/binary` | Base64 file bytes |
| POST | `/artifacts/link-file` | Link a path |
| GET | `/artifacts` | List (`kind` / `limit`) |
| GET | `/artifacts/recent` | Recent (default 10, max 50) |
| GET | `/artifacts/find` | Exact title (`title` required) |
| PATCH | `/artifacts/:id` | Revise |
| GET | `/artifacts/:id` | Read-back |
| GET | `/artifacts/:id/read` | Budgeted tool read |
| GET | `/artifacts/:id/chunk` | Windowed tool read |

Together these cover `ARTIFACT_TOOL_DEFINITIONS`. A missing row, another
tenant's row, or `skill-draft` is 404 on detail/revise, same as
`mountArtifacts`. Text over `maxContentChars` and binary over
`maxBinaryBytes` are 413.

Per-run rate limiting is not implemented here. Wrap `resolveRunScope` or the
mounted app.

## Persistence

Schema `artifacts`: tables `artifact`, `artifact_version`, `upload`,
`mail_attachment_ref`, plus ledger `artifacts.migrations`. Hard FKs into
`public.tenant` and `public.principal`. `ContentStore.put` runs inside the
caller's transaction.

## Development

```sh
git clone https://github.com/corbitsdev/corbits-artifacts.git
cd corbits-artifacts
bun install
docker run -d --name corbits-artifact-pg -p 5457:5432 \
-e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=artifact_core postgres:16
export ALLOW_DESTRUCTIVE_ARTIFACT_TESTS=1

bun run typecheck
bun run test # pretest dependency check, then unit + integration
bun run build # dist/ (JS + .d.ts)
bun run test:acceptance # builds, then examples/reference-host
```

Tests expect `postgres://postgres:postgres@localhost:5457/artifact_core`
(override with `ARTIFACT_DATABASE_URL`). Destructive tests require
`ALLOW_DESTRUCTIVE_ARTIFACT_TESTS=1` and an allowlisted database name
(`artifact_core`, or any name ending in `_test`). See
[CONTRIBUTING.md](./CONTRIBUTING.md).
74 changes: 74 additions & 0 deletions PRODUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# @corbits/artifacts — Product

## What it is

Persistent artifacts, versions, and file uploads as a mountable module for any
Interchange host. Backend only — this package ships no UI.

A host keeps the things agents and people make: text, links, uploaded files,
and the version history of each. `mountArtifacts` is the tenant-session
surface. `mountWorkflowArtifacts` is the parallel surface for a workflow run
that has no browser session. Both write the same rows.

## Why it exists

Agents produce files, drafts, and links that disappear unless the host stores
them. Every host that wants that store should not reimplement versions,
uploads, and tenant scoping. This package is the drop-in: mount it, run its
migrations, keep the bytes.

Two caller kinds exist and must not share one auth convention. A browser
session has Interchange `tenant` / `principal` and grants. A sidecar or
deployed agent has a bearer token and a run address. Mixing those into one
mount would make each harder to reason about, so the product is two mounts
over one store.

## Who it is for

- Interchange host operators who already have a Hono app, a Postgres pool, and
`RequireGrant`.
- Workflow hosts that authenticate sidecar or agent callers and need those
runs to create and read artifacts in the same tenant.
- Agent authors who pin `@corbits/artifacts/sidecar-bundle` so no agent owns
artifact client code.

There is nothing here for an end-user UI. Clients consume the HTTP surface the
host mounted.

## What users can do

- Persist an artifact with append-only versions, list and download it, and
soft-archive it without destroying history.
- Upload files through a pluggable `ContentStore` (inline Postgres bytes for a
minimal host; another backend without a rewrite).
- Authorize tenant-session mutations through the host's Interchange grants —
this package invents no owner, membership, or admin policy.
- Let a workflow run create, revise, find, and budget-read artifacts using
the host's sidecar (or agent-token) resolver.
- Grant agents the sidecar bundle so tool calls hit the run-scoped routes
through the hub credential handle.

## What it is not

- Not a service. It creates no app, opens no pool, and starts no background
work.
- Not a UI, directory, or session library.
- Not an authorization product. Grants are the host's. The workflow mount
trusts whatever `resolveRunScope` (and optional `agentToken`) returns.
- Not a parser. No PDF, spreadsheet, or text extractor lives here or is
planned.
- Not a rate limiter for runs. Per-run quota stays host-side.

## Goals

1. One store for tenant-session and workflow-run callers.
2. Host-owned app, pool, session, and grants.
3. Two mounts, two auth conventions, no mix.
4. Bytes and artifact row atomic on write; tenant-scoped on read.
5. Honest public docs: README for install and the mount snippet;
this file for intent; `ARCHITECTURE.md` for structure;
`IMPLEMENTATION.md` for wire and config.

## License

LGPL-2.1-only.
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ mountWorkflowArtifacts(workflowApi, {
resolveRunScope: (bearerToken, runAddress) =>
hub.resolveWorkflowRun(bearerToken, runAddress),
});
app.route("/workflow-artifacts", workflowApi);
app.route("/api/workflow-artifacts", workflowApi);
```

Agent tools live in `@corbits/artifacts/sidecar-bundle`. `examples/reference-host` is a complete `@intx/hub-api` host with this module mounted.
Expand All @@ -70,7 +70,9 @@ Agent tools live in `@corbits/artifacts/sidecar-bundle`. `examples/reference-hos

`mountArtifacts` registers tenant-session routes (list, import, upload, versions, download, archive) and authorizes through the host's `requireGrant`. `mountWorkflowArtifacts` is the parallel mount for sidecar/agent callers: a bearer plus run address, no browser session. Both persist rows in Postgres and blobs through a pluggable `ContentStore` (`InlineContentStore` for a minimal host). The host never shares app creation, pooling, or auth with this package.

See [ARCHITECTURE.md](./ARCHITECTURE.md) for the data model and mount options.
See [PRODUCT.md](./PRODUCT.md) for intent, [ARCHITECTURE.md](./ARCHITECTURE.md)
for the data model and mount options, and
[IMPLEMENTATION.md](./IMPLEMENTATION.md) for routes, headers, and ceilings.

## Development

Expand Down
Loading