Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ never seeds data on a client's behalf.
- **Parse at every trust boundary.** arktype schemas for env, request
bodies, and external data; never `as T` untrusted input.
- **Custom DB tables live on their own Postgres schema**, with foreign keys
back to Interchange's tenant/principal tables.
back to Interchange's tenant/principal tables. Each package ships one
idempotent migration, which the hub applies itself at boot.
- **Ancestor-walkable except grants.** Credentials, tools, and definitions
inherit down the tenant tree; grants never do.
- **This repo is public.** No secrets or credentials, ever — `.env.example`
Expand Down Expand Up @@ -107,5 +108,3 @@ passing; rebuilding coverage for the area it covered happens under CL-8150.
disagrees with it is wrong until a review changes the doc
- [docs/GLOSSARY.md](docs/GLOSSARY.md) — product-term to platform-term mapping
- [docs/local-dev.md](docs/local-dev.md) — running fully local with Ollama
- [docs/package-migrations.md](docs/package-migrations.md) — how a
package's own database migrations are written and applied
8 changes: 4 additions & 4 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@ stock workflow-deploy route. `tools/*` and `skills/*` are ordinary

Custom tables — `webhook-triggers`, `agent-directory`, and so on — live
on their own Postgres schema, each with foreign keys back to
Interchange's `tenant`/`principal` tables in `public`. See
[docs/package-migrations.md](docs/package-migrations.md) for how a
package's own migrations are written and applied. Anything Interchange
already does natively is deleted rather than kept as a parallel path.
Interchange's `tenant`/`principal` tables in `public`. Each package ships
one idempotent migration, which the hub applies itself at boot. Anything
Interchange already does natively is deleted rather than kept as a
parallel path.

## Vendoring

Expand Down
5 changes: 2 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,8 @@ Thanks for your interest in Corbits Workbench.
5. Never commit secrets. `.env.example` is the only tracked env file.
6. Never vendor code without a ledger row and kill date in
[VENDORED.md](VENDORED.md).
7. A package that owns its own product tables follows
[docs/package-migrations.md](docs/package-migrations.md): literal SQL,
a package-owned ledger table, applied transactionally.
7. A package that owns its own product tables ships one idempotent,
literal-SQL migration, applied by the hub itself at boot.
8. Security issues go through [SECURITY.md](SECURITY.md), never a public
issue.
9. Changed a publishable `workflows/*` or `packages/*-tools` package's
Expand Down
12 changes: 6 additions & 6 deletions IMPLEMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ check` (typecheck, lint, fmt check, test) is the one gate CI runs.

The platform schema (`public`) comes entirely from `@intx/db`'s own
migrations — this repo authors no SQL for it. Every package that owns
product tables gets its own Postgres schema, its own literal-SQL migration
list, and its own ledger table tracking which of its migrations have run;
`scripts/db-setup.ts` applies the platform's migrations and then each
installed package's, in an explicit order. See
[docs/package-migrations.md](docs/package-migrations.md) for the exact
rules and shapes. Custom tables are hard-removed when retired — no drop
product tables gets its own Postgres schema and one literal-SQL migration,
every statement `IF NOT EXISTS` so it needs no ledger table; the hub
applies the platform's migrations and then each mounted package's own,
in an explicit order, once at boot (`apps/hub/src/migrate.ts`).
`scripts/db-setup.ts` just creates the database if missing and calls the
same function. Custom tables are hard-removed when retired — no drop
migrations, no dead columns kept around; a schema cutover means resetting
the local database, not writing a data migration.

Expand Down
1 change: 0 additions & 1 deletion apps/hub/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
"@corbits/codex-provider": "github:corbitsdev/corbits-codex-provider#a1f475f341261c05fa8dade0c82f1246c175b5b3",
"@corbits/cron": "workspace:*",
"@corbits/error-sink": "workspace:*",
"@corbits/hub-api-client": "workspace:*",
"@corbits/inference-catalog": "workspace:*",
"@corbits/mailbox": "github:corbitsdev/corbits-mailbox#e85b81e4a2e0cc318dd33a4f0f92337b6fac7f38",
"@corbits/memory": "github:corbitsdev/corbits-memory#e74da20f148a302dff5400915fe504ee2395e913",
Expand Down
59 changes: 59 additions & 0 deletions apps/hub/src/migrate.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
// Boot-time migrations for the hub, run once before `createHubServer`
// mounts anything. Interchange's own platform schema is authored and
// applied by `@intx/db`'s `runMigrations`; every mounted Corbits
// library that owns its own product tables applies its own single,
// idempotent migration right after, exactly the way upstream
// Interchange applies its schema — no separate migration-runner
// service, no ledger table, just the hub doing the work itself.
import { runMigrations } from "@intx/db";
import { applyCronMigrations } from "@corbits/cron";
import { applyWebhookTriggersMigrations } from "@corbits/webhook-triggers";
import { createMailboxDb, runMailboxMigrations } from "@corbits/mailbox";
import { runArtifactMigrations } from "@corbits/artifacts";
import { runMemoryMigrations } from "@corbits/memory";

export interface HubMigrateConfig {
host: string;
port: number;
user: string;
password: string;
database: string;
schema?: string;
}

function databaseUrlFrom(config: HubMigrateConfig): string {
return (
`postgres://${encodeURIComponent(config.user)}:${encodeURIComponent(config.password)}` +
`@${config.host}:${String(config.port)}/${config.database}`
);
}

/**
* Apply the platform schema plus every mounted Corbits package's own
* migration, in the order the hub mounts them: cron, webhook-triggers,
* mailbox, artifacts, memory. `db` is the same drizzle handle the rest
* of the hub uses, so `runArtifactMigrations` (which takes a drizzle
* db, not a URL) shares the one connection pool rather than opening
* its own.
*/
export async function migrateHub(
config: HubMigrateConfig,
db: Parameters<typeof runArtifactMigrations>[0],
): Promise<void> {
const databaseUrl = databaseUrlFrom(config);
const schema = config.schema ?? "public";

await runMigrations(config, { schema });
await applyCronMigrations(databaseUrl, { tenantSchema: schema });
await applyWebhookTriggersMigrations(databaseUrl);

const mailboxDb = createMailboxDb(databaseUrl);
try {
await runMailboxMigrations(mailboxDb.db);
} finally {
await mailboxDb.close();
}

await runArtifactMigrations(db);
await runMemoryMigrations(databaseUrl);
}
16 changes: 10 additions & 6 deletions apps/hub/src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ import {
mountMailbox,
} from "@corbits/mailbox";
import { createMemory, loadMemoryConfig } from "@corbits/memory";
import { applyCronMigrations, createCronTicker, mountCron } from "@corbits/cron";
import { createCronTicker, mountCron } from "@corbits/cron";
import {
createHubMailboxAuthorizeSender,
createHubPersistMailWithSessionEnsure,
Expand All @@ -86,7 +86,6 @@ import {
InlineContentStore,
mountArtifacts,
mountWorkflowArtifacts,
runArtifactMigrations,
type WorkflowArtifactEnv,
} from "@corbits/artifacts";
import {
Expand All @@ -96,6 +95,7 @@ import {
import type { DB } from "@intx/db";
import { sidecar, workflowRun } from "@intx/db/schema";
import path from "node:path";
import { migrateHub } from "./migrate";

// The same condition registry `mountHubRoutes` builds by default when no
// registry is supplied -- kept as one local constant so every Corbits
Expand Down Expand Up @@ -184,14 +184,20 @@ export async function createHubServer({
// dedicated, droppable schema. Production deployments leave it
// unset and run against postgres' default search_path.
const pgSchema = process.env["PG_SCHEMA"];
const { db } = createDB({
const dbConfig = {
host: process.env["DB_HOST"] ?? "localhost",
port: Number(process.env["DB_PORT"] ?? 5432),
user: process.env["DB_USER"] ?? "postgres",
password: process.env["DB_PASSWORD"] ?? "postgres",
database: process.env["DB_NAME"] ?? "interchange",
...(pgSchema !== undefined && { schema: pgSchema }),
});
};
const { db } = createDB(dbConfig);

// Platform schema plus every mounted Corbits package's own migration,
// applied once at boot before anything below reads or writes the
// database.
await migrateHub(dbConfig, db);

const auth = createAuth(db);

Expand Down Expand Up @@ -493,8 +499,6 @@ export async function createHubServer({
}),
};

await runArtifactMigrations(db);
await applyCronMigrations(corbitsDatabaseUrl);
const artifactContentStore = InlineContentStore;

{
Expand Down
29 changes: 0 additions & 29 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

57 changes: 0 additions & 57 deletions docs/package-migrations.md

This file was deleted.

1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
"build": "bun run scripts/run-all.ts build",
"test": "bun run scripts/run-all.ts test",
"dev": "bun run scripts/dev.ts",
"publish-tools": "bun scripts/publish-tools.ts",
"setup": "bun scripts/db-setup.ts",
"setup:memory": "bun run scripts/setup-memory.ts",
"reset": "bun scripts/reset.ts"
Expand Down
8 changes: 1 addition & 7 deletions packages/cron/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,7 @@ export {
type CronField,
type ZonedParts,
} from "./cron";
export {
cronScheduleTable,
cronMigrations,
applyCronMigrations,
type CronMigration,
type ApplyCronMigrationsReport,
} from "./schema";
export { cronScheduleTable, applyCronMigrations } from "./schema";
export {
createCronTicker,
cronSenderAddress,
Expand Down
2 changes: 1 addition & 1 deletion packages/cron/src/migrations.ts
Original file line number Diff line number Diff line change
@@ -1 +1 @@
export { applyCronMigrations, type ApplyCronMigrationsReport } from "./schema";
export { applyCronMigrations } from "./schema";
Loading
Loading