Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/bench-provisioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,10 @@ Connecting hands the person forward immediately, and the wait on the
other side is as short as it can honestly be: the setup agent (Myra)
leads `DEFAULT_WORKFLOWS`, so she is deployed before anything else, and
the land hop drops the person into her room the moment she can answer.
The digest, recurring-task and research seeds finish behind them —
nobody needs a scheduled digest in their first second.
The other catalog workflows (digest, recurring-task, research) are not
seeded at all — they are deployable through the catalog instantiate route
(CL-7073) once someone picks one, not on a critical path nobody is waiting
on.

While she is still coming up, the shared warm loader is the whole
screen: one honest headline and a rotating tip, never an internal count.
Expand Down
16 changes: 10 additions & 6 deletions docs/local-rip.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,12 @@ Either path:
`packages/connections/src/credential-test.ts`) before storing anything;
2. plants it as a credential on your bench alongside that provider's
curated model catalog;
3. deploys and (unlike the OAuth callback's own fast half) confirms every
default workflow the platform ships: **echo**, **assistant**, and
**workbench-digest** (`packages/seeding/src/seed.ts`'s
`DEFAULT_WORKFLOWS`).
3. deploys and (unlike the OAuth callback's own fast half) confirms the
one default workflow every real signup gets: **assistant**
(`packages/seeding/src/seed.ts`'s `DEFAULT_WORKFLOWS`, CL-7074).
**echo** and **workbench-digest** are on-demand `CATALOG_WORKFLOWS`
entries now — deployed only when something asks for them by name,
never automatically at signup.

Expect the page to show a short "setting up your workbench" wait while
`/complete-setup` polls, then land on a "Your first routines are running"
Expand All @@ -109,8 +111,10 @@ through the hub's native asset REST routes). Descendants inherit it;
`seedTenant` does not pack. Isolated
tests run with no explicit tenant config so the walkthrough's personal
bench is itself the root — then the same publish happens once onto that
bench, and **echo**, **workbench-digest**, and **assistant** all come up
live. `scripts/e2e/local-rip.test.ts` asserts exactly that. The default
bench, and **assistant** comes up live by default; `scripts/e2e/local-rip.test.ts`
also deploys **echo** and **workbench-digest** from `CATALOG_WORKFLOWS`
onto that same bench (the on-demand path, exercised directly rather
than assumed) to prove they still come up live too. The default
self-serve story is the other way: the hub ensures a root tenant at boot
(`WORKBENCH_DEFAULT_TENANT`, default `workbench`), and first-login
personal benches parent under it.
Expand Down
28 changes: 24 additions & 4 deletions docs/seed-reconciliation.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,30 @@ answers 503 — never a 404, which would read as "no such template".
## Default scheduled workflows (onboarding / boot-time seeding)

Seed never POSTs `/routines`. Native `ScheduleTrigger` ticks digest;
last-30-days-research stays a deployed workflow in `DEFAULT_WORKFLOWS`,
not a wrapper row. `scripts/db-setup.ts` drops a leftover `routines`
schema after the digest enablement handoff. There is no preset-wrapper
prune: seed does not plant wrapper rows.
last-30-days-research stays a deployed workflow, not a wrapper row.
`scripts/db-setup.ts` drops a leftover `routines` schema after the
digest enablement handoff. There is no preset-wrapper prune: seed does
not plant wrapper rows.

## Default vs. on-demand catalog workflows (CL-7074)

`DEFAULT_WORKFLOWS` (`packages/seeding/src/seed.ts`) is the set every
real signup gets automatically: `assistant` (Myra), and nothing else. A
fresh bench used to also pay a git push and a sidecar probe for `echo`,
`workbench-digest`, and `last-30-days-research` — three workflows
nobody had asked for yet. Those three now live in `CATALOG_WORKFLOWS`,
same shape (`DefinitionWithAgentSteps`-backed `DefaultWorkflow`
entries), deployed the same way (`ensureWorkflowAsset` → `pushWorkflow`
→ `ensureDeployment`), but only when something asks for one by name —
the catalog/instantiate surface, or a test standing up its own fixture
bench — never automatically at signup.

There is no orphan-retire for an entry that moved from
`DEFAULT_WORKFLOWS` to `CATALOG_WORKFLOWS`: an asset already deployed
on an existing bench from before the move is left exactly as it is.
`CATALOG_TEST_WORKFLOWS` remains the separate, never-reaches-a-real-
signup set for workflows that exist only to exercise the platform
continuously.

## Default skills (boot-time seeding)

Expand Down
66 changes: 19 additions & 47 deletions packages/onboarding/test/complete-credential.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -478,12 +478,7 @@ describe("completeCredentialSetup", () => {
kind: "seeded",
tenantId: TENANT_ID,
tenantSlug: TENANT_SLUG,
workflows: [
"assistant",
"echo",
"workbench-digest",
"last-30-days-research",
],
workflows: ["assistant"],
});
expect(seedCatalogCalls).toHaveLength(1);
expect(seedTenantCalls[0]?.model.provider).toBe("anthropic");
Expand Down Expand Up @@ -528,12 +523,7 @@ describe("completeCredentialSetup", () => {
kind: "seeded",
tenantId: TENANT_ID,
tenantSlug: TENANT_SLUG,
workflows: [
"assistant",
"echo",
"workbench-digest",
"last-30-days-research",
],
workflows: ["assistant"],
});
expect(seedCatalogCalls).toHaveLength(1);
expect(seedTenantCalls).toHaveLength(1);
Expand Down Expand Up @@ -578,12 +568,7 @@ describe("completeCredentialSetup", () => {
kind: "seeded",
tenantId: TENANT_ID,
tenantSlug: TENANT_SLUG,
workflows: [
"assistant",
"echo",
"workbench-digest",
"last-30-days-research",
],
workflows: ["assistant"],
});
expect(seedCatalogCalls).toHaveLength(1);
expect(seedCatalogCalls[0]?.provider).toBe("groq");
Expand Down Expand Up @@ -970,12 +955,7 @@ describe("completeCredentialSetup", () => {

expect(result.kind).toBe("seeded");
if (result.kind === "seeded") {
expect(result.workflows).toEqual([
"assistant",
"echo",
"workbench-digest",
"last-30-days-research",
]);
expect(result.workflows).toEqual(["assistant"]);
}
});

Expand Down Expand Up @@ -1458,8 +1438,8 @@ describe("completeCredentialSetup", () => {
// (one POST each for model and offering); the rest of the chain is
// still a single provider/credential row.
const anthropicCatalogSize = CATALOG_SEEDS.anthropic.models.length;
expect(assetCreatePosts).toBe(4);
expect(deploymentCreatePosts).toBe(4);
expect(assetCreatePosts).toBe(1);
expect(deploymentCreatePosts).toBe(1);
expect(catalogModelCreatePosts).toBe(anthropicCatalogSize);
expect(catalogProviderCreatePosts).toBe(1);
expect(catalogOfferingCreatePosts).toBe(anthropicCatalogSize);
Expand All @@ -1468,8 +1448,8 @@ describe("completeCredentialSetup", () => {
// existing row rather than leaving it untouched (the CL-6103 fix,
// updated by CL-6123 to no longer require a probe first).
expect(credentialRotatePatches).toBe(1);
expect(assets.length).toBe(4);
expect(deployments.length).toBe(4);
expect(assets.length).toBe(1);
expect(deployments.length).toBe(1);
});

test("a pasted key with no metadata stays an ordinary api_key credential", async () => {
Expand Down Expand Up @@ -1563,12 +1543,7 @@ describe("completeCredentialSetup", () => {
principalId: PRINCIPAL_ID,
tenantDomain: "alice-user1.bench.local",
deployed: [],
pending: [
"assistant",
"echo",
"workbench-digest",
"last-30-days-research",
],
pending: ["assistant"],
message: "Your workbench is ready — agents will come online shortly.",
});
});
Expand Down Expand Up @@ -1767,12 +1742,7 @@ describe("ensureSeeded (the slow half)", () => {

expect(result).toEqual({
kind: "seeded",
workflows: [
"assistant",
"echo",
"workbench-digest",
"last-30-days-research",
],
workflows: ["assistant"],
});
expectNoConfirmation(seedTenantCalls);
expect(seedTenantCalls[0]?.model.provider).toBe("anthropic");
Expand All @@ -1783,6 +1753,9 @@ describe("ensureSeeded (the slow half)", () => {
// (bench-provisioning's `runOnce` → `ensureSeeded` → `seedTenant`),
// and `seedTenant` works through the list in order at ~20s each — so
// the order handed in here is the order a fresh signup experiences.
// CL-7074 narrowed DEFAULT_WORKFLOWS to just the setup agent, so
// "before anything else" is now trivially satisfied by being the
// only entry — this still pins that fact rather than assuming it.
const workflowOrder: string[] = [];

await ensureSeeded({
Expand All @@ -1806,7 +1779,6 @@ describe("ensureSeeded (the slow half)", () => {
});

expect(workflowOrder[0]).toBe(SETUP_AGENT_ASSET_NAME);
expect(workflowOrder.length).toBeGreaterThan(1);
});

test("two overlapping calls for the same tenant never double-deploy — the same 409-then-list tolerance seedTenant already has", async () => {
Expand Down Expand Up @@ -1984,10 +1956,10 @@ describe("ensureSeeded (the slow half)", () => {

expect(first.kind).toBe("seeded");
expect(second.kind).toBe("seeded");
expect(assetCreatePosts).toBe(4);
expect(deploymentCreatePosts).toBe(4);
expect(assets.length).toBe(4);
expect(deployments.length).toBe(4);
expect(assetCreatePosts).toBe(1);
expect(deploymentCreatePosts).toBe(1);
expect(assets.length).toBe(1);
expect(deployments.length).toBe(1);
});

// CL-6264: tonight's live failure — completeCredentialSetup ->
Expand Down Expand Up @@ -2056,8 +2028,8 @@ describe("ensureSeeded (the slow half)", () => {

expect(result).toEqual({
kind: "seeded-pending-agents",
deployed: ["assistant", "echo"],
pending: ["workbench-digest", "last-30-days-research"],
deployed: ["assistant"],
pending: [],
message: "Your workbench is ready — agents will come online shortly.",
});
});
Expand Down
21 changes: 12 additions & 9 deletions packages/onboarding/test/complete-setup-routes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -631,7 +631,13 @@ describe("POST /complete-setup", () => {
// itself — deploying only what is missing on a later pass — is
// covered by "a half-provisioned bench keeps its row and converges on
// a later pass" in ./bench-provisioning.test.ts.
test("a bench whose agents are not all live keeps its pending row for the drain", async () => {
//
// CL-7074 narrowed DEFAULT_WORKFLOWS to just the setup agent, so
// "partway through" no longer means "one of several live, the rest
// pending" — it means the one default workflow's asset exists but
// has not gone live yet (the sidecar push landed, the deploy
// confirmation has not).
test("a bench whose agent is not yet live keeps its pending row for the drain", async () => {
const liveWorkflow = DEFAULT_WORKFLOWS[0];
if (liveWorkflow === undefined) {
throw new Error("DEFAULT_WORKFLOWS is empty");
Expand All @@ -653,10 +659,9 @@ describe("POST /complete-setup", () => {
})),
),
);
// Only the first workflow made it live; the rest are still waiting
// on the drain.
// The asset exists, but no deployment has gone live yet.
hub.get(`/api/tenants/${TENANT_ID}/workflows/deployments`, (c) =>
c.json([{ definitionAssetId: "ast_0", status: "deployed" }]),
c.json([]),
);
const server = Bun.serve({ port: 0, fetch: hub.fetch });
const pendingSeedStore = createInMemoryPendingSeedStore(testCipher());
Expand Down Expand Up @@ -691,11 +696,9 @@ describe("POST /complete-setup", () => {
kind: "provisioning",
tenantId: TENANT_ID,
tenantSlug: TENANT_SLUG,
// The one live workflow here IS the setup agent — she deploys
// first (CL-6462), so this bench is already chat-ready.
setupAgentReady: true,
deployed: [liveWorkflow.assetName],
pending: DEFAULT_WORKFLOWS.slice(1).map((w) => w.assetName),
setupAgentReady: false,
deployed: [],
pending: [liveWorkflow.assetName],
});

// Not finished yet — clearing the row here would strand the
Expand Down
47 changes: 15 additions & 32 deletions packages/onboarding/test/connect-deploys-nothing.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -282,8 +282,17 @@ describe("POST /complete — connecting deploys nothing", () => {
});

describe("GET /provisioning-status", () => {
test("reports live progress while agents are still coming online", async () => {
const { hub } = fakeHub({ seededWorkflows: ALL_WORKFLOWS.slice(0, 2) });
// CL-7074 narrowed DEFAULT_WORKFLOWS to just the setup agent, so the
// "some deployed, some still pending" progress bar this route used to
// report (CL-6462, when the default set was echo/assistant/
// workbench-digest) can no longer happen for a real bench: with one
// default workflow, provisioning-status is binary — nothing deployed
// yet (`provisioning`, setup agent not ready) or fully deployed
// (`ready`). This test replaces the old two-test "partial progress"
// coverage with that binary reality; `setupAgentReady` and
// `kind: "ready"` are asserted at their other call sites below.
test("reports provisioning, with the setup agent not yet ready, before she deploys", async () => {
const { hub } = fakeHub({ seededWorkflows: [] });
const server = Bun.serve({ port: 0, fetch: hub.fetch });
const store = createInMemoryPendingSeedStore(testCipher());
try {
Expand All @@ -296,42 +305,16 @@ describe("GET /provisioning-status", () => {
const response = await app.request("/api/onboarding/provisioning-status");
const body = (await response.json()) as {
kind: string;
setupAgentReady: boolean;
deployed: string[];
pending: string[];
};

expect(response.status).toBe(200);
expect(body.kind).toBe("provisioning");
expect(body.deployed).toEqual(ALL_WORKFLOWS.slice(0, 2));
expect(body.pending).toEqual(ALL_WORKFLOWS.slice(2));
} finally {
server.stop(true);
}
});

test("says the setup agent is ready the moment she is live, while the rest are still pending", async () => {
// CL-6462: this one flag is what lets the land hop drop someone into
// a conversation without waiting out the whole seed set.
const { hub } = fakeHub({ seededWorkflows: [SETUP_AGENT_ASSET_NAME] });
const server = Bun.serve({ port: 0, fetch: hub.fetch });
const store = createInMemoryPendingSeedStore(testCipher());
try {
const app = mountAuthenticated(
createOnboardingRoutes(
routeDeps({ hubUrl: `http://localhost:${server.port}`, store }),
),
);

const response = await app.request("/api/onboarding/provisioning-status");
const body = (await response.json()) as {
kind: string;
setupAgentReady: boolean;
pending: string[];
};

expect(body.kind).toBe("provisioning");
expect(body.setupAgentReady).toBe(true);
expect(body.pending.length).toBeGreaterThan(0);
expect(body.setupAgentReady).toBe(false);
expect(body.deployed).toEqual([]);
expect(body.pending).toEqual(ALL_WORKFLOWS);
} finally {
server.stop(true);
}
Expand Down
9 changes: 4 additions & 5 deletions packages/onboarding/test/provision.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -860,11 +860,10 @@ describe("provisionPersonalTenantIfNeeded", () => {
seeded: true,
tenantId: "ten_new",
});
// Attempt 1 fails creating the echo asset. The retry re-runs from
// scratch: one create call per default workflow — echo, assistant,
// workbench-digest, last-30-days-research — on top of the one failed
// attempt.
expect(assetCreateAttempts).toBe(5);
// Attempt 1 fails creating the (only) default workflow's asset. The
// retry re-runs from scratch: one create call for the default set
// (assistant, CL-7074), on top of the one failed attempt.
expect(assetCreateAttempts).toBe(2);
});

test("a fully seeded personal bench reports existing-member with seeded: true, and backfills a grant added to SEED_GRANTS after it was provisioned", async () => {
Expand Down
1 change: 1 addition & 0 deletions packages/seeding/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export type {
} from "./seed";
export {
CATALOG_TEST_WORKFLOWS,
CATALOG_WORKFLOWS,
DEFAULT_WORKFLOWS,
SEED_GRANTS,
inferenceCredentialName,
Expand Down
Loading
Loading