[pull] main from activepieces:main

brain/knowledge/ai-intelligence/ai-providers.md

@@ -146,11 +146,7 @@ renders and preserves every real price; the cheapest in the set is 0.01.
 - **`/v1/ai-tools` is registered only in the CLOUD and ENTERPRISE branches of `app.ts`, but the AI Center page that reads it is not edition-gated** — so a Community admin opening the Capabilities tab fired `useAiToolConfigs`, got Fastify's `Route not found`, and the page showed the global "Failed to load data" dialog of the day. Shipped that way from #13911 until the tab was gated on `ApFlagId.EDITION` in the page. What made this class of bug hard to place is that the surface was then page-independent (raised from `QueryCache.onError` in `query-client.ts`) and React Query's 3 default retries meant it landed several seconds later on whatever page you navigated to next (the report was against `/platform/setup/general`). The tab now renders `DataFetchErrorState` in place instead, so a repeat would at least accuse the right page. When a screenshot's edition is in doubt, read the sidebar: **Billing & subscription** and **Usage** carry a lock only when `edition === COMMUNITY`, every other lock there is plan-driven. Any new EE-only route needs its UI entry point gated the same way, `enabled:` on the query or hiding the surface.
 - **The AI Center providers page renders its whole state machine off one query, so every branch needs an `isLoading` gate above it.** `ProvidersTab` derives the empty state, the grouped list *and* the `?config=<id>` detail view from `useAiProviderConfigs()`. While that query is pending, `data` is `undefined` — `configs` is `[]`, so the empty "Connect your first provider" screen paints for a frame, and a hard refresh on a `?config=` URL finds no matching row and falls through to the list before flipping to the detail page. The parent `SuspenseWrapper` does not cover it: the route chunk and the suspense-backed projects collection resolve first, then the component mounts with no providers yet. Any new branch added to this page belongs below the `if (isLoading) return <ProvidersSkeleton />` gate, not beside it.
 - **A save bar in the AI Center needs a full-height column, not `fixed` and not bare `sticky`.** The page sits inside *two* nested scrollers — `PlatformLayout`'s `flex flex-col flex-1 overflow-auto` and the AI Center's own `flex-1 min-w-0 overflow-auto` beside the `w-60` tab rail — so `fixed inset-x-0` centres the bar on the viewport, roughly a sidebar-width left of the content column it belongs to. Swapping it to `sticky bottom-4` fixes the centring for free but pins nothing when the content is shorter than the viewport: the bar just lands wherever the content ends, mid-page. The working shape is `min-h-full` + `flex flex-col` on the tab-content wrapper, `flex-1` on the `TabsContent`, `grow` on the panel root, then `sticky bottom-4 mt-auto` on the bar — `mt-auto` handles the short page, `sticky` handles the long one. `min-h-full` resolves only because every ancestor up to `SidebarInset h-full` has a definite height; that chain is what makes the whole thing work, so check it before reusing this elsewhere.
-
-### Key files
-
-Entry point: `aiProviderModule`, registered in `packages/server/api/src/app/app.ts` right after `aiProviderService(app.log).setup()`.
-
+- **A chat *tier* is our own idea, so it has no meaning on a provider we ship no curated model list for — resolve it by asking the key what it offers.** `ACTIVEPIECES_CHAT_TIERS` (fast/smart/premium) only maps to a real id through `ALLOWED_CHAT_MODELS_BY_PROVIDER`, which covers Anthropic, OpenAI, Google, Vertex and Activepieces. Everything else — Bedrock, Azure, Mistral, xAI, DeepSeek and the other OpenAI-compatible vendors — has an empty candidate list. The resolver used to invent an id from the tier (strip the vendor prefix, dots to dashes: `anthropic/claude-haiku-4.5` -> `claude-haiku-4-5`), which Bedrock rejected as "The provided model identifier is invalid" (GIT-1840, Pylon 5836). #15355 removed the invention and left a throw in its place, which was worse: every agent turn on those providers died with `ENTITY_NOT_FOUND: this AI provider key allows no text model a chat turn can run on` before the configured model handled anything (Pylon 6069). **The missing ingredient was never a smarter guess, it was the key's own model list.** `agentHelpers.resolveModelId` is now the single resolver for every surface: it tries our catalogue first and, only when that has nothing to say, reads `aiProviderService.listModels` (process-cached, cleared nightly by cron, already filtered by the key's `modelScope`), preferring a model whose id *contains* the tier's `nativeModelId` — Bedrock spells them `eu.anthropic.claude-sonnet-4-6-...`, so a tier still selects the model it names wherever the vendor hosts it, and otherwise degrades to the first text model the way `pickAllowedModel` already does for an admin-listed catalogue. Picking from a list the provider returned is the opposite of the fabrication #15355 removed. **Two traps, both learned the hard way.** First, an agent surface and generic chat are not symmetric: an agent *carries* a real model id (its picker lists `listModels` output unfiltered, precisely because these providers have no `ALLOWED_CHAT_MODELS_BY_PROVIDER` entry) while chat carries only a tier name, because `ChatModelSelector` falls back to tier options when `getCuratedChatModels` returns `undefined`. Second, **being listed proves the key can see a model, not that the account can invoke it** — so where a model is already settled the fast round reuses it rather than choosing again (`getAgentConfig` passes the model the turn resolved; the tool RPC reads the run's own model off the conversation or the agent via `pinnedModelOf` — and it has to read the *same* half the run did: an agent-page chat runs on `agent.draft` (`agent-conversation-controller.ts`), a Run Agent step on `agent.published`, so reading the wrong one silently lends the tool a stale model whenever the two differ). Resolving the fast tier independently once picked a Haiku the account could not invoke and failed a turn whose configured Sonnet worked fine. A borrowed model still has to pass the key's current `modelScope`, or resolution falls through to the list, which applies it. Nothing crosses the worker, so a rolling deploy has no version-skew window.
 - `packages/server/api/src/app/ai/` — backend module: provider controller, service, entity, module, plus the sibling ai-tool-config files
 - `packages/server/api/src/app/ai/providers/` — per-vendor strategies keyed by `AIProviderName`
 - `packages/server/api/src/app/platform/billing-provider.ts` — `assertCreditsAndAppSumoNotExceeded` credit gate

brain/knowledge/ai-intelligence/mcp-server.md

@@ -38,6 +38,9 @@ Exposes an Activepieces project as an MCP server so AI clients (Claude Desktop,
 
 ### Gotchas
 
+- **The consent page (`/mcp-authorize`) must treat an `ONBOARDING` token as signed out.** `isLoggedIn()` is only a JWT-expiry check; `GET /v1/projects` answers 403 for that principal, so gating on it alone rendered an empty project select with Authorize disabled forever. It now also checks `isOnboarding()` and sends the visitor to `/sign-in?from=…`, where the drawer resumes at the name step and returns to consent.
+- **`/verify-email` cannot see the OAuth `from=`** (the link is server-generated), so both `SignUpForm` and `SignInForm` store it via `pendingRedirect` in `navigation-utils` when they show the check-your-email note, and `/verify-email` hands it back as `/sign-in?from=…`. Only matters when passwordless is off — the code flow never leaves the page.
+- **Deny goes through `POST /v1/mcp-oauth/deny`**, which verifies the `authRequestId` JWT and returns the client's redirect with `error=access_denied` (it used to be `history.back()`, a no-op in the tab the client opened, so the client never heard back). The consent page only *decodes* the JWT for display: `clientName`, the platform/project scope, and `exp` so an expired request says "go back to {client}" instead of inviting a retry. The browser clock only drives that notice, never the buttons (a clock 10 minutes fast would otherwise lock the user out of a request the server still accepts); Authorize and Deny are disabled only after the server itself answers 400 `invalid_request`. Switch account is a full page load, not a `navigate`, because the query cache would otherwise hand the previous account's projects to the next one. It also preselects a single project and names the signed-in account with a Switch account link, because a browser already signed into another workspace would otherwise authorize it silently.
 - **`mcp_server.token` is dead — nothing reads it.** It is written by the `getOrCreate` defaults and by both `/rotate` routes (`mcpServerService.rotateToken` / `rotatePlatformToken`), and consulted by **no authenticator**, so "rotating" it rotates a secret that grants nothing. It is still on the public `McpServer` zod schema, so the API keeps shipping a secret-shaped 72-char string that authenticates nothing — do not reach for it as a credential, and do not tell a self-hoster to. The settings panel is consistent with reality already (`mcp-credentials.tsx` renders the URL and *"Authentication is handled via OAuth"*, never a token). Deleting the column, the two routes, and the schema field is a breaking API-response change and has not been done.
 - **`mcp_oauth_token.clientKey` is decided once, at sign-in.** `exchangeCode` derives it from the registration's
   redirect URIs via `mcpOAuthClientIdentity`, so the grants list can filter and group in SQL instead of loading

brain/knowledge/data-storage-observability/index.md

@@ -65,6 +65,19 @@ Gotchas:
 - The Google `USER_SIGNED_UP` audit event is gated on `signedUp` too; before that it fired on every login.
 - SAML users are never stamped: they are provisioned by their platform admin, not by a campaign link.
 
+### Product Lifecycle Telemetry
+
+Server-side PostHog events for the moments between sign-up and payment: `onboarding.completed` (name step done), `invite.sent` / `invite.accepted`, `checkout.started`, `plan.changed`, `plan.cancelled`, `plan.reactivated`, `trial.started`, plus `flow.published` from the service so API and approval publishes count too. Billing events are emitted by `platform-plan-telemetry.ts`; the entitlement refresh in `autumn-utils.ts` compares the plan before and after and emits `trial.started` or `plan.changed` when it changed, skipping free plans (`FREE`, `FREE_LEGACY`, `APPSUMO`). Platforms are PostHog groups (`groupIdentify`, type `platform`) set on creation and refreshed on plan change. Every event, person and group carries `deployment` (`DeploymentKind`: cloud, self_hosted, dev) because self-hosted instances report into the same project.
+
+Gotchas:
+- Pre-login email-code events are keyed by the identity id; `aliasIdentity` merges that person into the user once the user exists, and `trackForIdentity` in `passwordless-auth.service.ts` switches to the user id as soon as one exists for identity+platform.
+- Billing and onboarding events are in `CLOUD_ONLY_TELEMETRY_EVENTS`; the tracked-events catalog on the platform settings page hides groups whose events are all cloud-only, so `billing` never shows on self-hosted.
+- In a development environment the web sends nothing to PostHog unless `localStorage.ap_posthog_dev` is `'1'`.
+- **The event is `plan.changed`, not `plan.upgraded`** — the entitlement refresh only knows the plan differs, not which way, so a downgrade reaches the same code path. Direction is derived in PostHog from the `previousPlan` and `plan` properties. A trial converting to paid emits **nothing**: it is the same `planId` with the trial simply ended, so the plan-equality check returns early, and no previous trialing state is persisted anywhere to compare against (`trialEndsAt` comes from the live Autumn subscription, `platform_plan` has no trial column, and the customer state cache holds only balances).
+- **`flow.published` is emitted once, by `flowService`** — the browser capture in `flow-hooks.tsx` was removed when the service emit landed, because the two fired on exactly the same UI publishes. The service emit also counts publishes no user pressed Publish for: the connection-swap republish in `app-connection.handler.ts` and the git-sync project-release import both dispatch `LOCK_AND_PUBLISH`.
+- **A platform group's `name` only leaves Cloud** (`pickPlatformGroupPii`): a self-hosted platform name is usually the customer's company name, the same reason `pickTelemetryPii` exists. The group itself is still created on every edition, so `plan`, `createdAt` and `deployment` stay groupable off Cloud.
+- **Google and SAML emit `signed.in` only for a returning user**, in the `else` branch of the `signedUp` check, mirroring the password path where sign-up emits `signed.up` and only a later sign-in emits `signed.in`. Emitting it on every claim made SSO sign-in counts run high against password.
+
 ### Flow Failure Alerts (EE)
 
 Email on flow-run failure. First failure per flowVersion per 24h window sends; rest suppressed via Redis counter `flow_fail_count:<flowVersionId>` (1-day TTL). Personal projects: single owner-only receiver toggle; team projects: any number of receivers. Platform admins can bulk sub/unsub across projects (max 5 concurrent). Receivers stored/compared lowercase. Edition check (`paidEditions`) in service, no plan flag. No Issues feature — email links straight to the run page. EE/Cloud only.

brain/knowledge/flows-execution/flows.md

@@ -8,7 +8,7 @@ Flows are the core automation primitive: a versioned directed graph of trigger +
 
 ### Entities & services
 - **Flow** — persistent record: status (ENABLED/DISABLED), folderId, publishedVersionId, externalId, operationStatus (NONE/DELETING/ENABLING/DISABLING), ownerId, createdBy (`FlowCreator`: `{type: MCP|AGENT, id}`, null for humans → "AI" badge).
-- **FlowVersion** — immutable-once-LOCKED snapshot of the graph. DRAFT is editable; current schemaVersion is `'22'`. Holds trigger (full graph JSONB), connectionIds, agentIds, notes.
+- **FlowVersion** — immutable-once-LOCKED snapshot of the graph. DRAFT is editable; current schemaVersion is `'27'` (`LATEST_FLOW_SCHEMA_VERSION`). Holds trigger (full graph JSONB), connectionIds, agentIds, notes.
 - **Folder** — simple per-project grouping, case-insensitive unique.
 - Core service: `flow.service.ts`; single controller endpoint `POST /v1/flows/:id`.
 
@@ -24,6 +24,7 @@ Flows are the core automation primitive: a versioned directed graph of trigger +
 - **A CODE step's compiled size is its `packageJson`, not its code — bundling *inlines* `node_modules`, it does not exclude it.** This gets assumed backwards a lot: there is no `node_modules` at runtime precisely *because* esbuild inlines every dependency into the step's `index.js`. Measured on cloud, Aug 2026: a step with **1,870 characters** of user source and `{"pdfkit":"0.14.0","aws-sdk":"2.1531.0","uuid":"9.0.1"}` compiles to **24.13 MB**, of which 24.13 MB is `node_modules` — **21.16 MB of it `aws-sdk` alone**. v2 of that SDK resolves its ~200 service clients by dynamic `require`, so esbuild cannot tree-shake it and inlines all of them; `@aws-sdk/client-*` (v3) would be a few hundred KB. Fleet-wide there were 13,918 compiled steps totalling 1.4 GB, **61 of them over 10 MB**, and per *flow* the totals reach **190.7 MB across 40 code steps**. That per-flow number is the one that matters operationally, because `flowBundleStore.publish` holds a flow's entire compiled output in memory at once (three copies — see the OOM gotcha on [[workers]]). To find the offenders: esbuild leaves `// node_modules/<pkg>/…` markers in the output, so you can attribute bytes per package by summing the lines between markers.
 - **Step output nesting (schema v21+)**: every step output is wrapped as `{ output, error? }`; expressions must use the `['output']` accessor. The v20→v21 migration rewrites existing expressions via `expression-rewriter`.
 - **Migrations run on every single-version read, including the worker's own `getFlowVersion`** (`findOne` → `flowVersionMigrationService.migrate`). So no run can execute an unmigrated version, and restoring an old version migrates it forward too. The listing path is the exception: `flowVersionService.list()` paginates the repository directly and hands back whatever schema version the row holds, so a caller reading the version history can see unmigrated rows. The result is persisted, then short-circuits at `LATEST_FLOW_SCHEMA_VERSION`. The old version is saved to `backupFiles[oldSchemaVersion]` first — **that** is what a restore reads, so there's no need to keep an old code path alive "just in case". Nothing restores on its own: `flowVersionBackupService.get()` has no callers, so recovering a version is a deliberate manual step. A migration that throws pages on-call (`FLOW_MIGRATION_FAILED`).
+- **Template migrations run on a flow version that is not stored.** `migrateFlowVersionTemplate` (the `IMPORT_FLOW` pre-validation and the template controllers) builds a stand-in version with `flowId: ''` and passes no `MigrationContext`, so there is no project or platform either. A migration that looks anything up by flow id must treat "not found" as "nothing stored", never throw: the v23 piece-upgrade audit event used `findOneByOrFail`, which turned every import of an old template holding a register-listed piece into a 500 (the event-destinations "Generate handler flow" hit it with webhook `0.1.33`).
 - **Continue on Failure**: CODE/PIECE steps with `continueOnFailure.value: true` carry `onSuccess`/`onFailure` sub-trees under `settings.errorHandlingOptions.continueOnFailureBranches`.
 - **`addActionUtils.clone()` is the single chokepoint for renaming copied steps and rewriting their `{{ }}` references** — paste, duplicate step, and duplicate branch all route through it. It walks the **whole `settings`** deliberately, not just `settings.input`: router conditions live at `settings.branches[].conditions[][].firstValue/.secondValue` and loop expressions at `settings.items`, and an `'input' in settings` guard silently skipped both (GIT-1075). Two things to respect when touching it: `settings.sourceCode` is excluded on purpose (it holds a user program, and a literal `{{ … }}` in a code step was being rewritten), and the remap must stay a **single pass** over the name map — applying one rename at a time rewrites its own output whenever a copied step's new name equals another copied step's old name, which happens on cross-flow paste into a flow that lacks the clipboard's names.
 - **Paste order follows selection order, not flow order.** `_getActionsForCopy`'s `.sort((a, b) => allSteps.indexOf(a) - ...)` compares deep clones, so `indexOf` is always `-1` and the sort is a no-op. Harmless for a single step; it decides the chaining order for a multi-step paste.

packages/core/shared/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@activepieces/shared",
-  "version": "0.177.0",
+  "version": "0.178.0",
   "type": "commonjs",
   "sideEffects": false,
   "main": "./dist/src/index.js",