[pull] main from activepieces:main

.github/workflows/ci.yml

@@ -149,7 +149,7 @@ jobs:
           set -euo pipefail
           pids=()
 
-          npx turbo run test --filter=@activepieces/engine --filter=@activepieces/shared --filter=@activepieces/sandbox --filter=@activepieces/ai-providers --filter=@activepieces/pieces-framework --filter=web &
+          npx turbo run test --filter=@activepieces/engine --filter=@activepieces/shared --filter=@activepieces/sandbox --filter=@activepieces/ai-providers --filter=@activepieces/core-execution --filter=@activepieces/pieces-framework --filter=web --filter=worker &
           pids+=($!)
 
           npx turbo run test-ce test-ee test-cloud check-migrations --filter=api &

brain/knowledge/ai-intelligence/ai-agents.md

@@ -28,7 +28,13 @@ A flow step type (the `run_agent` action of `@activepieces/piece-ai`) that runs
 - **The enums and pure functions have exactly one home: `core/piece-types/src/lib/agents.ts`.** Do not re-declare `AgentToolType`, `McpAuthType`, `buildAuthHeaders`, `TASK_COMPLETION_TOOL_NAME`, or `mcpToolNameUtils` in `core-execution` — re-export them. They used to be duplicated byte-for-byte across both packages, which was silently load-bearing: if `createToolName` drifted, the tool names `migrate-v16` persisted would stop matching runtime names and every piece/flow/MCP call on a migrated flow would degrade to `ToolCallType.UNKNOWN`. `mcp-tool-name-util.test.ts` asserts both entry points resolve to the *same object*, so a re-fork fails the test rather than shipping.
 - The four `core/execution/src/lib/agents/` files are **not** uniform. `mcp-tool-name-util.ts` and `mcp.ts` are pure re-export shims (1 and 6 lines). `index.ts` and `tools.ts` re-export the canonical enums and functions but still **own** the execution-side plain-`zod` schema definitions — `tools.ts` declares the `AgentTool` union and the `McpAuth*` schemas, `index.ts` declares `AgentOutputField`, `MarkdownContentBlock`, `ToolCallContentBlock` and `AgentStepBlock`. Adding a field to one of those schemas means editing it there *and* in the `zod/mini` twin in `agents.ts`.
 - **A flow-step run must not reuse chat's resolution logic.** Four separate production failures came from this one assumption while moving the step server-side, each looking like its own bug. `resolveChatProvider` made a step need Chat's provider configured before it would run at all, so an instance that never uses Chat could not run an agent step — and it bit twice, because `resolveFastModel` reached the same helper underneath, so every *configured piece tool* failed with a bare `ENTITY_NOT_FOUND` long after the main model had been fixed. Grep for the transitive callers, not just the direct ones. `resolveModelIdForProvider` treats its argument as a *tier* id and falls back to the tier default when it is not in the curated chat list — a step configured for `claude-sonnet-4.5` silently ran `4.6`, because a step names a concrete model while chat names a tier. And the chat tool set reaches an unattended run, where a tool that asks the user a question is worse than useless: the agent opened a connection picker, read the empty answer as a refusal, and stopped. When a value crosses between the two surfaces, check what it *means* on each side, not just that the types line up.
-- **A worker RPC failure reaches the worker as `error.message` and nothing else.** The envelope in `core/execution/src/lib/engine/rpc.ts` drops `ActivepiecesError.params` and the stack, so three unrelated causes (conversation gone, no chat-enabled provider, pinned provider has no row) all arrive as the same bare `ENTITY_NOT_FOUND` — unreadable in the failed-job list. `createRpcServer` logs the intact error on the app side; read *that* log, not the worker's.
+- **A worker RPC failure carries `{ code, entityType }` now, but still no stack.** The envelope in `core/execution/src/lib/engine/rpc.ts` used to serialize `error.message` alone, so three unrelated causes (conversation gone, no chat-enabled provider, pinned provider has no row) all arrived as the same bare `ENTITY_NOT_FOUND`. `apErrorOf` now also ships an `ActivepiecesError`'s code and entity type, which the client re-attaches to the thrown error — read it with `apErrorOf(error)`, never by parsing the message. Deliberately **not** the whole `params`: it is typed `unknown`, and socket.io JSON-encodes this ack from inside a `catch` where nothing handles a throw, so one cyclic or BigInt-bearing params object would send no ack at all and stall the caller for the full 60s RPC timeout (the engine side would `process.exit(4)` on the unhandled rejection). `rpc.test.ts` pins this with a cyclic params case and a JSON-round-tripping fake socket — keep the projection narrow.
+- **A failed agent run is a user's misconfiguration far more often than our bug, and only our bugs belong in the failed set.** `EXECUTE_AGENT_RUN` re-threw on everything except credit exhaustion, so ~5,900 unrecoverable user-config failures accumulated in the BullMQ failed set over one 30-day retention window (`REDIS_FAILED_JOB_RETENTION_DAYS`) and buried the real bugs. `classifyAgentRunError` (`run-agent-turn.ts`) splits them, and a user-class failure returns `EngineResponseStatus.USER_FAILURE`, which `job-broker.completeJob` completes exactly like `OK` while naming the outcome. Four things it gets deliberately right, each of which is a way to get it wrong:
+  - **The user-fault statuses are an allow-list (401/403/404), not `!APICallError.isRetryable`.** The SDK calls every 4xx non-retryable, so the tempting one-liner blames the user for a 400 from an illegal generated tool name or a 413 from a prompt still over the window after compaction — requests *we* built, and exactly the laundering the split exists to prevent.
+  - **The managed `activepieces` provider is never user-fault on auth.** It runs on our own OpenRouter key, so a 401 there fails every platform at once and must page.
+  - **Credit is read from a status or the specific `insufficient_quota` marker, never loose patterns over a response body.** OpenAI signals billing exhaustion as a *retryable* 429 with the marker in the **body**, so credit is checked before the retryable verdict — but scanning a body for `credits`/`402` made a provider 500 whose HTML error page said "credits" complete as a billing failure and hide a real outage.
+  - **`ENTITY_NOT_FOUND` counts only for an AI-provider `entityType`, and `VALIDATION` counts for nothing.** A bare not-found is our bug; the `VALIDATION` that reaches this surface is the conversation concurrency lock, and a conversation stuck `STREAMING` is a state worth keeping visible.
+  A completed job stores no `errorMessage`, so the `warn` log carrying `agentRun.errorClass` is the only remaining record.
 - **Whatever enqueues an agent run must pre-check the same thing the worker resolves.** The chat route asked "is any provider enabled for chat" while the worker looked up the run's *pinned* provider, and the flow-step route checked nothing at all — so a run enqueued fine and could only fail. Both now call `agentHelpers.assertRunProviderConfigured`, which mirrors the worker's lookup. A pre-check that answers a *different* question than the worker is worse than none: it makes the failure look impossible.
 - **Everything the agent job does before its try/catch has no recovery.** `getAgentConfig` used to run outside it, so a config failure sent no error to the chat client and never called `releaseFlowStep` — the flow run sat PAUSED until `AP_PAUSED_FLOW_TIMEOUT_DAYS`. Anything added above that block needs its own failure path, or a paused run leaks.
 - **Build the unattended tool set as an allow-list.** Removing chat tools by name failed three times running — display tools, then build-plan and phase tools, then `ap_discover_action_auth` and `ap_load_guide`, which live with the local tools and so survived a filter written by tool group. Grouping tracks where a tool was constructed, not whether it assumes someone is reading. A flow step gets exactly what it is listed: its configured piece actions, the public-web readers, and the structured-output tool. Anything added to chat later stays out by default.

brain/knowledge/connections-auth/ce-authentication.md

@@ -31,8 +31,9 @@ The core (all-editions) auth layer: user identity creation, sign-in, and JWT ses
 - **A nil `projectId` on the principal means "go to /create-platform" in four separate places.** Anything that mints a platform-less session has to satisfy all of them, not just the route guard.
 - **Platform naming reads the email domain first, and "is this a work address" is a denylist of consumer brands.** `ahmad@activepieces.com` yields `"Activepieces"` while `ahmad@gmail.com` yields `"Ahmad's Platform"`. Two details are easy to get wrong when touching `signup-names.ts`. The denylist is keyed on the **registrable label**, not the full domain, so `yahoo.co.uk` is caught by the single entry `yahoo`. And the label is picked as the second-to-last domain part, stepping back one more when the part before the TLD is itself a public suffix (`co`, `com`, `ac`, ...), so `mail.activepieces.com`, `activepieces.co.uk` and `eu.activepieces.co.uk` all resolve to `Activepieces` rather than to `Mail`, `Co` or `Eu`. It is a heuristic, not a public-suffix list: a company sitting on an unlisted two-part suffix gets the suffix as its name. Only new signups are affected; existing platforms keep their names.
 - **The route no longer decides sign-in vs sign-up — the card does.** `/sign-in`, `/sign-up` and `/create-platform` all render the same `AuthLanding`; `/sign-up` is a bare redirect to `/sign-in`. Which form you get is a function of two flags: with `SMTP_CONFIGURED` the card opens on the email-code step and the classic password form exists *only* behind the "Use password" link; without it you land on a password form directly, and `USER_CREATED` picks sign-up (first ever account, no mode switch offered) over sign-in. So the same URL renders three different DOMs across Cloud, a seeded self-host, and a fresh install — anything scripting this screen has to branch, and password sign-*up* is simply unreachable once SMTP is on.
+- **The sign-in URL's query string survives the email-code journey but not a federated one.** `/sign-up` forwards its search to `/sign-in`, and the card never navigates, so `?foo=bar` is still there at the end. Google/SAML instead do `window.location.href = …` and only `from`, `providerName` and `activepiecesLogin` ride along in the OAuth `state`; the customer returns on `/redirect` and goes to `from` or `/create-platform`. Anything that has to outlive sign-in for *every* provider belongs in `localStorage`, not in the URL.
+- **`from` gets you back to the route but not to its query string — `AuthenticatedDefaultRoute` used to drop it.** Both `DefaultRoute` and `AllowOnlyLoggedInUserOnlyGuard` build `from` as `location.pathname + location.search`, so a param on the original URL survives sign-in and `useRedirectAfterLogin` navigates back to it. The last hop was where it died: landing on `/` authenticated renders `AuthenticatedDefaultRoute`, which navigated to `determineDefaultRoute(...)` with no `search`, so anything hanging off `/?x=1` was gone before the project routes (and the guards mounted inside them) rendered. That `Navigate` now forwards a single allow-listed param (`TRIAL_KEY_QUERY_PARAM`, in `route-utils.ts` beside `NEW_FLOW_QUERY_PARAM`), which is what lets a trial activation link reach the signed-in screen that consumes it. It deliberately does **not** forward the whole search string: `AuthenticatedDefaultRoute` also serves the `/*` catch-all, so blanket forwarding would push the query string of every unmatched URL into the default route for whatever page later sits there to read. A param that must survive that hop has to be added to the allow-list.
 - **`/create-platform` is that same card opening on its name step**, off the ONBOARDING token rather than a route param — submitting the name is what mints the platform and project and swaps ONBOARDING for USER. The field is the *person's* `Full Name` (`data-testid="auth-full-name"`), not a workspace name. **Only the emailed-code path reaches it**: password sign-up and Google already collected a name, so those sessions are provisioned in the same request and land in the product with one form submission.
-
 ### Key files
 Entry point: `authenticationService`, a log-taking factory called per request from `authentication.controller.ts`, registered as `authenticationModule` in `app.ts`.
 

brain/knowledge/engineering/architecture-spine.md

@@ -33,6 +33,9 @@ Activepieces: open-source AI-first workflow automation platform (self-hosted or
 
 ## Gotchas
 
+**`has no exported member` after merging `main` is a stale `dist/`, not broken code.** The app typechecks resolve `@activepieces/core-*` through each package's built `.d.ts`, not its source, so a symbol `main` added to a thin core package is invisible to `packages/web` and `packages/server` until that package is rebuilt. It reads exactly like a bad merge — `tsc` names a real export that is right there in the source. Confirm by grepping the symbol in `packages/core/<name>/dist/`, then `npx turbo run build --filter=@activepieces/core-<name>`. Hit 2026-08 merging `main` into a feature branch: `AI_PROVIDER_ENTITY_TYPES` (added by #15097) was in `core/piece-types/src` and re-exported from its index, but absent from `dist/`, so web's typecheck failed on `core/shared` importing it.
+
+
 **`distributedLock().runExclusive` waits for the *whole* `timeoutInSeconds` under contention — never put one on a request path.** `distributed-lock-factory.ts` configures Redlock with `retryCount = Math.ceil(timeout / 200)` and `retryDelay: 200`, so the retry budget is exactly the lock TTL: a `timeoutInSeconds: 15` lock retries 75 times before giving up, and each retry is its own Redis round-trip. N concurrent requests contending on one key therefore generate up to N×75 pure-retry commands against shared Redis *while* every one of them stalls for up to 15s. Read-mostly checks belong on the cache with the fetch scheduled behind the response (`rejectedPromiseHandler` + `distributedStore.runOnceWithin` gives cluster-wide dedupe without a lock); reserve `runExclusive` for genuine write serialization off the hot path. Surfaced 2026-08 in the Autumn credits gate (PR #14436, `f0638438`), where an exhausted or cold platform made every webhook, AI-proxy call and chat turn take a reverify lock plus a `platform_plan` SELECT plus a 5s Autumn HTTP call inline — a ~20s worst case on the highest-volume path in the product. Related: [[ee-platform-plans-billing]].
 
 **Don't `.max()` a business limit on a request body — cap server-side.** A `.max()` on a request-body field rejects the *whole* request with a 400 the moment a user crosses it, so a user editing a list that reaches 50 items loses their entire save. Reserve `.max()` for a true trust-boundary DoS guard (Fastify's global body limit already covers gross abuse) and let business limits just *apply*: accept the input and `slice(0, MAX)` in the service layer, so the write always succeeds with the limit quietly enforced. Surfaced 2026-07 on `POST /v1/chat/memory`, where the schema's `.max(50)`/`.max(280)` duplicated a `slice` the save helper already did — redundant *and* a data-loss bug.

brain/knowledge/engineering/web-feature-anatomy.md

@@ -74,3 +74,5 @@ Verify with `npx turbo run lint --filter=web`, or `npm run lint-dev` for the who
 - **`npx turbo run serve --filter=web -- --mode=cloud` cannot do OAuth2 connections.** The provider redirects to `cloud.activepieces.com` after sign-in instead of your local frontend. Use API-key or basic-auth connections, or run a fully local backend.
 - **`--mode=cloud` also floods the terminal with `[vite] http proxy error: /ingest/... ETIMEDOUT 127.0.0.1:3000`.** The mode only redirects the API (`API_BASE_URL` → `https://cloud.activepieces.com` in `lib/api.ts`); PostHog still posts to the *relative* `api_host: '/ingest'` (a same-origin reverse proxy so ad blockers don't drop ingestion — `providers/telemetry-provider.tsx`, mirrored in prod by the `fastifyHttpProxy` in `server.ts`). Vite proxies `/ingest` to `127.0.0.1:3000`, which isn't running. Cloud flags also turn telemetry *on* (`TELEMETRY_ENABLED` + `EDITION=cloud`), unlike a local CE backend — so posthog-js keeps polling `/ingest/flags` and flushing `/ingest/e` every few seconds. Harmless, but note the same setup sends real dev clicks to production PostHog whenever `/ingest` does resolve; the clean fix is skipping `posthog.init` under `import.meta.env.DEV`.
 - **`packages/web`'s lint script only globs `src/**`, so nothing under `packages/web/test/` is ever linted** — not by CI's `lint` job, not by `npm run lint-dev`. Running `npx eslint 'test/**/*.{ts,tsx}'` from `packages/web` today reports 21 errors nobody has seen, so a new web test needs a manual eslint pass or it ships with errors. Most common trap: `testing-library/render-result-naming-convention` fires on any local helper whose name merely *starts with* `render` even when testing-library is not involved — renaming `render` to `renderTabText` does not silence it, only a name that doesn't begin with `render` does.
+- **`AllowOnlyLoggedInUserOnlyGuard` calls its hooks after two early returns, and the linter only lets it.** `react-hooks/rules-of-hooks` does not flag member-expression calls, so `platformHooks.useCurrentPlatform()` / `flagsHooks.useFlags()` sail past it — but add a bare `useSomething()` there and the rule fires, correctly: `isLoggedIn()` can change between renders, so those calls really are conditional. Anything new that needs to run once a session is authenticated belongs in a null-rendering component placed inside the returned `<SocketProvider>` subtree, which mounts only after the guard passes. That is why automatic trial activation is `<AutomaticTrialActivation />` and not a hook.
+- **The layering is lint-enforced, not just a convention.** `packages/web/.eslintrc.json` has an `import/no-restricted-paths` zone making the codebase unidirectional: `src/app` may import `src/features`, and both may import `src/lib`/`hooks`/`components`/`types`/`utils` — never the reverse (the one exception is `app/query-client.ts`). So a hook that a public route needs belongs in `src/lib`, but anything rendering a feature's components has to live in that feature; you cannot keep the pair in one `lib` file. It fails as an `import/no-restricted-paths` **error**, not a warning, so it blocks lint.