[pull] main from activepieces:main

Dockerfile

@@ -113,7 +113,8 @@ RUN node -e "\
 # Remove workspaces not needed at runtime: pieces except the 5 the api imports,
 # plus web/cli/tests-e2e/embed-sdk whose deps (react & friends) would otherwise land
 # in the runtime node_modules. dist/packages/web is already built and kept.
-# Then drop the removed entries from the root workspaces list and regenerate bun.lock.
+# Then drop the removed entries from the root workspaces list, drop the engine's test-only
+# core-piece devDependencies (their workspaces are gone), and regenerate bun.lock.
 RUN rm -rf packages/pieces/core packages/pieces/custom \
       packages/web packages/cli packages/tests-e2e packages/ee && \
     find packages/pieces/community -mindepth 1 -maxdepth 1 -type d \
@@ -124,6 +125,7 @@ RUN rm -rf packages/pieces/core packages/pieces/custom \
       ! -name microsoft-teams-bot \
       -exec rm -rf {} + && \
     node -e "const fs=require('fs');const p=JSON.parse(fs.readFileSync('package.json','utf8'));p.workspaces=p.workspaces.filter(w=>fs.existsSync(w.replace('/*','')));fs.writeFileSync('package.json',JSON.stringify(p,null,2))" && \
+    node -e "const fs=require('fs');const f='packages/server/engine/package.json';const p=JSON.parse(fs.readFileSync(f,'utf8'));p.devDependencies=Object.fromEntries(Object.entries(p.devDependencies).filter(([n])=>!n.startsWith('@activepieces/piece-')));fs.writeFileSync(f,JSON.stringify(p,null,2))" && \
     rm -f bun.lock && bun install
 
 ### STAGE 2: Run ###

brain/knowledge/ai-intelligence/ai-agents.md

@@ -46,7 +46,7 @@ Three things follow. `run_agent` has two modes: pointed at a saved agent (`agent
   - **The user-fault statuses are an allow-list (401/403/404), not `!APICallError.isRetryable`.** The SDK calls every 4xx non-retryable, so the tempting one-liner blames the user for a 400 from an illegal generated tool name or a 413 from a prompt still over the window after compaction — requests *we* built, and exactly the laundering the split exists to prevent.
   - **The managed `activepieces` provider is never user-fault on auth.** It runs on our own OpenRouter key, so a 401 there fails every platform at once and must page.
   - **Credit is read from a status or the specific `insufficient_quota` marker, never loose patterns over a response body.** OpenAI signals billing exhaustion as a *retryable* 429 with the marker in the **body**, so credit is checked before the retryable verdict — but scanning a body for `credits`/`402` made a provider 500 whose HTML error page said "credits" complete as a billing failure and hide a real outage.
-  - **`ENTITY_NOT_FOUND` counts only for an AI-provider `entityType`, and `VALIDATION` counts for nothing.** A bare not-found is our bug; the `VALIDATION` that reaches this surface is the conversation concurrency lock, and a conversation stuck `STREAMING` is a state worth keeping visible.
+  - **`ENTITY_NOT_FOUND` counts only for an AI-provider `entityType`, and `VALIDATION` counts for nothing.** A bare not-found is our bug; the `VALIDATION` that reaches this surface is the conversation concurrency lock, and a conversation stuck `STREAMING` is a state worth keeping visible. So a user-config refusal thrown from `getAgentConfig` must be `ENTITY_NOT_FOUND`/`AIProvider` with its text as `ActivepiecesError`'s second argument: text left in `params` never crosses the RPC, which is how a step naming a model the Activepieces credits do not serve failed as a bare internal `VALIDATION` on every scheduled run. The billing text heuristic reads provider text only and skips any error carrying our own code, because `isProviderBillingError` matched our own "Activepieces AI credits" wording; our credit exhaustion is always `QUOTA_EXCEEDED`. Those bad models came from `ap_list_ai_models`, which offered the AI builder the whole OpenRouter catalog for Run Agent steps; it now lists only what the resolver accepts, while `listModels` keeps the full catalog because Ask AI and image lookups need it.
   A completed job stores no `errorMessage`, so the `warn` log carrying `agentRun.errorClass` is the only remaining record.
 - **A tool name is user text on four paths, and only the worker sees all of them.** `createToolName` is applied by the flow-tool dialog and the piece-tool stores, but a knowledge-base name was stored as typed and the AI piece's `toolName` is free `ShortText`. That string becomes the AI-SDK `ToolSet` key verbatim, which is how a name earned a 400 from Anthropic — *our* request, never the user's fault, which is why widening the status allow-list to 400 would have been the wrong fix. `mcpToolNameUtils.toValidToolName` is the guard, applied by `agentToolPolicy.withValidNames` in `execute-agent-run` — the one place the flow-step, chat and eval enqueue paths converge (`agent-conversation-controller` validates tool names not at all; `agent-run-controller` checks only the reserved prefix and duplicates, and does it on the *raw* names, so it cannot see a collision the rewrite creates). Four things it has to get right:
   - **The pattern is the intersection of every provider we ship, not the one from the error we happened to see.** Anthropic's `^[a-zA-Z0-9_.-]{1,64}$` is the loosest: OpenAI and Bedrock reject `.`, and Gemini requires a leading letter or underscore. Guarding with Anthropic's rule leaves `handbook.pdf` — the obvious name for a knowledge base file — still failing everywhere else. The guard is `^[a-zA-Z_][a-zA-Z0-9_-]{0,63}$`.

brain/knowledge/ai-intelligence/ai-providers.md

@@ -191,6 +191,9 @@ the URL. The file is published from the console in `ap-analytics`; see decision
 - **The AI Center providers page renders its whole state machine off one query, so every branch needs an `isLoading` gate above it.** `ProvidersTab` derives the empty state, the grouped list *and* the `?config=<id>` detail view from `useAiProviderConfigs()`. While that query is pending, `data` is `undefined` — `configs` is `[]`, so the empty "Connect your first provider" screen paints for a frame, and a hard refresh on a `?config=` URL finds no matching row and falls through to the list before flipping to the detail page. The parent `SuspenseWrapper` does not cover it: the route chunk and the suspense-backed projects collection resolve first, then the component mounts with no providers yet. Any new branch added to this page belongs below the `if (isLoading) return <ProvidersSkeleton />` gate, not beside it.
 - **A save bar in the AI Center needs a full-height column, not `fixed` and not bare `sticky`.** The page sits inside *two* nested scrollers — `PlatformLayout`'s `flex flex-col flex-1 overflow-auto` and the AI Center's own `flex-1 overflow-auto` column — so `fixed inset-x-0` centres the bar on the viewport, roughly a sidebar-width left of the content column it belongs to. Swapping it to `sticky bottom-4` fixes the centring for free but pins nothing when the content is shorter than the viewport: the bar just lands wherever the content ends, mid-page. The working shape is `min-h-full` + `flex flex-col` on the tab-content wrapper, `flex-1` on the `TabsContent`, `grow` on the panel root, then `sticky bottom-4 mt-auto` on the bar — `mt-auto` handles the short page, `sticky` handles the long one. `min-h-full` resolves only because every ancestor up to `SidebarInset h-full` has a definite height; that chain is what makes the whole thing work, so check it before reusing this elsewhere.
 - **A chat *tier* is our own idea, so it has no meaning on a provider we ship no curated model list for — resolve it by asking the key what it offers.** `ACTIVEPIECES_CHAT_TIERS` (fast/smart/premium) only maps to a real id through `ALLOWED_CHAT_MODELS_BY_PROVIDER`, which covers Anthropic, OpenAI, Google, Vertex and Activepieces. Everything else — Bedrock, Azure, Mistral, xAI, DeepSeek and the other OpenAI-compatible vendors — has an empty candidate list. The resolver used to invent an id from the tier (strip the vendor prefix, dots to dashes: `anthropic/claude-haiku-4.5` -> `claude-haiku-4-5`), which Bedrock rejected as "The provided model identifier is invalid" (GIT-1840, Pylon 5836). #15355 removed the invention and left a throw in its place, which was worse: every agent turn on those providers died with `ENTITY_NOT_FOUND: this AI provider key allows no text model a chat turn can run on` before the configured model handled anything (Pylon 6069). **The missing ingredient was never a smarter guess, it was the key's own model list.** `agentHelpers.resolveModelId` is now the single resolver for every surface: it tries our catalogue first and, only when that has nothing to say, reads `aiProviderService.listModels` (process-cached, cleared nightly by cron, already filtered by the key's `modelScope`), preferring a model whose id *contains* the tier's `nativeModelId` — Bedrock spells them `eu.anthropic.claude-sonnet-4-6-...`, so a tier still selects the model it names wherever the vendor hosts it, and otherwise degrades to the first text model the way `pickAllowedModel` already does for an admin-listed catalogue. Picking from a list the provider returned is the opposite of the fabrication #15355 removed. **Two traps, both learned the hard way.** First, an agent surface and generic chat are not symmetric: an agent *carries* a real model id (its picker lists `listModels` output unfiltered, precisely because these providers have no `ALLOWED_CHAT_MODELS_BY_PROVIDER` entry) while chat carries only a tier name, because `ChatModelSelector` falls back to tier options when `getCuratedChatModels` returns `undefined`. Second, **being listed proves the key can see a model, not that the account can invoke it** — so where a model is already settled the fast round reuses it rather than choosing again (`getAgentConfig` passes the model the turn resolved; the tool RPC gets the run's own model from the worker, `modelId` on `executePieceTool`). One more trap, found in review on 2026-09-27: a stored name is not the model the run ran on. The worker stamps the resolved *tier id* on an agentless inline run's conversation, so reading it back as a model hands a bare `smart` to a catalogue-less own-key provider, and no rule on the string can save it, because a deployment may be named `smart` too. So nothing is read off the conversation or the agent for the fast round any more: the worker says which model the turn resolved, `modelId` on `ExecutePieceToolRequest` (core-execution), and the API borrows exactly that. The field is required, not optional, because the dispatch gate keeps worker and app on one release, so an older worker never reaches a newer API. Resolving the fast tier independently once picked a Haiku the account could not invoke and failed a turn whose configured Sonnet worked fine. A borrowed model still has to pass the key's current `modelScope`, or resolution falls through to the list, which applies it. Nothing crosses the worker, so a rolling deploy has no version-skew window.
+- **Non-chat models are filtered out centrally in `fetchModels`, not per provider strategy.** OpenAI's `/v1/models` returns `whisper-1`, `tts-1`, `gpt-4o-transcribe`, `text-embedding-3-*`, `omni-moderation-latest` and `sora-*`, and every strategy that cannot tell them apart typed them `TEXT` — so they were offered in Ask AI / Summarize / Classify / Extract and died at run time on chat completion (#14496). One `aiProviderUtils.isChatModelId` denylist now runs inside `fetchModels`, which is the single choke point behind both the project route and the admin `configs/:id/models` route, so a new provider is covered without opting in. Three things about it are load-bearing: it tests **only `TEXT`-typed rows**, so image models are untouched; it **never judges a name the customer chose** — admin-typed configs (`'models' in config` — VERTEX, CUSTOM, CLOUDFLARE_GATEWAY) skip it because an admin who typed an id meant it, Azure skips it (`modelIdsAreCustomerNamed` on the strategy) because it lists deployment names like `support-voice-agent` and filters on `deployment.model` itself, and a fine-tune is judged by its base model only (`ft:<base>:<org>:<label>:…`, Azure `<base>.ft-…`) so an org called `voiceflow` or a `content-moderation` label does not hide a working chat model; and `validateConnection` **bypasses `fetchModels` entirely**, so a key that lists nothing but whisper still validates as a working key. The rule is a denylist on purpose — an unknown id is treated as chat, so a self-hoster's own model on a custom endpoint never disappears, at the cost of a new modality leaking until a pattern is added. Because pieces may not import `@activepieces/core-*` (eslint `no-restricted-imports`), the openai piece's `isLLM` reaches the same ruleset through a `packages/pieces/framework/src/index.ts` re-export. Still leaking, knowingly: vendor-branded generative models on the five compat vendors that publish no modality data (`cogview-*`, `wan*`, `music-*`, `video-01`), whose real ids nobody could verify without five paid keys — xAI is handled structurally, see the next bullet.
+- **xAI is the one compat vendor asked what its models output, because it is the one that answers.** `listVendorModels` hits `/v1/language-models` for XAI first and keeps a model only when `output_modalities` includes `text`; the OpenAI-shaped `/v1/models` that the other five vendors use carries no modality field at all, which is why `grok-2-image-1212` and the Grok Imagine image/video models were offered as chat. A model that declares no modalities falls back to the id denylist. Every failure mode degrades to the old path rather than emptying the picker: a 404, a changed shape, or an empty `models` array all fall through to `/v1/models`, and because that fallback is what authenticates, a bad key still fails validation with the vendor-prefixed error. The endpoint pair (`/language-models`, `/image-generation-models`) was already known in-repo — `packages/pieces/community/grok-xai` calls both. Adding a vendor to `VENDORS_DECLARING_MODALITIES` is the whole opt-in; do it only with a real `/models` response in hand, not from recollection of a catalogue.
+- **Google is filtered on `supportedGenerationMethods`, and that is what keeps Imagen out of the image picker.** A model survives only if the field is absent or includes `generateContent`, which is the only way to exclude `aqa` (`generateAnswer`) and `gemini-embedding-001` (`embedContent`) — no name pattern catches them. The side effect is deliberate: `imagen-*` was classified `IMAGE` (because `'imagen'.includes('image')`) and offered in Generate Image, but the AI piece reaches Google image generation through `generateText`/`:generateContent` (there is no `google` case in `buildNativeImageModel`), and Imagen only answers `:predict` — so those options always failed at run time. Gemini image models do expose `generateContent` and are unaffected; Vertex is where Imagen actually works, and it is admin-typed so the filter never sees it.
 - `packages/server/api/src/app/ai/` — backend module: provider controller, service, entity, module, plus the sibling ai-tool-config files
 - `packages/server/api/src/app/ai/providers/` — per-vendor strategies keyed by `AIProviderName`
 - `packages/server/api/src/app/platform/billing-provider.ts` — `assertCreditsAndAppSumoNotExceeded` credit gate