Skip to content

deployment-cloudflare.md

Latest commit

 

History

History
504 lines (399 loc) · 32.2 KB

deployment-cloudflare.md

File metadata and controls

504 lines (399 loc) · 32.2 KB
Raw

Cloudflare Deployment

Supported Runtime

The supported Cloudflare target is a hybrid Worker plus Container deployment. Most routes need Flask, normal Python packages, SQLite file access, and Gunicorn, so they run in the Container. Roleplay under /v1/roleplay, native LinkAPI traffic under /linkapi/*, and Codex Everywhere OpenAI traffic under /codex-easy/* take direct Worker paths. Kimi model discovery is Worker-local; valid Kimi chat is authenticated at the edge and streamed through the Container because Kimi's edge rejects Worker-origin egress.

wrangler.jsonc deploys:

  • Worker entry: cloudflare-worker.mjs
  • Container class: MultiLLMProxyContainer
  • Container image: Dockerfile
  • Durable Object binding: MULTILLM_PROXY_CONTAINER
  • Container instance count: max_instances=1
  • Roleplay Durable Object: RoleplaySession, one instance per session
  • Observability: enabled

OpenCode routes are authenticated at the Worker edge and then forwarded through the Container. OpenCode's edge rejects Worker-origin HTTP signatures with Cloudflare error 1010, while the Container's normal HTTP client is accepted. Roleplay sessions use the same Container egress path whenever an OpenCode model is selected; other configured roleplay providers remain direct Worker fetches.

Cloudflare-native roleplay

POST /v1/roleplay and its OpenAI-compatible /roleplay/v1/chat/completions alias keep authentication, validation, session state, memory, and adaptive selection in the Worker. The Worker authenticates the caller with ROLEPLAY_API_KEY or the bootstrap ADMIN_API_KEY, validates a bounded JSON request, and sends it to the session's ROLEPLAY_SESSION Durable Object. An OpenCode generation wakes the Container only for accepted upstream egress; other providers remain direct Worker fetches. Each session stores recent dialogue, a compact continuity digest, idempotency keys, and per-model EWMA latency/reliability statistics.

Production GLM routing starts generic requests on NanoGPT z-ai/glm-5.3-flash, followed by uncensored Flash and GLM-5.2 inside the 1x subscription pool. Full GLM-5.3 is excluded from automatic subscription routing and remains available through the explicit roleplay:5.3 selector. roleplay:glm-speed ranks the 1x pool by observed streaming TPS after bounded exploration. OpenCode and NavyAI remain later provider tiers. LinkAPI is Kimi-only and OpenRouter is omitted from the roleplay chain. Explicit 400, 401, 402, 403, 404, 413, 415, 422, 429, and 503 responses trigger automatic fallback. Transport errors and other 5xx responses stop because their generation outcome is ambiguous.

At the local compaction threshold, the selected model decides whether older dialogue should become a bounded continuity digest. At the hard input limit, compaction is required; failure stops before final generation. This can create one additional billed request, but never silently drops history.

JanitorAI-origin roleplay streams ignore the client's numeric output ceiling. When a provider reports finish_reason: length, the Worker joins as many as ROLEPLAY_MAX_AUTO_CONTINUATIONS continuation legs (eight by default) into one SSE response with a single terminal [DONE]. A provider finish_reason: stop uses the separate ROLEPLAY_MAX_OUTPUT_CONTRACT_REPAIRS budget (one by default) when a caller-mandated response lacks visible story content or its final IMAGE PROMPT: block lacks canonical required fields. An image-only leg is discarded and regenerated from the beginning; current and legacy labels are aliases. No-progress repair text is suppressed, multiple image blocks are collapsed to one, and only the cleaned answer is stored.

Use GET /v1/roleplay/models for configured tiers and GET /v1/roleplay/metrics?session_id=... for bounded per-session routing metrics. Neither route returns stored dialogue. See the roleplay guide.

The Worker routes Container-backed app traffic to the named primary container. Keeping one container instance avoids stale auth/model state from multiple Python worker processes until durable storage is introduced.

Codex Everywhere Worker Fast Path

Configure the preferred upstream credential as a Worker secret. The existing CODEX_API_KEY secret name is supported as a fallback alias.

npx wrangler secret put CODEX_EASY_API_KEY

The upstream origin is fixed to https://codex-easy.ai. If a client appends /v1, configure $PROXY_BASE_URL/codex-easy; if it expects /v1 in the configured base, use $PROXY_BASE_URL/codex-easy/v1.

Operation Direct Worker route Caller credential
Key-group model catalog /codex-easy/v1/models Authorization: Bearer $ADMIN_API_KEY
OpenAI Responses /codex-easy/v1/responses Authorization: Bearer $ADMIN_API_KEY
Chat Completions /codex-easy/v1/chat/completions Authorization: Bearer $ADMIN_API_KEY
Images /codex-easy/v1/images/* Authorization: Bearer $ADMIN_API_KEY

The fast path accepts only the bootstrap ADMIN_API_KEY and replaces it with CODEX_EASY_API_KEY or its alias before contacting Codex Everywhere. It bypasses Flask dashboard-user authentication, application-level request-size checks, RPM/TPM/daily limits, Flask request/rate-limit accounting, and request metrics. Use the Container-backed /v1/responses or /v1/chat/completions route with a codex-easy:<model> model ID when those controls are required.

Catalog results are specific to the API-key group, so discover current model IDs through /codex-easy/v1/models. Image routes work only for image-generation key groups. The Worker preserves raw JSON, SSE, binary, and multipart bodies, except that a JSON GPT Image generation missing moderation is normalized once to insert "low". On the Codex Everywhere and LinkAPI raw OpenAI fast paths, it retains a Responses prompt_cache_key and forwards Chat's X-Grok-Conv-Id. For Grok requests, xAI recommends these fields for cache routing, but neither the proxy nor either provider guarantees a cache hit. Generation POSTs are single-attempt; the proxy does not provide idempotency.

Kimi Code protected routes

Configure the upstream credential as a Worker secret:

npx wrangler secret put KIMI_CODE_API_KEY

The upstream base is fixed to https://api.kimi.com/coding/v1.

Operation Proxy route Execution path Caller credential
Model catalog /kimi-code/v1/models Worker-local configured catalog Authorization: Bearer $ADMIN_API_KEY
Chat Completions /kimi-code/v1/chat/completions Edge auth, then Container raw pass-through Authorization: Bearer $ADMIN_API_KEY

Kimi Code generation is Chat Completions only in this integration. Use k3 on the raw route, reasoning_effort: "max" for K3's strongest reasoning setting, and a stable prompt_cache_key when the conversation prefix is stable. Cache affinity is upstream behavior and a hit is not guaranteed.

The Worker validates ADMIN_API_KEY before waking the Container. It serves model discovery locally; for valid chat, Flask removes the caller credential and sends KIMI_CODE_API_KEY upstream. The raw route bypasses unified request-size checks, RPM/TPM/daily limits, and accounting. Use /v1/chat/completions with model kimi-code:k3 when those controls are needed. Requests and streams are preserved, and generation is single-attempt to avoid duplicate work and billing.

LinkAPI Worker Fast Path

Configure the upstream credential as a Worker secret:

npx wrangler secret put LINKAPI_KEY

LINKAPI_BASE_URL is an optional, non-secret Worker variable. It defaults to LinkAPI's HK direct origin, https://hk.linkapi.ai, for both Worker and Container requests. It is restricted to the Worker's allowlist of official LinkAPI hosts; arbitrary HTTPS origins are rejected. LINKAPI_API_KEY remains a compatibility alias, but LINKAPI_KEY is the preferred name.

Native routes keep their upstream protocol shape:

Protocol Worker route Caller credential
Claude Messages /linkapi/v1/messages x-api-key: $ADMIN_API_KEY
OpenAI Responses /linkapi/v1/responses Authorization: Bearer $ADMIN_API_KEY
OpenAI compatible /linkapi/v1/chat/completions Authorization: Bearer $ADMIN_API_KEY
OpenAI model catalog /linkapi/v1/models Authorization: Bearer $ADMIN_API_KEY
OpenAI image generation /linkapi/v1/images/generations Authorization: Bearer $ADMIN_API_KEY
OpenAI image editing /linkapi/v1/images/edits Authorization: Bearer $ADMIN_API_KEY
Gemini native /linkapi/v1beta/models/{model}:generateContent Prefer x-goog-api-key: $ADMIN_API_KEY; ?key=$ADMIN_API_KEY is compatibility-only

The fast path accepts only the bootstrap ADMIN_API_KEY and replaces the caller credential with LINKAPI_KEY before contacting LinkAPI. It intentionally bypasses Flask dashboard-user authentication, application-level request-size checks, RPM/TPM/daily limits, Flask request/rate-limit accounting, and request metrics. When those controls are required, use the Container-backed /v1/chat/completions endpoint with a linkapi:<model> model ID, or /v1/images/generations with an image-capable linkapi:<model> model ID.

Gemini clients should prefer x-goog-api-key. Query-string ?key= remains available for compatibility, but it places the caller key in the URL, where clients and intermediaries may retain it, even though automatic Worker invocation logs are disabled.

The Worker streams request and response bodies without translating native SSE frames. A JSON GPT Image generation missing moderation is normalized once to insert "low"; explicit auto and low values remain unchanged. Its raw OpenAI routes preserve Responses prompt_cache_key and Chat X-Grok-Conv-Id; for Grok, xAI recommends those shapes for cache routing, but neither this proxy nor LinkAPI guarantees a cache hit. The proxy never retries generation POSTs and does not provide idempotency, avoiding accidental duplicate generations and billing. A caller should retry only when the selected upstream protocol and endpoint explicitly document an idempotency guarantee, using its own retry policy.

LinkAPI's live pricing page lists gpt-image-2-c for /linkapi/v1/images/generations and /linkapi/v1/images/edits. Use native Gemini generateContent for Gemini Flash Image models such as gemini-2.5-flash-image. See the LinkAPI image guide for complete examples.

AIHubMix Container Routes

Configure the upstream credential as a Worker secret:

npx wrangler secret put AIHUBMIX_API_KEY

wrangler.jsonc sets AIHUBMIX_BASE_URL=https://aihubmix.com and AIHUBMIX_BACKUP_BASE_URL=https://api.inferera.com. Both values are validated against the fixed official-host allowlist before use. The Worker forwards /aihubmix/* into Flask rather than contacting AIHubMix directly, preserving the native multipart body and the unified image translators.

Use /aihubmix/v1/models for the live provider catalog. Use /v1/images/generations with aihubmix:gpt-image-2-free, aihubmix:gemini-3.1-flash-image-preview-free, or aihubmix:doubao-seedream-4-0 for the normalized OpenAI Images interface. See the AIHubMix guide for native routes and request examples.

Adaptive and Explicit Context Optimization

Normal Container-backed GLM-5.2 requests on /v1/chat/completions safely preprocess context above GLM_AUTO_OPTIMIZE_TRIGGER_TOKENS (8,000 by default). Only old high-confidence image-prompt blocks are eligible; surrounding assistant story text, the newest full prompt, protected directives, recent turns, media, tools, and reasoning data remain intact. This automatic path never summarizes ordinary text or makes an extra provider request. /v1/responses and provider-specific routes remain unchanged.

The process-local prompt-analysis cache stores SHA-256 keys and prompt-span offsets only, never conversation text or generated replies. Configure it with CONTEXT_ANALYSIS_CACHE_ENABLED, CONTEXT_ANALYSIS_CACHE_TTL_SECONDS (300), and CONTEXT_ANALYSIS_CACHE_MAX_ENTRIES (2,048). Each Container has an independent cache.

POST /optimize/v1/chat/completions remains the explicit route for any unified provider:model and uses the normal Flask authentication, request-size, rate-limit, accounting, and metrics path.

The default deterministic mode makes no additional provider call. Above optimization.trigger_input_tokens, it can compact high-confidence older detailed image prompts while retaining the newest detailed image prompt, recent turns, system/developer instructions, multimodal exchanges, tool chains, and reasoning/thinking data. target_input_tokens defaults to 75% of the target provider's configured prompt limit. Reported token counts are provider-neutral byte-based estimates, not exact tokenizer usage or billing values.

summarize mode requires an explicit optimization.summary_model in provider:model form. When summarization is needed, it makes one additional billed and separately rate-limited summary call before the final request. Summary providers are restricted to codex-easy, kimi-code, or linkapi to keep transport single-attempt; calls use a bounded 45-second read timeout and two-slot per-process pool, and failed or capacity-denied summaries are not retried. Inspect X-MultiLLM-Optimization, X-MultiLLM-Optimization-Mode, X-MultiLLM-Estimated-Input-Before, X-MultiLLM-Estimated-Input-After, X-MultiLLM-Image-Prompts-Compacted, X-MultiLLM-Messages-Summarized, X-MultiLLM-Optimization-Target-Met, X-MultiLLM-Optimization-Cache-Hits, X-MultiLLM-Optimization-Cache-Misses, and X-MultiLLM-Summary headers for optimization metadata. Upstream JSON and SSE response bodies remain unchanged, and browser clients can read these headers through CORS.

The summary model must use the final model's provider by default. Eligible historical user/assistant plaintext is sent verbatim to that summary provider and can contain sensitive text. Prefer the same provider, remove secrets, and protect excluded history with optimization.preserve_message_indices. Cross-provider history transfer requires optimization.allow_cross_provider_summary: true; without that explicit disclosure opt-in the route returns 400. The bounded digest is inserted as an untrusted historical assistant message.

Set OPTIMIZER_MAX_REQUEST_BYTES to control the pre-parse ingress cap (default 16 MiB) and OPTIMIZER_SUMMARY_TIMEOUT_SECONDS for the summary read timeout (default 45 seconds, clamped to 5-120). The transformed final body still must satisfy the selected provider's body/prompt/output limits. The target model/key, output cap, and RPM/daily capacity are checked before a paid summary.

State and Container Instances

Container disk is ephemeral: it is reset whenever the Container sleeps (after 15 idle minutes) or a deploy replaces it. Durable state therefore lives outside it:

  • D1 (INTELLIGENCE_DB). Dashboard accounts and key hashes (with AUTH_STORAGE_BACKEND=d1, which wrangler.jsonc sets), automatic routes, request usage for rate limits and daily budgets, login throttling, disabled models, free-route cooldowns, workbench profiles and comparison results, the last good provider model catalog, and the intelligence gateway's policy, reservations and integration keys. See control-plane persistence for how each is cached and what happens during a D1 outage.
  • Durable Objects. Roleplay sessions (RoleplaySession) survive isolate and Container restarts, with a 30-day inactivity TTL by default.

The Worker still passes SQLite paths under /tmp (AUTH_DB_PATH, RATE_LIMIT_DB_PATH, MODEL_REGISTRY_DB_PATH). With D1 they hold only this Container's login-throttling record, which backs up the shared one, and nothing there needs to survive a restart. Keep ADMIN_API_KEY configured as the bootstrap admin key: it authenticates even while D1 is unavailable, and the direct LinkAPI and Codex Everywhere fast paths authenticate against it because they do not wake the Container.

Apply migration 0006_control_state.sql before deploying code that uses it (npm run deploy does). Until it exists, /ready reports the missing tables, the Container keeps that state in memory with a logged warning, and admin saves to it answer 503.

What stays per process

Each Container process keeps the following in memory only. It is rebuilt after a restart and is not shared between instances:

  • NanoGPT key-pool health (the active key and keys cooling down after a rejection) and the NanoGPT speed-suffix breaker.
  • Provider circuit breakers.
  • Caches: responses, prompt analysis, provider usage figures, and verified API keys (up to 60 seconds).
  • Dashboard metrics and the recent-request history.
  • Groq per-key token accounting and the image-relay catalog refresh schedule.
  • The in-memory side of the shared state: usage and cooldowns not yet sent to D1, and the cached copies described in control-plane persistence.

Running more than one Container instance

wrangler.jsonc keeps max_instances: 1, and the Worker sends every Container request to one named instance (getContainer(env.MULTILLM_PROXY_CONTAINER, "primary")). The shared D1 state makes more instances safe, but running them is a deployment decision that needs:

  1. Migration 0006 applied and AUTH_STORAGE_BACKEND=d1, so accounts and all control-plane state are in D1 rather than on one instance's disk.
  2. The same FLASK_SECRET_KEY and JWT_SECRET on every instance. They are Worker secrets, so this holds by default; dashboard sessions, CSRF tokens, video job IDs and the keyed identity hashes in D1 depend on them.
  3. A Worker change that spreads requests, for example getRandom(env.MULTILLM_PROXY_CONTAINER, N) from @cloudflare/containers, with max_instances raised to at least N. Raising max_instances alone changes nothing.
  4. Accepting per-instance behavior. Limits can briefly admit more than configured (by what other instances admitted within one RATE_LIMIT_SYNC_SECONDS interval); each instance learns provider failures and NanoGPT key health on its own; metrics cover one instance each; a disabled model reaches the other instances within 30 seconds, and an account change made on another instance within 60.
  5. Budgeting D1 load. Each busy instance makes one usage sync every RATE_LIMIT_SYNC_SECONDS (5 by default), a cooldown refresh every 10 seconds while the free pool is in use, and a model-override read every 30 seconds.

Required Secrets

npx wrangler secret put ADMIN_API_KEY
npx wrangler secret put FLASK_SECRET_KEY
npx wrangler secret put JWT_SECRET

Provider secrets are optional and should be set only when used:

npx wrangler secret put OPENAI_API_KEY
npx wrangler secret put ANTHROPIC_API_KEY
npx wrangler secret put ANTHROPIC_API_KEY_1
npx wrangler secret put OPENROUTER_API_KEY
npx wrangler secret put OPENCODE_GO_API_KEY
npx wrangler secret put CODEX_EASY_API_KEY
npx wrangler secret put KIMI_CODE_API_KEY
npx wrangler secret put LINKAPI_KEY
npx wrangler secret put NANOGPT_API_KEY
npx wrangler secret put NANOGPT_API_KEY_1
npx wrangler secret put NAVYAI_API_KEY
npx wrangler secret put AIHUBMIX_API_KEY
npx wrangler secret put GEMINI_API_KEY
npx wrangler secret put GROQ_API_KEY_1
npx wrangler secret put CHUTES_API_TOKEN

For Vertex-style Google auth:

npx wrangler secret put GOOGLE_APPLICATION_CREDENTIALS_JSON
npx wrangler secret put PROJECT_ID
npx wrangler secret put LOCATION
npx wrangler secret put GOOGLE_ENDPOINT

Optional dashboard single sign-on through Cloudflare Access needs a Worker secret and a few variables; see Dashboard single sign-on:

npx wrangler secret put CF_ACCESS_PROOF_SECRET   # at least 32 random characters
# Variables: CF_ACCESS_TEAM_DOMAIN, CF_ACCESS_AUD, CF_ACCESS_ALLOWED_EMAILS,
# optionally DASHBOARD_SSO_ONLY=true and ADMIN_USERNAMES.

The Worker verifies the Access token and forwards only a signed identity assertion to the Container; CF_ACCESS_AUD stays on the Worker. Migration 0010 adds the control_audit_events table behind the administrator audit log at /admin/audit.

Do not place API keys in wrangler.jsonc vars. Use Worker secrets. NanoGPT's non-secret deployment default is subscription-only text routing via NANOGPT_BILLING_MODE=subscription; wrangler.jsonc supplies the corresponding subscription base URL to both the Worker roleplay path and the Container.

Deploy Steps

Local checks before a change reaches main:

npm ci
python scripts/validate_sqlite_schema.py
python scripts/check_static_secrets.py
python -m pytest -q
npm run test:worker
npm run test:integration
npm run cf:migrations:check
node scripts/key_data_preflight.mjs --check
node scripts/check_worker_bundle.mjs --config wrangler.jsonc --outdir /tmp/main-worker

Pushing to main deploys; see How deploys work. Never deploy with a bare wrangler deploy: its preflight verifies prerequisites but does not apply migrations.

How deploys work

A push to main runs the ci workflow. When it passes, the deploy workflow (.github/workflows/deploy.yml) deploys the commit that ci tested:

  1. Select the commit. The gate job skips the commit if main has moved on, because the newer commit's run deploys both; an older commit never overwrites a newer one. Deploy runs never overlap (concurrency: deploy-production), and a newer queued run replaces an older queued one. Run workflow on the deploy workflow redeploys the head of main once ci has passed for it.
  2. Apply migrations. scripts/verify_d1_migrations.mjs --check checks the files, the job lists the pending ones, wrangler d1 migrations apply multillm-intelligence --remote applies them, and the remote check fails unless every file is recorded as applied. Any failure stops the run before new code ships.
  3. Deploy the Knowledge Worker, then the main Worker. The Knowledge Worker goes first because the main Worker binds it (KNOWLEDGE_SERVICE) and may call what the new version adds. The main Worker and its Container go last, and only when the repository variable DEPLOY_MAIN_WORKER is true (see Workers Builds). Before upload, the main build verifies the migrated account schema, required bindings and secret names, and the private Worker's tagged expansion capability. Main activation uses --containers-rollout=none first, followed by the Container rollout. Each version is tagged with the full commit SHA, and its message links to the run.
  4. Verify. scripts/deploy_status.mjs --verify retries for up to ten minutes until no migration is pending, GET /ready answers 200 (every D1 table exists and the Container is healthy), the Knowledge Worker's live version carries the commit, and the main Worker's live version carries the commit with a bundle release fingerprint (BUILD_ID from scripts/build_release_metadata.py) equal to the one computed from the checkout. When Workers Builds deploys the main Worker, the check waits for it to publish the same commit or fingerprint and only warns if it has not.
  5. Summarize. The job summary lists the pending migrations, each step's outcome, the deployed state and every check.

The job installs dependencies from the frozen lockfile exactly as the cloudflare-build ci job does, so Wrangler is the locked version, and every action is pinned to a commit SHA.

Migrations

Migrations are applied before the code that uses them. The main build refuses upload until they are recorded and the required account columns exist. The schema must still support old and new code during rollout:

  • Make only additive changes: new tables, new nullable or defaulted columns, new indexes, using IF NOT EXISTS. Add each new table to REQUIRED_D1_TABLES in worker/d1-schema.mjs so /ready reports it until it exists.
  • Name files NNNN_lowercase_words.sql, numbered from 0001 without gaps or duplicates. When two branches add the same number, the one merged second renumbers its file to the next free number; ci fails until it does. Never rename an applied migration: Wrangler tracks them by file name and would apply it again.
  • To drop or rename a table or column, first ship code that no longer uses it, then add a migration containing a line -- destructive-migration: <reason>. Without that line npm run cf:migrations:check fails in ci.

Workers Builds

Workers Builds is connected to multillm-proxy and runs npx wrangler deploy on every push to main, without migrations and without waiting for ci. If it and the deploy workflow both deploy the main Worker, they race. Choose one option.

(a) GitHub Actions deploys everything (recommended). Migrations always land before the code, and nothing ships unless ci passed.

  1. In the Cloudflare dashboard, open Workers & Pages → multillm-proxy → Settings → Builds and select Disconnect. To keep builds (and preview builds of other branches) as versions that are never promoted, set the deploy command to npx wrangler versions upload instead; those versions do not update the Container image.
  2. Then, in GitHub Settings → Secrets and variables → Actions → Variables, add the repository variable DEPLOY_MAIN_WORKER with the value true. Doing it after step 1 means no push is deployed by both.
  3. Add Containers Edit to the API token, and run the deploy workflow once from the Actions tab to deploy the current head of main.

(b) Keeping automatic promotion is not supported for the key-controls rollout. Workers Builds can publish the main Worker and Container before migrations or the private Worker, even when GitHub CI fails. Disconnect it or use wrangler versions upload without automatic promotion, and deploy through scripts/deploy.sh. The preflight's D1, secret-list and deployments-read permissions inside Builds have not been verified. See the staged rollout and rollback procedure.

One-time setup

  1. Create the API token. In the Cloudflare dashboard, go to Manage Account → Account API Tokens → Create Token, create a custom token, and limit it to this account:

    Permission Needed for
    Workers Editor, scoped to the Workers multillm-proxy and multillm-knowledge (legacy name Workers Scripts Edit) Deploying both Workers with their Durable Objects, Workflow and cron trigger; reading deployments, versions and the deployed bundle
    D1 Edit Applying migrations and reading the migration table
    Containers Edit Option (a) only: pushing the Container image and rolling it out

    The main Worker runs on workers.dev and the Knowledge Worker has no public URL, so no zone permission is needed; add Zone → Workers Routes → Edit for a zone only if you attach a route there. If a deploy is refused for the Knowledge Worker's Workflow or the account's workers.dev subdomain, widen the Workers scope to all Workers, still as Editor. deploy_status.mjs also asks the Workers Builds API which commit it deployed; if the token cannot read builds, it compares the bundle fingerprint instead. Set an expiry date and rotate the token.

  2. Create the environment. In GitHub Settings → Environments, create production. Optionally add Required reviewers to approve each deploy, and limit Deployment branches and tags to main.

  3. Add the secrets CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID to the production environment (find the account ID). Only the deploy job uses that environment, the workflow passes the secrets only to the steps that call Cloudflare, and nothing prints them. Until they exist, every deploy run stops at its first step with a message pointing here.

  4. Choose option (a) or (b) under Workers Builds.

  5. Optionally set the repository variable DEPLOY_STATUS_URL to check /ready on another origin than WORKBENCH_WORKER_URL in wrangler.jsonc.

Checking what is deployed

npm run cf:status               # for people
npm run cf:status -- --json     # for agents

scripts/deploy_status.mjs prints each Worker's live version, where it came from and the commit it was built from (its version tag, or the Workers Builds record), the local checkout's commit and fingerprint, pending migrations and /ready. It needs wrangler login or CLOUDFLARE_API_TOKEN; with CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID set it also reads the deployed bundle fingerprint and Workers Builds commits. It prints no secrets. wrangler secret put creates a new live version, which can show no commit until the next deploy.

Manual deploys

Set D1_DATABASE explicitly to the database in both selected Worker configurations, then run npm run deploy (scripts/deploy.sh). For another environment set MAIN_WORKER_CONFIG, KNOWLEDGE_WORKER_CONFIG and DEPLOY_STATUS_URL as well. The script checks migrations, applies them, deploys the private Worker, then the main Worker with Container rollout disabled, then the Container. Preflight runs before each main upload. From a clean checkout it tags both versions with the commit. Extra arguments go to the final deploy. Never run it concurrently with another deploy.

Each release uses a fresh build-only image identifier. The command does not finish successfully until application detail shows a newer Container version and image, and /ready returns matching Worker/Container artifact fingerprints. "No changes" is not rollout proof. See deployment verification. Status reads also require --database or D1_DATABASE; alternate configs are selected with --config and --knowledge-config. No default database is queried.

Alerts. Worker logs are sampled at 100% (head_sampling_rate: 1): the Worker writes little besides structured failure lines such as account_storage_failed, knowledge_edge_account_lookup_failed and d1_schema_missing, about 50,000 events a month at current traffic. In Observability, save queries on those events and on HTTP 5xx for multillm-proxy and multillm-knowledge, and create a notification (or a monitor on /ready) so an outage pages someone instead of being found by users.

Runtime Tuning

Defaults:

GUNICORN_WORKERS=1
GUNICORN_THREADS=8
GUNICORN_TIMEOUT=120
GUNICORN_GRACEFUL_TIMEOUT=30

With D1, more Gunicorn worker processes behave like more instances: each keeps its own ledger and shares usage through D1. Read State and Container Instances before adding either.

Rollback

Use the Container-first rollback procedure until A/B rollback has passed real staging. A Worker-only rollback across earlier strict RPC decoders can fail unrestricted requests. Keep migrations and stored restrictions; A refuses controlled dispatch while preserving unrestricted and management access. Do not clear policies or roll back D1 migrations to make a rollback appear healthy.

Sources