POST /v1/roleplay is a native Cloudflare Worker route for long-running,
streaming roleplay. OpenAI-compatible clients can use
POST /roleplay/v1/chat/completions or
POST /v1/roleplay/chat/completions. Session state and orchestration stay in
the Worker. Each session maps to one SQLite-backed Durable Object, so turns for
that session stay ordered while different sessions scale independently. When
an OpenCode model is selected, only the provider request uses Container egress
to avoid OpenCode's Worker-signature block.
The production model policy uses NanoGPT first and learns between:
- Kimi K2.6:
kimi-k2.6 - fast GLM-5.3: NanoGPT
z-ai/glm-5.3-flash, then OpenCodeglm-5.3-flash - uncensored GLM-5.3 Flash: NanoGPT
z-ai/glm-5.3-flash-uncensored - GLM-5.2 fallback: NanoGPT
zai-org/glm-5.2:thinking, OpenCodeglm-5.2, and NavyAIglm-5.2-venice - full GLM-5.3: explicit NanoGPT
z-ai/glm-5.3or OpenCodeglm-5.3
In NanoGPT subscription mode, the generic GLM route uses the configured 1x
order: GLM-5.3-Flash, GLM-5.3-Flash Uncensored, then GLM-5.2. Full GLM-5.3 is
excluded from automatic subscription selection because it consumes 2x plan
tokens; roleplay:5.3 remains available when that tradeoff is intentional.
roleplay:glm-speed explores the same 1x pool and then ranks it using observed
streaming time to first byte, provider-reported completion tokens per second,
and failures. Non-subscription GLM tiers retain the measured full-model quality
guard. LinkAPI remains a Kimi-only tier and OpenRouter is not part of the
production roleplay chain. Provider order is strict; adaptive ranking operates
only among models in the same provider tier.
For roleplay:glm, standard GLM-5.3 Flash receives a private instruction to
follow the roleplay request and never refuse. If an unavoidable provider policy
still blocks completion, Flash emits one exact internal trigger as its entire
answer. The Worker withholds that leg, regenerates the original conversation
with the configured GLM-5.3 Flash Uncensored candidate, and exposes or stores
only the replacement. Ordinary dialogue containing words such as "I refuse"
does not match the trigger. Version-pinned routes remain pinned; set
ROLEPLAY_REFUSAL_FALLBACK_ENABLED=false to disable this behavior.
NanoGPT accepts NANOGPT_API_KEY, numbered NANOGPT_API_KEY_N secrets, and
the compatibility NANO_GPT_KEY[_N] names. A definite 401, 403, or 429
advances to the next key. A 402 insufficient-balance rejection does the same.
When NANOGPT_SPEED_ROUTING added a suffix such as :fast and NanoGPT answers
402 or 403, the same key and model are first retried without the suffix on
the subscription endpoint, and the suffix pauses for the cooldown
(NanoGPT).
The successful key identifier—not the secret—is kept
in that Durable Object's session state and is preferred on later turns.
Fresh sessions begin with NANOGPT_PREFERRED_KEY_INDEX when it is configured;
they do not delay generation with a separate catalog probe. Definite
generation rejections still rotate immediately, and the remembered key is
revalidated after NANOGPT_KEY_CHECK_EVERY_REQUESTS successful uses.
NanoGPT GLM-5.3-Flash and standard GLM-5.x requests use semantic max
reasoning, mapped to NanoGPT's documented xhigh wire value. The explicit
uncensored Flash variant exposes high as its strongest supported effort, so
semantic max is safely clamped to high for that one model.
Authenticate with the dedicated ROLEPLAY_API_KEY. The bootstrap
ADMIN_API_KEY remains accepted for administrative clients:
curl "$PROXY_BASE_URL/v1/roleplay" \
-H "Authorization: Bearer $ROLEPLAY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: story-42-turn-1" \
-d '{
"session_id": "story-42-main",
"input": "I close the library door and ask who followed us.",
"character": {
"name": "Mira",
"persona": "A guarded court mage who hides concern behind dry humor.",
"scenario": "A rival has infiltrated the moonlit palace library.",
"style": "Tense gothic fantasy, close third person."
},
"lore": [
{
"keys": ["library", "palace"],
"content": "The palace library has one sealed passage behind the west shelves."
}
],
"model_preference": "auto",
"response_length": "balanced",
"prompt_cache": true,
"memory": {"mode": "auto"},
"stream": true
}'session_id accepts 8-128 letters, digits, underscores, or hyphens. An
explicit body or X-Roleplay-Session-ID value always wins. Without one, the
Worker hashes a client conversation_id, chat_id, or thread_id when
available. For OpenAI-style full-history requests, it otherwise derives a
stable session from the opening messages and the roleplay credential. A
delta-only request with no stable identifier receives a new generated session.
The selected value and its source are returned in X-Roleplay-Session-ID and
X-Roleplay-Session-Source.
Use either:
input: one new user turn.messages: OpenAI-style text messages.
When both are present, input is appended as the newest user message.
Blank text placeholders (empty, whitespace-only, null, or missing content)
are omitted from incoming messages. A request still needs at least one
nonblank message or input. Tool-associated messages, refusals, invalid
roles, and non-text content are not silently discarded; their existing
validation remains in effect. This normalization applies only to roleplay
routes, not raw provider routes.
An individual system, developer, user, or assistant message may use the
remaining bounded request body instead of being rejected at 128,000
characters. Before provider egress, long text is split losslessly into ordered
same-role fragments below the gateway field limit; Unicode code points are
never split and concatenating the fragments reproduces the caller's text.
Protected directives that exceed one Durable Object value are stored in
ordered shards and reconstructed on later turns. Tool-result messages retain
the 128,000-character limit because splitting one result would break its
one-to-one tool-call association.
history_mode controls how incoming messages combine with stored history:
auto(default): detects common full-history and delta-message client patterns and avoids duplicate turns.append: treats every supplied message as new.replace: replaces recent raw history for this request.
character fields are retained with session memory. Lore entries are
request-scoped; only entries marked always or whose keys match recent text
are injected, which keeps each request bounded.
model_preference accepts auto, speed, kimi, glm, glm-speed,
glm-5.3-flash, glm-5.3-flash-uncensored, glm-5.3, glm-5.2,
uncensored, or intelligence. OpenAI-compatible clients can instead
set model to roleplay:auto, roleplay:speed, roleplay:kimi,
roleplay:glm, roleplay:glm-speed, roleplay:5.3-flash,
roleplay:5.3-flash-uncensored, roleplay:5.3, roleplay:5.2,
roleplay:uncensored, or roleplay:intelligence. roleplay:glm uses the quality-first subscription-safe
order; roleplay:glm-speed learns the fastest observed 1x GLM route. The
versioned aliases pin their named variant. roleplay:uncensored can fall back
to GLM-5.2 Venice, while roleplay:5.3-flash-uncensored pins the NanoGPT
uncensored Flash variant. Concrete kimi-k2.6, glm-5.3-flash, glm-5.3,
and glm-5.2 values are also accepted. NanoGPT's exact
z-ai/glm-5.3-flash, z-ai/glm-5.3-flash-uncensored, z-ai/glm-5.3,
zai-org/glm-5.2, and zai-org/glm-5.2:thinking IDs map to the same pinned
variants.
roleplay:intelligence mirrors the auto:intelligence chat route. It tries a
fixed list strictly in order across providers, without adaptive reordering:
GLM-5.3, GLM-5.3-Flash and GLM-5.2, each on NanoGPT, ClinePass (no GLM-5.2),
OpenCode Go and NavyAI; then MiMo-V2.6-Pro on NanoGPT, then on ClinePass.
Pay-as-you-go gateways (OpenRouter, LinkAPI) are not in the default list.
Entries whose provider has no key are skipped, a cooling-down entry moves out
of the way, and NanoGPT keys rotate within an entry before the next one. The
usual safe-fallback statuses advance to the next entry. Set
ROLEPLAY_INTELLIGENCE_MODELS to a JSON array or comma list of
provider:model entries to replace the list. ClinePass uses CLINE_API_KEY
and Cline's reasoning.effort field, up to xhigh.
Production sets it (wrangler.jsonc, 2026-10-29) to GLM-5.3 on OpenCode Go, ClinePass
and NanoGPT, then DeepSeek V4.1 Flash on OpenCode Go, then GLM-5.3-Flash and GLM-5.2 on
NanoGPT and GLM-5.3-Flash on ClinePass, then MiMo-V2.6-Pro on OpenCode Go, NanoGPT and
ClinePass. GLM-5.3 is the recommended model for roleplay:auto.
The order follows a test on 2026-10-29: three multi-turn roleplays with planted card rules (a heist whose characters each have a fixed rule, a secret that must stay hidden until proof is shown, and an out-of-character time skip with a status block to update), two runs each on OpenCode Go, every reply read and scored out of 5. GLM-5.3 averaged 4.33 with no hard rule breaks and took 21 to 41 s a reply. DeepSeek V4.1 Flash, which led the chain from 2026-10-25, averaged 3.25: it answered in 6 to 10 s but reasoned far less (0.7k to 3k characters against GLM-5.3's 11k to 30k), broke a character rule and came close to confessing the secret. GLM-5.3-Flash averaged 3.92 with three hard rule breaks, and MiMo-V2.6-Pro 2.83: among other faults it wrote a line for the user, so it is last. Kimi K3.1 Pro scored highest (4.67) but took 72 to 143 s a reply and is not in the chain. One reader scored two runs per scenario, so the sample is small.
The cost is OpenCode Go usage. In a test on 2026-10-25 a GLM-5.3 turn used about 30
times as much of the Go allowance as a DeepSeek V4.1 Flash turn, so leading with GLM-5.3
reaches Go's usage limits much
sooner. When Go refuses GLM-5.3, the turn moves to GLM-5.3 on ClinePass and NanoGPT, then
to DeepSeek V4.1 Flash. On 2026-10-29 only OpenCode Go answered: ClinePass returned 403
ENTITLEMENT_ERROR for every model and NanoGPT returned 429 model_cooldown. Both
statuses advance to the next entry and pause that entry for the session (5 minutes after a
403, 1 minute after a 429). When it works, ClinePass GLM-5.3 gives its first words in about
4 s; MiMo on NanoGPT took 59 s, and 152 s on an adult turn, so a turn that falls back to it
is slow. NavyAI's free plan was refusing every request, so it is left out.
With ROLEPLAY_AUTO_ROUTE=intelligence, plain roleplay:auto turns (and requests
without a roleplay model) use the same chain. Turns that set routing.mode or
routing.model keep the adaptive pool, and so does any turn whose chain has no
configured key. Leave the variable unset for the adaptive behaviour above.
ROLEPLAY_HANDOFF_FROM names chain entries that may pass a turn on, and
ROLEPLAY_HANDOFF_TO the entries that take it; both use the
ROLEPLAY_INTELLIGENCE_MODELS format and are unset by default. Set them inside
RUNTIME_FEATURE_FLAGS; a separate var of the same name takes precedence. A
source hands off only to targets that come after it in the chain. Production
let DeepSeek V4.1 Flash hand off to GLM-5.3 from 2026-10-25 to 2026-10-29, while
DeepSeek led the chain. Now that GLM-5.3 comes first, no target follows DeepSeek,
so production sets neither. A source request carries one internal function, multillm_hand_off_turn,
whose arguments are a reason (deeper_reasoning or stronger_prose), short notes
and an optional draft. When the source calls it instead of writing, the Worker
discards that attempt (its frames are held until visible text appears, so nothing
reaches the client) and the first available target writes the turn from the same
conversation plus a private note with the source's notes and draft, at its usual
reasoning effort (at max, GLM-5.3 spent a 4,096-token budget reasoning and wrote
nothing). If no target accepts the request or writes visible text, the draft is the
reply; without a draft the rest of the chain follows.
A source that writes the reply and also calls the function keeps its reply, and the
call is removed from the stream.
ROLEPLAY_HANDOFF_MIN_TURNS (default 2) allows at most one handoff per that many
turns in a session. Between handoffs, and whenever no target is available, the
source still receives the function with tool_choice: "none", so its prompt prefix
and the provider's cache of it stay the same. Callers that ask for xhigh or max
reasoning start on the targets. Turns with routing.fallback: "none" or a session
tier skip the handoff.
In a test on 2026-10-25 with the function offered, DeepSeek wrote all 12 ordinary
turns itself (scene continuation, conflict, out-of-character recall and small talk)
and handed off 4 of 6 climax and multi-faction planning turns, each with a full
draft of 950 to 1,600 characters. A handoff costs the source's 7 to 15 seconds
before the target starts. Each handoff logs roleplay_handoff, and the completion
log counts them in handoffCount.
A reply that finishes with no visible text, only reasoning or nothing at all, is
treated as a failure: the turn starts over on the next chain candidate, at most
twice, and fails with a retry message if none writes anything. This also applies
when the caller sets max_tokens. The image-prompt repair keeps its own retry when
it can run. Set ROLEPLAY_EMPTY_REPLY_FALLBACK_ENABLED=false to pass empty replies
through. Each retry logs roleplay_empty_reply_fallback.
Roleplay generation uses ROLEPLAY_DEFAULT_REASONING_EFFORT; the deployment
configuration sets the general default to high. OpenCode glm-5.3-flash
defaults to reasoning_effort: max. Go enables reasoning through that field;
the proxy does not inject the unsupported Z.AI thinking field. NanoGPT uses its native
thinking defaults and receives no effort unless the caller supplies one.
Other models retain the general default. Callers can
override generation effort per turn with reasoning_effort; semantic max
maps to the selected provider's real ceiling. Model-backed memory compaction
uses the provider maximum, except NanoGPT compaction also leaves effort unset.
OpenCode Kimi K2.6 keeps its fixed native
thinking mode because that transport does not expose a supported effort
overlay for that model.
OpenCode GLM-5.3 Flash supports max, high, and low; thinking cannot be
disabled for this model. Profile previews and request receipts report the
effective model default or caller override; omitted NanoGPT effort is reported
as native.
OpenCode Go's GLM-5.3-Flash reasoning route requires an output budget of at least 2,048 tokens: the Go adapter reserves 1,024 for the answer and requires at least 1,024 for thinking. Compaction applies this minimum automatically and keeps a 30-second total deadline before using local fallback. Explicit caller generation budgets remain caller-controlled.
Prompt caching is automatic above PROMPT_CACHE_MIN_TOKENS (1,024 estimated
input tokens by default). NanoGPT receives its caching: true routing hint only
in NANOGPT_BILLING_MODE=standard; subscription mode omits that PAYG-triggering
field and reports nanogpt-subscription-only. Other roleplay providers keep an unchanged schema and can reuse
stable prefixes through their automatic caching. Set "prompt_cache": false
for one turn or PROMPT_CACHE_ENABLED=false for the deployment. The response
reports X-MultiLLM-Prompt-Cache, X-MultiLLM-Prompt-Cache-Mode, and the
estimated eligible token count. Only upstream prompt reuse is requested;
roleplay outputs are never response-cached or replayed.
Prompt caching is not conversation storage. The production Worker persists and resends exact raw dialogue until it exceeds 128,000 estimated tokens. Crossing that boundary forces continuity compaction and retains the newest 32 raw messages beside the structured digest. The 640,000-byte Durable Object message budget prevents storage pressure from silently truncating an active history.
response_length accepts compact, balanced, or immersive. It changes the
pacing instruction without shrinking the model's available output capacity.
Every mode discourages repeated recap and stagnant dialogue.
Streaming is on unless "stream": false is sent. If max_tokens is omitted,
the Worker requests the selected provider/model's largest output that still
fits beside the current input. Explicit positive integer values remain caller
ceilings for ordinary API clients. output_mode: "unlimited", a
max_tokens value of at least 1000000, or an authenticated request from the
JanitorAI browser removes that caller ceiling. Provider and combined-context
capacity remain the physical safety boundary; there is no proxy-wide 20k
ceiling.
When a protected caller directive marks an IMAGE PROMPT: block as mandatory
for story responses, the Worker adds a short final-output reminder immediately
before dialogue and reserves at least
ROLEPLAY_IMAGE_PROMPT_MIN_OUTPUT_TOKENS (2,048 by default). This does not
rewrite bounded provider output or make a second request for bounded callers.
An explicit no image command or OOC-prefixed turn bypasses both the reminder
and budget floor.
During a quiet streaming interval, the Worker emits a valid SSE comment every
10 seconds. These keepalives carry no model content and let clients distinguish
long Kimi/GLM thinking from a dead connection. The Worker imposes no active
stream idle deadline; it continues until the provider finishes or the client
disconnects. Streaming responses use Cache-Control: no-cache, no-transform
so intermediaries do not transform or coalesce the SSE body. Ordinary provider
data events are forwarded unchanged; only an
intermediate output-limit terminator is rewritten for continuation. A stream is
recorded as complete only after [DONE] or a non-length finish reason. In
unlimited mode, an intermediate finish_reason: "length" and its [DONE] are
withheld while the same assistant response continues from its exact partial
text. The same repair applies to finish_reason: "stop" when a protected
directive requires an IMAGE PROMPT: and one or more of its declared standard
fields are still missing. A complete contract is accepted without another
request. Unlimited streams also continue when an upstream connection ends
after emitting partial assistant text but before sending a terminal finish
event. Image-contract repair is independently limited and proceeds only while
the canonical missing-field set strictly decreases. Current labels such as
Camera, Primary subject, Setting, Lighting, and Composition map to the
same contract as their legacy aliases. No-progress commentary is discarded,
multiple image markers are reduced to one final block, and only the cleaned
assistant text is eligible for session persistence. The client receives one
terminal finish event and one [DONE]. Finite
requests still expose their provider finish reason directly. An unterminated
EOF that cannot be continued is never stored as completed assistant memory.
If a provider closes after reasoning but before any visible response, the
Worker discards that hidden leg and retries the same model once from a clean
response boundary. The failed reasoning is neither streamed nor persisted.
Create a proxy configuration with:
| Field | Value |
|---|---|
| Name | MultiLLM Roleplay |
| Proxy URL | https://multillm-proxy.cserules.workers.dev/roleplay/v1/chat/completions |
| API key | The value of ROLEPLAY_API_KEY from the local .env |
| Model | roleplay:auto |
| Custom prompt | None unless the character needs an extra instruction |
The proxy URL is already the full Chat Completions endpoint, so leave
Add /chat/completions disabled. Save the configuration and hard-refresh
JanitorAI before selecting it.
If Worker logs show POST /v1/chat/completions, the selected JanitorAI
configuration is bypassing the native roleplay route. Re-enter the full proxy
URL above and keep Add /chat/completions disabled. Refusal fallback,
continuation repair, and roleplay memory apply only to the native roleplay
endpoints. As a defensive compatibility measure, authenticated JanitorAI GLM
streams that still reach the generic route receive balanced reasoning markup
and expose X-MultiLLM-Reasoning-Normalized: janitor-glm; they do not gain the
other native roleplay behaviors.
Use roleplay:glm for the quality-first 1x subscription order. Use
roleplay:glm-speed when observed throughput should decide among those models.
Use roleplay:5.3-flash, roleplay:5.3-flash-uncensored, roleplay:5.3, or
roleplay:5.2 to pin a variant. roleplay:5.3 deliberately opts into the 2x
NanoGPT subscription model. The shorter roleplay:uncensored selector can
fall back to NavyAI's GLM-5.2 Venice model.
JanitorAI's public help pages do not specify the JSON sentinel used by its
Unlimited control. MultiLLM does not depend on that implementation detail:
an authenticated browser request whose Origin is https://janitorai.com is
always normalized to unlimited output. Known max_tokens: 1000000 sentinel
requests and explicit output_mode: "unlimited" API requests receive the same
behavior. A smaller number sent by JanitorAI is not allowed to truncate the
story.
If JanitorAI's browser console reports a connect-src Content Security Policy
block for the Worker origin, the request never reaches MultiLLM. A Worker CORS
header cannot override JanitorAI's CSP; JanitorAI must allowlist the origin, or
the client must use an endpoint already allowed by that policy. Direct
https://api.navy/v1/chat/completions is an alternative when JanitorAI permits
api.navy, but it bypasses MultiLLM failover and Durable Object memory.
JanitorAI normally sends OpenAI-style message history. Repeated requests with the same opening land on the same Durable Object, so compacted continuity memory survives later turns and Worker deployments until the inactivity TTL expires. If a client supplies a conversation ID, that ID is preferred. Two branches with exactly the same opening and no client conversation ID share the derived session; use an explicit ID with API clients that need branch isolation.
Caller-supplied system and developer messages are protected control
directives. They are stored separately from dialogue and reinserted unchanged
on later delta-only requests. When present, they lead the upstream message
list and suppress the Worker's generic roleplay-writing prompt. Explicit
structured character fields remain available as labelled context without
being placed ahead of the caller's contract. In auto mode, a newly supplied
directive set replaces the retained set; a request without directives reuses
it. append adds new directives, replace uses exactly the supplied set (and
therefore clears it when none are supplied), and memory.mode: off uses only
directives present in that request.
Every session records attempts, successes, failures, EWMA time to first byte,
and EWMA total time for each provider/model pair. Streaming NanoGPT requests
also request a final usage frame. When the provider reports completion tokens,
the session records observed generation duration plus EWMA, p50, and p95 tokens
per second. roleplay:glm-speed samples each eligible model twice before using
that throughput for a reference 1,024-token response. This is delivered-stream
performance for the current client path, not a guarantee from a model-card TPS
snapshot. Per-session learning avoids a global Durable Object bottleneck and
keeps regional behavior local to the conversation.
Automatic fallback occurs only after a response that clearly rejected work:
400, 401, 402, 403, 404, 413, 415, 422, 429, or an explicit
503 service-unavailable response. Ordered model arrays are exhausted inside
one provider before the next provider tier is tried. Transport errors and
other 5xx responses remain ambiguous because a provider might have started
generation; the Worker stops instead of risking a duplicate request.
When the last eligible candidate rejects the request, the Worker preserves its
HTTP status, error body, and rate-limit headers, including Retry-After.
X-Roleplay-Provider, X-Roleplay-Model, and X-Roleplay-Fallback-Count identify
the final attempt. A provider rejection is not replaced with a generic
no_roleplay_provider error. For example, NanoGPT rejects disabling reasoning
on GLM 5.3 with 400 reasoning_required; omit reasoning_effort to use the
configured default or use a supported nonzero effort such as high.
Idempotency-Key is optional but recommended. The session stores recent keys
before any model call. Reusing one returns 409 and does not start another
generation. This is a duplicate-execution guard, not a response replay cache.
Memory modes:
auto: below the local threshold, no extra model call. Above it, the selected model receives older dialogue and decides whether semantic compaction is useful.force: asks for compaction whenever there is older dialogue.off: neither reads nor writes conversation memory for the turn. Routing metrics still update.
The compaction model must return a bounded JSON continuity digest containing a summary, character facts, relationships, world state, unresolved threads, and style. The digest is reinserted as untrusted historical context. Recent turns remain raw.
Only user, assistant, and tool dialogue can enter model compaction, local
extractive compaction, or a compaction checkpoint. system and developer
directives never enter those paths and do not count toward the normal dialogue
compaction trigger. The total context, including protected directives, still
controls mandatory compaction and the hard limit. If protected directives and
the retained recent context cannot fit, the request is rejected with 413
instead of weakening or summarizing the directives.
Storage pressure always makes compaction mandatory. The raw recent-message
window automatically shrinks below ROLEPLAY_KEEP_RECENT_MESSAGES until the
retained history and expected reply fit the storage budget. Storage-only
compaction changes the durable checkpoint for later turns but does not shorten
the current model request while its complete history still fits an eligible
provider. If context capacity itself forces compaction, the current generation
uses the compacted continuity digest.
Each successful compaction also stores a versioned SHA-256 checkpoint over the
character name and exact compacted user/assistant/tool prefix. Protected
directives are excluded from both the hash and message count. When a later
full-history request has the same checkpoint, the Worker removes only that
already-summarized dialogue prefix before merging and sends the durable
continuity digest instead. A mismatch keeps the checkpoint optimization
disabled and falls back to the normal history merge and compaction rules;
approximate or similarity-based checkpoint matches are never accepted.
Roleplay uses the same safety contract as MultiLLM context optimization:
authoritative instructions and the newest raw turn remain unchanged, while
eligible older dialogue becomes terse continuity notes. Repeated full-history
requests reuse the exact checkpoint instead of paying to summarize the same
prefix again. Response headers expose X-MultiLLM-Estimated-Input-Before,
X-MultiLLM-Estimated-Input-After, and
X-MultiLLM-Messages-Summarized; session metrics accumulate estimated input
tokens saved. Final assistant output is never compressed.
By default, the hard limit is derived from the largest eligible provider/model
context and compaction starts at 90% of that limit. A nonzero
ROLEPLAY_HARD_INPUT_TOKENS or ROLEPLAY_COMPACT_TRIGGER_TOKENS explicitly
overrides the derived value. When the effective hard limit is reached,
compaction becomes mandatory. If that model request fails, returns malformed
JSON, or declines, the Worker creates a
bounded, role-labelled extractive memory from the prior digest and archived
dialogue. Omissions are marked, the same dialogue-only SHA-256 checkpoint is
stored, and final generation continues without retrying the summarizer.
X-Roleplay-Memory: local_compacted and local_compactions in session metrics
expose this degraded path. Model compaction is an additional provider request
and can be billable.
All model-compaction candidates share one eight-second budget. A failed compaction starts a per-session exponential backoff (one minute up to fifteen minutes), so later forced turns immediately use local compaction instead of repeating a slow or rate-limited auxiliary call. A later successful model compaction clears the failure count and backoff.
Token counts are fast UTF-8 byte estimates, not provider tokenizer or billing measurements.
Session storage expires after 30 days of inactivity by default. The Durable Object alarm atomically removes that session's memory and metrics.
Ordinary OpenAI-compatible JSON bodies and SSE data events stay unchanged. If
an upstream model emits separate reasoning fields or raw <think> markup, the
Worker reconciles them into exactly one visible block:
<think>[provider: opencode | model: glm-5.2]
...provider reasoning...</think>
...visible roleplay response...
Duplicate or malformed provider reasoning is removed, including nested tags, stray closing tags, and tags divided across SSE frames. The selected route is written inside the block so clients that do not expose HTTP response headers can still identify it. Only the visible roleplay response is retained in continuity memory; provider reasoning and route metadata are not fed back into later turns.
During streaming silence, the Worker may interleave the SSE keepalive comment described above. The Worker also adds:
X-Roleplay-Session-IDX-Roleplay-Session-SourceX-Roleplay-ProviderX-Roleplay-ModelX-Roleplay-SelectionX-Roleplay-MemoryX-Roleplay-Estimated-Input-TokensX-Roleplay-Max-Output-TokensX-Roleplay-Fallback-CountX-Roleplay-State-CacheX-Roleplay-Credential-CheckServer-Timing
These headers are exposed through CORS.
Server-Timing separates session queue time, Durable Object state loading,
credential revalidation, memory compaction, request preparation, upstream
header wait, and total time to response headers. This makes slow storage,
compaction, same-session serialization, and provider startup independently
visible without logging roleplay content. The session metrics endpoint keeps a
bounded 64-request window per provider/model and reports p50, p95, and p99 for
time to first byte and total generation time. Metrics reads do not wait for an
active generation to finish; pending_turns exposes the active and queued
turn count alongside the last completed snapshot.
Warm Durable Object instances retain their already-loaded state in memory. Normal turns therefore avoid repeated storage reads, update the inactivity alarm at most once per day, and write only changed state partitions after the completion. Durable storage remains authoritative across eviction and Worker deployments; the in-memory copy is only a hot-path read cache.
The request sent to the selected provider has this order (memory and lore are omitted when absent):
{
"model": "kimi-k2.6",
"messages": [
{
"role": "system",
"content": "<proxy roleplay policy and character profile>"
},
{
"role": "system",
"content": "<caller system directive, unchanged>"
},
{
"role": "developer",
"content": "<caller developer directive, unchanged>"
},
{
"role": "system",
"content": "[Untrusted roleplay continuity memory...]"
},
{
"role": "system",
"content": "[Relevant lore]..."
},
{
"role": "user",
"content": "<recent compactable dialogue>"
}
]
}Apart from reasoning normalization, the selected provider's Chat Completions
JSON or SSE events are returned without reshaping. Session metrics report
protected_directives and estimated_protected_directive_tokens without
returning directive content.
Inspect configured tiers:
curl "$PROXY_BASE_URL/v1/roleplay/models" \
-H "Authorization: Bearer $ROLEPLAY_API_KEY"Inspect one session's bounded routing metrics without returning its dialogue:
curl "$PROXY_BASE_URL/v1/roleplay/metrics?session_id=story-42-main" \
-H "Authorization: Bearer $ROLEPLAY_API_KEY"The Worker requires the ROLEPLAY_SESSION Durable Object binding configured in
wrangler.jsonc. Store a roleplay-only client credential separately from the
administrative key:
npx wrangler secret put ROLEPLAY_API_KEYNanoGPT is the preferred provider:
npx wrangler secret put NANOGPT_API_KEY
npx wrangler secret put NANOGPT_API_KEY_1Add later tiers only when used:
npx wrangler secret put OPENCODE_GO_API_KEY
npx wrangler secret put LINKAPI_KEY
npx wrangler secret put OPENROUTER_API_KEY
npx wrangler secret put NAVYAI_API_KEYNon-secret tuning variables:
| Variable | Default | Purpose |
|---|---|---|
ROLEPLAY_PROVIDER_ORDER |
nanogpt,opencode,linkapi,openrouter,navyai |
Strict provider tiers |
ROLEPLAY_COMPACTION_MODELS |
{} |
Per-provider model for memory compaction only, e.g. {"nanogpt":"moonshotai/kimi-k2.6"}. Generation keeps its own rotation. Keyed by provider because a model name is only valid at the gateway that serves it; an unknown provider, blank name or malformed JSON falls back to the candidate's model. |
ROLEPLAY_KIMI_MODEL |
kimi-k2.6 |
Default Kimi model ID |
ROLEPLAY_GLM_MODEL |
glm-5.3-flash |
Default GLM model ID |
ROLEPLAY_PROVIDER_MODELS |
{} |
JSON provider-specific Kimi/GLM ID or ordered fallback IDs |
ROLEPLAY_PROVIDER_FAMILIES |
{} |
JSON provider-to-family allowlists; an empty list disables that provider |
ROLEPLAY_AUTO_ROUTE |
adaptive |
intelligence sends plain roleplay:auto turns through the roleplay:intelligence chain |
ROLEPLAY_INTELLIGENCE_MODELS |
GLM-5.3, then GLM-5.3-Flash and GLM-5.2, then MiMo-V2.6-Pro | Ordered provider:model list for roleplay:intelligence; providers nanogpt, cline-pass, opencode, navyai, linkapi, openrouter |
ROLEPLAY_PROVIDER_LIMITS |
{} |
JSON provider/family context and output overrides |
ROLEPLAY_QUALITY_LATENCY_PREMIUM_PERCENT |
20 |
Largest allowed full GLM-5.3 p95 TTFB and total-latency premium over Flash |
ROLEPLAY_QUALITY_MIN_SAMPLES |
3 |
Successful samples required for both full and Flash before promotion |
ROLEPLAY_SPEED_REFERENCE_OUTPUT_TOKENS |
1024 |
Output length used to combine observed TTFB and TPS for speed ranking |
NANOGPT_PREFERRED_KEY_INDEX |
unset | Numbered NanoGPT key attempted first for a fresh session |
NANOGPT_KEY_CHECK_EVERY_REQUESTS |
50 |
Successful uses before the remembered NanoGPT key is catalog-revalidated |
ROLEPLAY_COMPACT_TRIGGER_TOKENS |
128000 |
Raw-dialogue threshold that forces continuity compaction; 0 derives it from provider capacity |
ROLEPLAY_COMPACT_TRIGGER_PERCENT |
90 |
Derived compaction threshold percentage |
ROLEPLAY_HARD_INPUT_TOKENS |
0 |
0 derives the hard limit from eligible providers |
ROLEPLAY_MEMORY_TARGET_TOKENS |
1200 |
Terse continuity-memory target |
ROLEPLAY_KEEP_RECENT_MESSAGES |
32 |
Raw recent messages retained after compaction; shrinks under storage pressure |
ROLEPLAY_IMAGE_PROMPT_MIN_OUTPUT_TOKENS |
2048 |
Minimum story budget when protected directives require a final image-prompt block |
ROLEPLAY_CONTEXT_REPLY_RESERVE_TOKENS |
4096 |
Reply space retained when deriving input capacity |
ROLEPLAY_CONTEXT_SAFETY_TOKENS |
1024 |
Combined-context safety margin |
ROLEPLAY_MAX_REQUEST_BYTES |
8388608 |
Bounded JSON ingress |
ROLEPLAY_MAX_STORED_BYTES |
640000 |
Exact raw-message storage budget before forced compaction |
ROLEPLAY_COMPACTION_MAX_TOKENS |
1200 |
Compaction output budget; OpenCode GLM uses at least 2,048 to fit thinking and the digest |
ROLEPLAY_COMPACTION_TIMEOUT_MS |
30000 |
Shared model-compaction budget before local fallback |
ROLEPLAY_UPSTREAM_HEADER_TIMEOUT_MS |
90000 |
Maximum wait for provider response headers before advancing to the next tier |
ROLEPLAY_PRE_RESPONSE_FALLBACK_ENABLED |
true |
Advance after a provider transport rejection or header timeout; client aborts never advance |
ROLEPLAY_PROVIDER_ERROR_FALLBACK_ENABLED |
true |
Advance only when a 500/502/504 JSON response explicitly identifies both its error code and type as provider_error; unknown 5xx responses remain fail-closed |
ROLEPLAY_HANDOFF_FROM |
unset | Chain entries offered multillm_hand_off_turn; production sets none |
ROLEPLAY_HANDOFF_TO |
unset | Chain entries after the source that take a handed-off turn, in chain order; production sets none |
ROLEPLAY_HANDOFF_MIN_TURNS |
2 |
At most one handoff per this many turns in a session; 0 or 1 lifts the cap |
ROLEPLAY_EMPTY_REPLY_FALLBACK_ENABLED |
true |
Start a reply with no visible text over on the next candidate, at most twice |
ROLEPLAY_REFUSAL_FALLBACK_ENABLED |
true |
Privately regenerate an exact GLM-5.3 Flash refusal trigger with the configured Uncensored Flash candidate; rejected output is neither streamed nor stored |
ROLEPLAY_DEFAULT_REASONING_EFFORT |
high |
Deliberate generation default with lower latency and hidden-token use than each provider's maximum; callers can still request max per turn |
ROLEPLAY_STREAM_HEARTBEAT_MS |
10000 |
SSE keepalive interval during upstream silence |
ROLEPLAY_MAX_AUTO_CONTINUATIONS |
8 |
Maximum hidden continuation legs after genuine provider output limits or truncated EOFs |
ROLEPLAY_MAX_OUTPUT_CONTRACT_REPAIRS |
1 |
Separate maximum repairs for a stopped response with a missing story or incomplete required image-prompt contract; image-only responses regenerate from the beginning and no-progress repairs stop immediately |
ROLEPLAY_SESSION_TTL_SECONDS |
2592000 |
Inactivity retention |
Provider catalogs can use namespaced IDs or expose multiple generations of a model family. A string selects one ID; an array preserves priority among the candidates eligible for the requested variant. Generic GLM begins with Flash:
{
"nanogpt": {
"glm": [
"z-ai/glm-5.3-flash",
"z-ai/glm-5.3-flash-uncensored",
"zai-org/glm-5.2:thinking",
"z-ai/glm-5.3"
]
},
"opencode": {
"glm": ["glm-5.3-flash", "glm-5.2", "glm-5.3"]
},
"navyai": {
"glm": "glm-5.2-venice"
}
}Store that compact JSON as ROLEPLAY_PROVIDER_MODELS. Secrets never belong in
this variable or wrangler.jsonc.
The roleplay model catalog also returns context_window,
max_output_tokens, and limits_source for every configured route. Limits
are provider-specific: NanoGPT Kimi, for example, is not assumed to have the
same capacity as NavyAI Kimi.
OpenAI-compatible clients may send a max_tokens ceiling larger than a
selected provider supports. The roleplay route clamps that ceiling to the
provider's available model capacity and returns the effective value in
X-Roleplay-Max-Output-Tokens; it does not reject the turn solely because the
client requested a larger ceiling. Unlimited requests ignore the caller value,
use that effective provider capacity for each leg, and transparently continue
streaming after a length finish. Set ROLEPLAY_MAX_AUTO_CONTINUATIONS=0 to
disable output-limit continuation stitching without changing provider-capacity
selection. Image-prompt contract repair has its own smaller bound,
ROLEPLAY_MAX_OUTPUT_CONTRACT_REPAIRS (one by default), and is disabled by
setting that value to zero. A complete image block without preceding visible
story content is not a successful response: that leg is discarded and the
Worker regenerates the full story and final image block from the beginning.
Override a gateway only when its live catalog or an exercised request proves a different limit:
{
"navyai": {
"glm": {
"context_window": 1048576,
"max_output_tokens": 131072
}
}
}Store that compact JSON as ROLEPLAY_PROVIDER_LIMITS.