feat(cursor): Cursor provider (chat + Tab) with cross-request session reuse#154
Draft
yyyr-p wants to merge 83 commits into
Draft
feat(cursor): Cursor provider (chat + Tab) with cross-request session reuse#154yyyr-p wants to merge 83 commits into
yyyr-p wants to merge 83 commits into
Conversation
…ading inline 169e592 changed the CC→Messages and Responses→Messages translators to emit all system/developer messages inline as `role: "system"`. Upstream Anthropic endpoints reject position-0 inline system — the most common Chat Completions pattern — so the initial system prompt must live in the top-level `system` field. Hoist the leading contiguous run of system/developer messages to `MessagesPayload.system`. Non-leading system/developer messages stay inline as `MessagesSystemMessage`, preserving their chronological position. Upstreams that reject inline system in non-leading positions can enable the existing `demote-interleaved-system-to-user` interceptor.
… image-in-system content
Tightens the leading system hoist introduced in this PR so the translator's
output stays faithful to the caller's content shape, and turns the
pre-existing silent drop of image parts in system content into an explicit
rejection.
Boundary preservation. The previous hoist concatenated every text part in a
leading system message with empty-string join, then concatenated multiple
leading system messages with `\n\n`, and wrapped the whole result in one
`MessagesTextBlock`. That collapsed two layers of caller-visible structure
(ContentPart array within a message; multiple messages within the prefix)
into a single string — e.g. `[{text:'A'}, {text:'B'}]` became `'AB'`,
glueing text fragments with no separator. Each ContentPart text and each
leading message now contributes its own `MessagesTextBlock` to
`MessagesPayload.system`. The same change applies to the Responses path,
where `payload.instructions` and leading input system messages also stay as
distinct blocks instead of being `\n\n`-joined into one. Downstream prompt
cache and `applyLastSystemCacheBreakpoint` (new helper in
`shared/via-messages/cache-breakpoints.ts`) place the breakpoint on the
last block, mirroring how tool and message breakpoints are applied.
Image-in-system rejection. Anthropic's Messages API only permits text in
the system field — both top-level `MessagesPayload.system` and inline
`MessagesSystemMessage.content`. The Chat Completions and Responses
translators previously filtered non-text parts silently, leaving callers
to wonder why their system-attached image was missing on the wire. They
now throw on `image_url` (Chat Completions) and `input_image` (Responses)
parts in system / developer messages. Upstream-probed wording for
reference: both Bedrock and Vertex return
`system.1.type: Input should be 'text'` for a top-level system image, and
Bedrock additionally returns
`messages.N: role 'system' supports text and tool_change blocks only`
for an inline system image; the translator's error message names the
caller-visible shape rather than the Anthropic field path.
Comment restoration. The original comment explaining `case 'system':
case 'developer':` mapping was deleted by the hoist change. We add a new
comment explicitly noting that this branch only runs for non-leading
system / developer messages (leading prefix is hoisted before
`buildMessagesInput` runs), that developer normalizes to `role:'system'`
because it is the same intent layer on the source wire, and that the
gateway's `demote-interleaved-system-to-user` interceptor flag is the
safety net for upstreams that reject inline `role:'system'` (Vertex
unconditionally; Bedrock when placement rules are violated).
`extractSystemText` in the Responses translator (used the loose
`'text' in block` shape match) is replaced by a single
`responsesSystemBlocks` helper that uses the same `input_text` /
`output_text` whitelist as the inline `translateSystemMessage` and shares
the image-rejection check, so both hoist and inline paths now agree on
which content blocks contribute and on what triggers the throw.
…/non-empty system mix Round 1 of review-and-cleanup. Adds two test gaps for invariants the previous commit documented in code but did not exercise. - `applyLastSystemCacheBreakpoint` now has direct unit coverage (no-op on undefined / empty input; marks only the last block when multiple are present), matching the local convention in cache-breakpoints_test.ts where each helper has its own test. - Chat Completions hoist: a leading empty system followed by a leading non-empty system / developer message — the empty must consume its prefix slot without contributing a block, the non-empty must still be hoisted as part of the same contiguous prefix.
Cleanup round 1. Removes the function-level prose above applyLastSystemCacheBreakpoint that restated the function name and duplicated the file-level "system block when system text is non-empty" placement note (the sibling applyLastToolCacheBreakpoint has no analogous comment), and condenses the empty-content explanation in the hoist loops on both translator pairs to just the one non-obvious clause (that an empty-content leading message still extends the contiguous prefix).
Round 2 cleanup. The dual `case 'system': case 'developer':` labels falling through to a single body already convey that developer normalizes to role:'system' here, so the dedicated comment line restating that fact reads as narration of the case label. Drop it from both translator pairs; the surrounding decision-recording about non-leading inline placement and the demote-interleaved-system safety net stays.
Round 2 cleanup. Drops three more lines of mechanical narration the audit flagged on the second pass: - The hoist comment's "An empty-content leading message still extends the contiguous prefix" sentence narrates the unconditional `prefixEnd++` directly below it. The behavior is visible in the loop and locked in by an explicit unit test, so the comment carries no additional information. Removed on both translator pairs. - The inline-branch comment on both translators re-described the hoist contract (which is already authoritative at the hoist call site) before getting to the Bedrock/Vertex divergence and interceptor-flag pointer that is the only load-bearing content. Compressed to one line. - The responses-via-messages top-level comment ended with "The cache breakpoint lands on the last block via applyLastSystemCacheBreakpoint", restating the next line of code. Removed.
Round 3 cleanup. The hoist comment in chat-completions-via-messages named the demote-interleaved-system-to-user interceptor flag for the non-leading inline case, which the inline-branch comment already covers with full Bedrock/Vertex divergence context — drop the duplicate reference and let each comment own its unique content. The neighboring test name for the leading-empty + leading-non-empty case described the algorithm's internal mechanism (prefix-slot consumption) rather than the observable contract; shorten to the contract-focused form that matches the surrounding test naming style in the file.
…ssagesTextBlock[]
Round 4 review recommendation, applied directly in-PR.
Both translator pairs' system-content helpers (convertSystemContent on
the Chat Completions side, responsesSystemBlocks on the Responses side)
previously had a dual return shape — string passed through verbatim;
content arrays became MessagesTextBlock[] — which forced the hoist call
sites to fork on `typeof converted === 'string'` and re-wrap a non-empty
string as a single text block. Collapse both helpers to a single
MessagesTextBlock[] return (string content "Hello" → [{type:'text',
text:'Hello'}]; empty content → []) so the hoist becomes a one-line
`systemBlocks.push(...convertSystemContent(message.content))` and the
inline path emits the blocks directly (or '' when empty, preserving the
empty-content wire shape).
Wire side-effect: inline non-leading MessagesSystemMessage with a
plain-string source-content now emits `{role:'system',
content:[{type:'text', text:...}]}` instead of `{role:'system',
content:'...'}`. Both shapes are equivalent under
`MessagesSystemMessage.content: string | MessagesTextBlock[]` and any
Anthropic upstream accepts both equally; the per-translator inline
tests are updated to assert the block-array form.
…t round trip
Round 4 review recommendation, applied directly in-PR.
The Messages → Chat Completions translator previously collapsed both
inline `MessagesSystemMessage` content and top-level
`MessagesPayload.system` block arrays into a single Chat Completions
system message with `\n\n`-joined string content — the exact boundary
loss the forward direction fixes in this PR, just on the reverse trip.
A CC→Messages→CC round trip would silently merge per-block segments
that callers placed deliberately and that the prompt cache cares about.
Switch both code paths to emit Chat Completions `ContentPart[]` text
parts when the source is a block array, one part per block. The
string-source branch still emits string content (no shape change for
single-string sources). Also tighten the empty-system guard to skip the
system message entirely when an empty array is passed (previously
`system ?` was truthy on `[]` and emitted `{role:'system', content:''}`,
which is wire-noise downstream models may reject).
The existing "joins in-array system text blocks with double newline" test
enshrined the boundary-loss behaviour; it is replaced by two tests
asserting per-part preservation for both the inline and the top-level
multi-block cases.
…tent blocks Round 4 cleanup audit, applied directly in-PR since responsesSystemBlocks is a helper introduced by this PR. Today ResponsesInputContent is `input_text | input_image | output_text`; the helper already handles all three (image throws; text variants accumulate). A future variant added to the union would have been silently dropped by the prior `if` chain. Replace the silent fall-through with an explicit throw mirroring the `unexpectedResponsesInputItem` exhaustiveness guard already used by `translateResponsesInput`, so any new variant forces an explicit opt-in.
…stem-array skip Post-application review polish. The new exhaustiveness throw in responsesSystemBlocks narrows `block` to ResponsesInputText by the time the .push runs, so the `as ResponsesInputText` cast on the text-extract line is dead. Remove it. Add a unit test asserting the empty-array system-skip behavior that came with the reverse-direction boundary-preservation refactor, so the behavior change is locked in.
…l calling Adds the Cursor agent provider (RunSSE read + BidiAppend write dual-channel HTTP/1.1, Workers-clean) — OAuth poll login, 129-model discovery, streaming chat, composer thinking, image input, and native multi-turn function calling. Provider (packages/provider-cursor): - proto/ codec, AgentTransport dual-channel transport, agent-translate, checksum, OAuth poll/import, access-token cache, models/pricing/quota, interceptors. - Multi-turn native MCP: in-process session reuse (cursor-session-state) keeps the live RunSSE stream paused at exec_mcp; the tool-result follow-up sends ExecMcpResult on the same stream (no prompt-fold). session-id encodes the sessionKey into the tool_call_id so the next request re-finds the session. - Turn end on the authoritative IU[14] turn_ended frame (heartbeat count no longer preempts it); KV blob seqno fix (handleKvMessage's incremented seqno return value was dropped, stalling the blob channel on follow-ups and yielding empty tool-result replies). Platform DurableHttpSession (cross-request HTTP response holder): - contract + Node in-process impl + Cloudflare Durable Object (first DO to own a live outbound socket) + broker + tests + wrangler v2 migration. Gateway/UI wiring: provider registry, cursor routes (authorize-url / poll / reimport / refresh-now), migration 0046, UpstreamCallOptions.apiKeyId, web CursorConfigPanel + provider-meta + upstream pages. pnpm -w typecheck/lint/test green (310 files, 3539 tests). Node e2e verified: turn1 tool_calls -> turn2 uses tool result -> turn3 conversational follow-up.
…ttpSession
Externalize the cross-turn Cursor session state so a tool-result follow-up can
resume on ANY instance (Cloudflare multi-isolate), not just the one that opened
the stream. Replaces the in-process cursor-session-state Map with a unified
path: the RunSSE read stream lives in the DurableHttpSession (addressable by
sessionKey); the scalar write-channel state lives in D1.
Why this works (measured): Cursor only ever set_blob's its conversation
checkpoints (write-only sink) and never get_blob's them on a follow-up, so the
blobStore never needs to travel — only requestId + seqno do.
D1 (Phase 1):
- migration 0047_cursor_sessions {session_key, request_id, append_seqno,
leftover, locked_until, created_at, refreshed_at} + cron sweep (30-min TTL).
- CursorSessionsRepo {claim (CAS single-flight), put, delete, deleteOlderThan}
on Sql + Memory backends; slim cursorSessions on the provider repo contract.
AgentTransport (Phase 2):
- split openChatStream into seed() + open/resume + driveReadLoop; the transport
reads a provided stream (DHS owns the socket) instead of fetching RunSSE.
- getAuthToken injection (replaces a captured token) so a long-session resume
re-mints credentials — fixes a latent stale-token 401.
- capture leftover (RunSSE bytes past the exec_mcp frame) for cross-instance
hand-off; usually empty since cursor pauses after exec_mcp.
fetch.ts (Phase 3):
- performOpen: acquire DHS (RunSSE), send RunRequest via the proxy-aware
fetcher, drive; persist {requestId, seqno, leftover} to D1 at the exec_mcp
pause, release the handle.
- performResume: claim D1, acquire DHS, sendToolResult (exec id+execId decoded
from the echoed tool_call_id — no server-side map), resume; cold-resume
fallback on miss/busy/lost-socket.
- recordUpstreamLatency wraps the acquire/sendToolResult round-trips (the
upstream fetch no longer happens on the provider isolate on CF).
Verified: Node 3-turn e2e (tool_calls -> uses tool result -> conversational
follow-up) through the new DHS+D1 path. pnpm -w typecheck/lint/test green
(311 files, 3553 tests). CF proxy dial + cross-instance e2e are follow-ups.
…proxy dial
The cross-instance resume produced an intermittent empty turn-2 (cursor
received a byte-perfect ExecMcpResult but only heartbeated, never
generating). Root cause was frame loss in the in-process
DurableHttpSession view handoff between turns — a dropped KV(set_blob)
frame went un-acked, wedging cursor's blob channel into an empty
continuation. Two races:
1. release() left the stale per-view dequeue waiter set, so the pump
delivered turn-2's first chunk to the dead view A. Now release()
resolves+clears it so the chunk buffers for the next acquirer.
2. the per-view ReadableStream's default highWaterMark of 1 pulled
ahead speculatively. Set highWaterMark 0 so it pulls 1:1 with reads
(mirrors the old direct-socket read).
CF DurableHttpSessionDO gets the parity fix: release() nulls the
consumer synchronously instead of waiting for the WS 'close' event
(safeSend drops, not buffers, a chunk whose send throws). [CF path
pending Phase 5 wrangler e2e.]
Verified: old in-process flow 6/6 reliable (cursor not flaky); new flow
8/8 + 3-turn 4/4 after fix. Includes Phase 4: DHS dials RunSSE through
@floway-dev/proxy (streaming) on Node + CF, provider resolves the proxy
list per upstream.
typecheck/lint green; cursor 138, platform-node 50, platform-cloudflare
43, gateway 1461 tests pass.
The DO pushed every upstream chunk to the attached consumer WebSocket as it arrived. Cursor emits its post-exec KV frames immediately after exec_mcp (before the ExecMcpResult), while turn-1's transport has already paused at exec_mcp and stopped reading — so those frames were forwarded to turn-1's WS, sat unread, and were lost when it closed on release. Turn-2 re-attached to an empty buffer and cursor stalled (~20% empty turn-2 in local wrangler e2e). Switch the WS body channel to credit-based (pull) flow, mirroring the Node in-process DHS (shared buffer + highWaterMark 0): - DO: deliver a buffered chunk only when the consumer credits one (one inbound WS message = one credit). A paused consumer stops crediting, so post-exec chunks stay in the DO buffer for the next acquire instead of being pushed at a closing WS. flushToConsumer leaves a chunk in the buffer if send throws (never drops); safeSend is retired. - Broker wsToByteStream: highWaterMark 0 + send one CREDIT_BYTE per pull, giving strict 1:1 pull→credit→chunk backpressure. Contract, Node impl, and FakeDurableHttpSession unchanged. Unit tests updated for the credit protocol (platform-cloudflare 44 pass); typecheck + lint green.
Cursor's upstream_success latency measured only the BidiAppend write round-trip (~100ms), hiding the model's queue/think/first-token time — because the RunSSE read of the first chunk was in no recordUpstreamLatency wrap. Add pullToFirstMeaningful to drain cursor's pre-output control frames (checkpoint/heartbeat/token/...) to the model's first real output frame, and wrap that span (plus tool-result send on resume) so upstream_success reflects TTFT. Skipping control frames is output-neutral; the first meaningful frame is handed to buildEvents unchanged, so exec-pause and translation stay there. Contained to provider-cursor; no gateway/platform/frontend/migration change.
flattenMessages dropped every tool-related message: role:'tool' results were skipped and tool_calls-only assistant turns contributed nothing. On a cold-open (live session lost / fresh RunSSE with zero server-side memory) that erased what the model had already called and learned, so it re-ran tools and lost context. Reconstruct the full transcript instead: render each assistant turn's text plus its tool calls (→ called name(args)) and each tool result as [Tool result: <name>]<content>, with the name recovered from the matching assistant tool_call id. Framing is explicit (which tool, args, result) — the degradation the old code avoided came from folding an unaddressed JSON blob, not from tool history itself. Past AND trailing pending rounds fold, since cold-open has no live channel to protect. live-resume (performResume, tool result over BidiAppend) is unchanged. Contained to provider-cursor. Verified: forced cold-open answers from the folded result without re-running the tool (3/3).
Cursor parses KvClientMessage.get_blob_result (field 2) as a GetBlobResult message whose blob_data is field 1. Floway put the raw blob bytes directly in field 2, so any cursor blob fetch (get_blob_args) read an empty/garbage blob and the turn stalled on heartbeats. Verified e2e against a live account: with the wrap the model generates in ~4s instead of hitting the 30s idle timeout. Latent until now because the pre-existing flow never triggered a server-side get_blob_args.
Phase A routed the system prompt to conversation_state.root_prompt_messages_json
(a SHA-256 blob). E2e against a live Cursor account showed the model silently
ignores root_prompt, and the dedicated AgentRunRequest.custom_system_prompt
(field 8) is rejected server-side ('unknown option --system-prompt', grpc-status
3) regardless of client version. Only inlining the system prompt into the
flattened user text reaches the model (verified: 100% obedience). Revert to
inlining; native injection is shelved with the history-turns research. The
privacy toggle and the get_blob_result wrap fix are kept.
…ibution Cursor's RunSSE agent stream reports no per-request token usage, and no usage endpoint is keyed by anything a simulated client controls (usage_uuid isn't surfaced to the agent protocol; GetClientUsageData returns nothing for an agent conversationId; filtered events carry no correlation id) — all verified live. So the cursor usage page was empty. Pull the real account-level usage instead, from the same dashboard RPC the Cursor IDE uses, reachable with the RunSSE Bearer + checksum auth: POST api2.cursor.sh/aiserver.v1.DashboardService/GetFilteredUsageEvents. - data plane: recordUsage now records a bare request row for a non-null but empty usage object; cursor's translator emits an all-zero usage frame on a clean finish so each request is counted (usage_requests) even with no per-request tokens. SourceStreamState.rememberUsage keeps a first empty usage so the request lands, without letting an empty frame wipe a real one. - provider-cursor/usage-sync.ts: paginate GetFilteredUsageEvents, aggregate the account's events into (model, hour) buckets (real input/output/cache tokens + cents). - gateway/cursor-usage-sync.ts: hourly scheduled sweep — per cursor upstream, back-fill completed hours by splitting each account bucket across the Floway keys that made requests in it, proportional to request count (idempotent usage.set). Cost is reproduced from cursor's cents via a synthetic unit price on the largest-token dimension. - UI: a disclaimer on the Cursor upstream settings panel — usage is an account-level approximation that includes non-Floway (IDE) usage and lags. Telemetry is display/export only (no billing/quota), so account-level approximation carries no correctness risk. No migration (reuses usage/ usage_requests).
GetUsableModels surfaces no context window, so every cursor model was hardcoded to 200k — large-context models (claude 300k, gpt/codex 272k, kimi 262k) all showed 200k. Cursor exposes no serialized numeric context field either (proto field 15/16 is never populated on the wire, verified across cli/desktop/web client-types and useModelParameters/ includeLongContextModels request flags), so the only machine-readable source is the '<N>k/M context window' phrase in each AvailableModels tooltipData.markdownContent. fetchCursorCatalog now fetches AvailableModels alongside GetUsableModels (best-effort — any failure falls back to an empty map so the token-minting refresh never breaks) and joins parsed normal-mode context by model name / legacySlug / idAlias. 200k stays as the fallback for models whose tooltip named no window (Auto) or that AvailableModels didn't cover. Max Mode's larger window is deferred.
…endpoint Adds fetchCursorDashboardUsage — a control-plane usage fetcher that hits the same POST cursor.com/api/dashboard/get-current-period-usage the browser Spending tab uses. Auth is the WorkosCursorSessionToken cookie built from the stored userId + minted access token; Origin/Referer/User-Agent are set to browser-shaped values (the endpoint rejects otherwise). Returns typed cents/percent fields; distinguishes session-expired (3xx redirect + 401/403) from other upstream failures so the caller can render a re-import hint rather than logout the operator. Never persisted. The existing rate-limit placeholder (isCursorRateLimited / parseCursorQuotaHeaders) is untouched — it stays a separate concern from the dashboard-usage fetch.
Mirrors the copilot-quota shape: minted access token via ensureCursorAccessToken (reuses cache, handles refresh-race), then fetchCursorDashboardUsage. Maps CursorDashboardSessionExpiredError → 401 with a re-import hint, upstream errors → 502; a dead refresh_token (CursorSessionTerminatedError) also surfaces as 401. Result is not persisted.
Client-side type for the GET /api/upstreams/:id/cursor/quota response body. Mirrors provider-cursor/src/quota.ts's CursorDashboardUsage.
The dashboard's authFetch treats any 401 as our own session expiring and force-clears auth. Mapping a Cursor session-cookie rejection (or a dead refresh_token) to 401 would log the operator out. Return 502 with a machine-readable \`kind: 'session_expired'\` in the body so the panel can still render a re-import hint without triggering the global logout.
Read-only card that shows Cursor's current-cycle spend (Total / Auto+Composer / API) with progress bars and the reset time. Accepts an initialQuota prop so the route-loader can prefetch and avoid a spinner on first paint; a Refresh button re-hits the same endpoint. Session-expired errors are detected by \`kind: 'session_expired'\` in the response body and render a re-import hint that points at the panel's existing Re-import button.
Adds a cursor branch to the [id].vue route loader so the dashboard prefetches GET /api/upstreams/:id/cursor/quota in parallel with the model list (skipped when the account isn't active — the panel guards the same check). The result threads through UpstreamEditPage → UpstreamConfigPanel → CursorConfigPanel as initialQuota / initialQuotaError, so CursorAccountCard renders real numbers on first paint without a spinner flicker (matches copilot's pattern). CursorConfigPanel now hosts CursorAccountCard in place of the inline one-liner account block, and drops the static "notional cost" paragraph — the card shows real spend numbers.
…ain) Pure formatter output from eslint --fix on the cursor-usage-display churn: CursorAccountCard import ordering, prefer-optional-chain on the two upstream lookups, and function-paren-newline on the map callback. No behavior change.
provider-cursor already has its own vitest.config.ts but was never registered in the root config's projects array, so \`pnpm run test\` skipped its 270 tests. Adding it — no other changes.
…ions
The /v1/completions bridge for Cursor Tab was returning
usage: {0, 0, 0} in every response body, so the shared passthrough
usage recorder stored zero-token rows for every Tab request. Cursor's
StreamCpp response carries no upstream usage, and its
aiserver.v1.AiService/CountTokens endpoint — while real — takes ~1s
per call (too slow for autocomplete) and ignores its model_name field
(one fixed BPE tokenizer, probed 2026-07-03).
Estimate statically with per-language byte/token ratios calibrated
against that CountTokens probe: code = 2.55 UTF-8 b/tok,
prose = 5.67 b/tok. Cursor Tab is code-domain by design; the code
ratio is the default. Prompt tokens = the file we sent Cursor
(request.contents); completion tokens = the raw StreamCpp output
text (pre-render), reflecting real upstream compute rather than the
tiny extracted insertion. 0 added latency.
Same ratios apply to all three paths (Zeta V0318, Zeta V0615, FIM);
the shared usageOf helper reads request.languageId (empty on Zeta =
defaults to code). completionsResponseBody now takes a required
CompletionsUsage argument so every response body carries real counts.
The previous edit-mode layout mixed unrelated elements in an ambiguous vertical order — Refresh/Re-import buttons floated as un-carded pills sandwiched between Privacy mode and Max Mode setting toggles, and the three setting toggles used two drifting typography scales (Privacy at xs, Max/Tab at sm). Reorder around three cards with a clear narrative: 1. Account & credentials — identity (email + state) with credential recovery actions (Refresh token / Re-import) bound to it, since those actions fix the identity they sit next to. Re-import's authorize URL now expands inline inside this card so button and response stay adjacent (was at the panel bottom, disconnected). 2. Subscription usage — read-only observation card. Same content as before; drops the redundant identity subcard that duplicated the email now shown in section 1. 3. Settings — three toggle rows (Privacy / Max Mode / Tab completion) in one card with divide-y dividers, all sharing the same text-sm/text-xs scale. Reads as members of one collection rather than three drifting mini-cards. Create mode's first-time authorize flow stays as a bottom-of-panel block, unchanged.
The dashboard's token-usage view shows non-zero numbers for cursor-tab requests thanks to the byte-per-token estimator, but the operator has no way to tell from the panel that those numbers are derived — not sourced from an upstream usage envelope. Someone reading the request against Cursor's own dashboard would notice the mismatch and rightly suspect a bug. Add a one-line clarification under the Tab completion toggle: usage is estimated (with the word 'estimated' emphasised), Cursor's Tab wire returns no per-request usage, counts come from a static byte-per-token ratio, expected accuracy ±10%.
Cursor's Privacy Mode (ghost-mode header) is a training-data guarantee, not a retention guarantee — requests may still be logged transiently. Reword to reflect what Cursor actually promises: "does not use request data for model training" rather than "does not retain request data".
Behavior-preserving cleanup pass over session-authored files: - completions.ts: drop the Phase 2/Phase 3 project-management framing from the file docstring (both phases have landed); drop the misleading one-liner comment on `languageIdForCompletion` that claimed to read a file extension when the body only reads `language`; narrow the param type by removing the unused `suffix?: unknown` field the function never reads. - quota.ts: remove the placeholder `parseCursorQuotaHeaders` function (zero non-test callers, always returned null) and its TODO(cursor)/JSDoc; the shape of a future header parser will be decided when real 429 headers are captured. Drop several redundant JSDoc/rationale comments the function names + signatures already carry, plus the section banner. Trim the file docstring so it no longer describes the removed placeholder as an ongoing concern. Trim `CursorDashboardUpstreamError`'s JSDoc — the prescriptive "surface as 502" hint can't enforce the convention and lives at the caller. - quota_test.ts: drop the placeholder-returns-null test that pinned the removed function. Kept (invariant lens flagged as guarding captured behavior or cross-package mirror contracts): `applyRewrite` inclusive/exclusive capture-anchored comment, `estimateCursorTabTokens` design comment (1s CountTokens latency, model_name ignored), `completionsResponseBody` usage-required warning, `CursorDashboardSessionExpiredError` JSDoc (names the WorkOS 3xx-redirect quirk), and the `CursorDashboardUsage` mirror-of-provider-cursor comment on the SPA type. Passed: pnpm run typecheck + lint + test, plus an independent static verifier that re-checked each change from scratch.
Round 2 cleanup — full `main..HEAD` scope on the cursor-provider branch
(the prior round only covered the tail-end session commits).
Dead code / dead exports removed:
- `proto/agent-messages.ts`: dropped one of two byte-identical
`SelectedImageInput` interface declarations, plus the trivial
`encodeResumeAction()` / `encodeEmptyConversationState()` wrappers
(both return `new Uint8Array(0)` — inlined at their single callers).
- `agent-transport.ts`: dropped `sendMcpResult`, `sendShellResult`,
`sendLsResult`, `sendReadResult`, `sendGrepResult`, `sendWriteResult`,
`sendResumeAction` — zero non-test callers. Floway rejects built-in
tools via `sendRejectedTool` and drives the MCP path through
`sendMcpResultRaw`; the outer wrappers were dead API surface. Narrowed
the proto import list correspondingly.
- `agent-translate.ts`: dropped `composerReasoningRemainder` (test-only
helper — no production caller ever surfaces composer CoT prefixes)
and its test block.
- `access-token-cache.ts`: dropped `getCursorAccessToken`,
`putCursorAccessToken`, `invalidateCursorAccessToken` — the live path
uses `ensureCursorAccessToken`, which self-mints/persists.
- `auth/oauth.ts`: dropped `isTokenExpired` — `access-token-cache.ts`
uses its own `isAccessTokenFresh` with a 5-min skew.
- `constants.ts`: dropped `CURSOR_BIDI_APPEND_PATH`,
`CURSOR_AVAILABLE_CPP_MODELS_PATH`, `CURSOR_GRPC_WEB_CONTENT_TYPE` —
zero non-self refs; the actual consumers (agent-transport,
stream-cpp-transport) hold their own local copies.
- `fetch.ts`: dropped the `isCursorRateLimited` guard and its
`synthetic429` helper — the branch was self-admitted unreachable
("quota parsing is a placeholder … until real 429 headers are
captured"). `isCursorRateLimited` stays exported in `quota.ts` so a
future 429 gate can call it back.
- `index.ts`: dropped a redundant `export type` line — the three
symbols were already exported via `export * from './models.ts'`, and
the removed line mis-declared two runtime functions as `export type`.
- `models.ts`: `...(chat ? { chat } : {})` → `chat`. `buildChatConfig`
always returns a non-null object literal.
- `apps/web/src/lib/pkce.ts`: narrowed the provider union — Cursor auth
is poll-based, not PKCE, so `'cursor'` was a phantom.
Duplicates collapsed:
- Three copies of `applyRewriteToFile` (byte-identical to
`applyRewrite` in `completions.ts`) collapsed into a single import
from `completions.ts` in `zeta-format.ts` and `zeta-v0615.ts`.
- `commonPrefixLen` / `commonSuffixLen` (byte-identical in both zeta
files) exported once from `zeta-format.ts`, imported in `zeta-v0615.ts`.
Wrapper collapsed:
- `buildCursorAuthorizeUrl` (a one-line rename around
`generateCursorAuthParams`) removed; its sole caller
(control-plane/upstreams/routes.ts) now imports
`generateCursorAuthParams` directly.
Test fixtures deduplicated:
- Eight test files' inlined `cursorSessions: { claim: async () => null,
put: async () => {}, delete: async () => {} }` replaced with
`cursorSessions: noopCursorSessionsRepo()` from
`@floway-dev/test-utils`, so a future addition to
`CursorSessionsRepoSlim` doesn't drift eight scattered stubs.
Stale / misleading / triplicated comments fixed:
- `auth/import.ts`: dropped "Step 7" planning-phase reference from the
file docstring; converted JSDoc to line comments to match sibling
files.
- `interceptors/chat-completions/index.ts`: reduced a
self-contradictory two-sentence comment ("order matters" ↔ "order is
positional") to just the invariant.
- `CursorAccountCard.vue`: dropped the "legacy state rows that predate
CursorCredentialHealth" line — Cursor is greenfield on this branch,
no legacy exists.
- `UpstreamEditPage.vue`: fixed the misleading "only editable config
field" phrasing (maxMode and tabCompletion.enabled are also editable,
inline-PATCHed from CursorConfigPanel).
- Deduped the triplicated "Cursor login is poll-based" prose across
`control-plane/upstreams/routes.ts`, `control-plane/schemas.ts`,
`CursorConfigPanel.vue` — kept the fullest section as canonical and
shrank the other two to a pointer.
- Trimmed `CursorDashboardUpstreamError`'s JSDoc — a prescriptive
"surface as 502" hint can't enforce the convention.
Passed: pnpm run typecheck + lint + test, plus an independent static
verifier that re-checked each change from scratch. Verifier's only
remaining note: `encodeConversationActionWithResume` and
`encodeAgentClientMessageWithConversationAction` are now unused (their
sole caller `sendResumeAction` was removed) — picked up in the next
round.
- Delete unreachable quotaSnapshot placeholder (CursorQuotaSnapshot, isCursorRateLimited, CursorQuotaSnapshotEntry, ALLOWED_QUOTA_SNAPSHOT_KEYS_MAP, assertCursorQuotaSnapshotEntry, per-account quotaSnapshot field, the unreachable branch in fetch.ts, and the gateway serializer/dashboard slot). Nothing populates it and the dashboard doesn't read it; the gate can be wired when real 429 headers are captured. - Delete unused access-token-cache exports getCursorAccessToken / putCursorAccessToken / invalidateCursorAccessToken (no production callers) and inline ensureCursorAccessTokenInner into ensureCursorAccessToken via a default parameter. - Delete duplicate SelectedImageInput interface declaration in proto/agent-messages.ts. - Drop agent-transport.ts local RUN_SSE_PATH / BIDI_APPEND_PATH / USER_AGENT / GRPC_WEB_PROTO in favor of the CURSOR_* re-exports from constants.ts so the CLI-version bump touches one file.
Cascade from Round 2's AgentTransport.send*Result deletion:
- `proto/agent-messages.ts`: remove `encodeConversationActionWithResume`
and `encodeAgentClientMessageWithConversationAction` — their sole
caller (`sendResumeAction`) is gone. Drop matching entries from the
`proto/index.ts` barrel.
- `proto/exec.ts`: the five `buildExecClientMessageWith{Shell,Ls,Read,
Grep,Write}Result` wrappers still have exactly one caller each —
the still-live `buildExecClientMessageWithRejectedTool` inside the
same file, which builds a wire-shaped reject payload per tool type.
Verified via grep before assuming otherwise (a Round-3 challenger
agent caught this cascade error before we could over-delete).
Downgrade the five from `export function` to file-private
`function` and drop them from the `proto/index.ts` barrel — no
external importer relies on them.
Passed pnpm run typecheck + lint + test.
…tighter varint - Add shared module-scope TEXT_ENCODER (proto/encoding.ts), TEXT_DECODER + TEXT_DECODER_FATAL (proto/decoding.ts); reuse across proto/*.ts, agent-transport.ts, stream-cpp-transport.ts, checksum.ts, auth/poll.ts, completions.ts. Roughly 40 per-call allocations removed on the streaming hot path (one text_delta frame previously allocated N decoders). - parseProtoFields now returns .subarray(...) views instead of .slice(...) for wire types 1/2/5. Frame payloads are owned copies from readConnectFrame and no caller mutates them, so the fresh alloc + memcpy per field was pure overhead. Callers that need ownership already slice explicitly. - encodeVarint uses a fixed 10-byte scratch instead of pushing into a number[] and re-materializing as Uint8Array.
…-call chunks - Extract wrapExecClient(id, execId, innerField, innerBytes): the 7 buildExecClientMessageWith*Result functions all built the same 3-part envelope (id + optional exec_id + inner message on tool-specific field). - Extract strField(fields, num) helper for wire-type-2 decoders in parseShellArgs/parseLsArgs/parseReadArgs/parseGrepArgs/parseWriteArgs/ parseMcpArgs + parseExecServerMessage's exec_id read. - Consolidate synthetic 405/503 into one syntheticErrorResponse(status, type, message) factory in fetch.ts; provider.ts's three unsupported* handlers now derive their shared 405 body from it. - Extract toolCallChunk() so agent-transport's driveReadLoop yields tool_call_started/tool_call_completed via one call instead of two identical copy-paste blocks that differ only by the chunk type.
- Zeta rewrite splice: delete the two local applyRewriteToFile copies in zeta-format.ts and zeta-v0615.ts and import applyRewrite from completions.ts. - commonPrefixLen / commonSuffixLen: export from zeta-format.ts and import into zeta-v0615.ts instead of a byte-identical copy. - Iterate messages backward in-place in fetch.ts's last-user-turn lookup instead of [...messages].reverse().find (dropped a full array clone per open()).
Read loops in agent-transport.ts and stream-cpp-transport.ts were doing two allocations per reader.read(): one for the full-length concat (buffer + value), another for the trailing slice of the still-unparsed remainder. Track the parse position across iterations and fuse both into a single alloc sized exactly for tail + value — with a zero-copy fast path when the prior buffer was fully consumed. Turns cumulative O(N*M) buffer growth into O(M) for a stream that parses cleanly on chunk boundaries. DurableHttpSessionDO (Cloudflare worker) had two queue.shift() sites that were O(n) on the chunk backlog — one in the DO itself (buffer between consumers), one in wsToByteStream's pull queue. Replace both with a head-index cursor and reclaim the underlying array when the head walks past the tail. pendingAssistantBlobs guard (E10 / E14) is intentionally deferred — the finding trades the "blob exceeded stream" warn against the analyzeBlobData cost, and dropping the warn is a behavior change; leaving as-is until the warning proves noisy in production.
Shared plumbing (gateway respond.ts, packages/platform durable-http-session contract, node in-process impl, packages/provider apiKeyId, gateway repo resolveDirectDialProxies) narrated its invariants around the one caller that happened to need them today. Rewrite each comment to state the invariant abstractly — an empty-usage frame is a general "count-the-request" signal, proxy-side dial is a general "streaming upstream" concern, apiKey scoping is a general per-(upstream, apiKey) concern — so a future stateful provider can adopt the same infra without a comment-level bulge. Code behavior is unchanged.
Bring in the 7 cursor-scoped simplify commits (59a6e66..7654bb0): - drop dead scaffolding and duplicate declarations - perf: hoist TextEncoder/TextDecoder, proto slice→subarray, tighter varint - refactor: dedup exec builders/parsers, synthetic errors, tool-call chunks - refactor: unify Tab-splice helpers + drop message-list clone - perf: fuse buffer concat/tail-slice + head-index DO chunk queues - docs: trim cursor-specific reveals from shared contract/impl comments - style: eslint import-order autofix over the simplify diff Conflict resolution (3 files, comment/wording only): - fetch.ts: take simplify's synthetic503 wrapper on syntheticErrorResponse - zeta-format.ts, zeta-v0615.ts: take simplify's more descriptive helper comments referencing FIM/V0318 shared origin
Cherry-pick the 3 cursor + cursor-infra fixes from code-review/cursor-provider
(b6ef3eea), skipping the 2 model-aliases fixes which are unrelated to the
cursor-provider branch and will be sent in a separate branch:
- provider-cursor: `replaceActiveAccount` now spreads state, preserving the
top-level `modelContext` map (observed context windows) across refresh-token
rotations and terminal-state transitions.
- provider-cursor: `parseToolCall` nested-wrapper heuristic now length-checks
the reparse so a scalar string starting with `\n` (0x0a is byte-identical
with `(field 1 << 3) | 2` plus a plausible varint) is no longer
indistinguishable from a wrapped `{ field 1 = string }`.
- platform-cloudflare durable-http-session-do: idle-alarm now discards
regardless of consumer attachment, matching the Node in-process counterpart.
Conflict on tool-calls.ts resolved by keeping simplify's hoisted TEXT_DECODER
alongside the length-checked reparse.
…T_TYPE Merge collision: Round-2 cleanup (93018b4) removed these two constants because their sole consumers held local copies. Simplify's later dedup pass (49b78f4) centralized on constants.ts as the single source and moved the inline copies out of agent-transport.ts. Post-merge, agent-transport.ts imports the two constants but they no longer exist — restore them so the now-centralized shape typechecks. CURSOR_AVAILABLE_CPP_MODELS_PATH stays deleted (no consumers post-merge).
The reparse guard from b6ef3eea checked `nested[0].value.length` against `tf.value.length` — but parseProtoFields silently truncates when the declared length overflows, so its returned .value.length is always (total - 1 - varintBytes). The equation 1 + varintBytes + innerLen === total therefore holds for BOTH a real wrapper and a false-positive raw string starting with \n, defeating the guard. Read the declared inner length directly via decodeVarint and compare that against the buffer size — parseProtoFields is no longer needed on this path. Extracted from the code-review branch's proto_test.ts: two regression tests (shell command with LF prefix, edit oldString with LF prefix) now pass.
Simplify's 59a6e66 dropped the always-null `quotaSnapshot` field from the CursorAccountCredential schema AND the state assertion's allowed-key allowlist. But every cursor upstream imported before the cleanup carries `quotaSnapshot: null` on each account — the original import path (auth/import.ts in f9ebb7f) wrote it unconditionally. Post-cleanup, assertCursorUpstreamState throws "unexpected key 'quotaSnapshot'" and every cursor request fails with a 500. Add 0050_cursor_drop_quotasnapshot.sql to scrub the key on the same `pnpm run db:migrate:remote` step that ships the code, so no cursor upstream enters a broken window. Idempotent (skipped when the key is already absent).
Menci
reviewed
Jul 4, 2026
| '$.accounts[0]', | ||
| json_remove(json_extract(state_json, '$.accounts[0]'), '$.quotaSnapshot') | ||
| ) | ||
| WHERE provider = 'cursor' |
Owner
There was a problem hiding this comment.
You should NEVER UPDATE a freshly CREATE-d table (with ABSOLUTELY NO DATA) immediately...
Menci
reviewed
Jul 4, 2026
| recordUpstreamLatency: recorder.record, | ||
| waitUntil: ctx.backgroundScheduler, | ||
| headers: inboundHeadersForUpstream(c), | ||
| apiKeyId: ctx.apiKeyId, |
Owner
There was a problem hiding this comment.
Land these prequirements changes separately before this PR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add Cursor as a 7th provider kind (alongside
copilot/custom/azure/codex/claude-code/ollama) using Cursor's own CLI OAuth flow, and expose two data-plane surfaces:RunSSE+BidiAppenddual channel — multi-turn native MCP tool calling, image input, reasoning-effort / Max-Mode routing, and privacy (ghost) mode./v1/completionsbridge exposing Cursor Tab (StreamCppon the geo-routedus-only.gcpp.cursor.shbackend) as an OpenAI-shaped edit-prediction endpoint. Zed'sopen_ai_compatible_apiprompt formats — Zeta 2.1 (V0318) markers, V0615 hashed-regions (cross-file jumps), plus FIM and plain-prefix fallback — are all detected from the prompt and rendered back.The interesting review surface is not the provider code — that is a mechanical implementation of Cursor's protobuf protocol whose shape is dictated by Cursor's own CLI. The interesting review surface is the new runtime stack that had to land underneath the provider to make it work on Workers at all.
Infrastructure:
DurableHttpSession— a new platform primitiveCursor's agent stream is not stateless. A turn opens a
RunSSEserver-stream on which the model producesexec_requestframes whose results the client is expected to write back on a companionBidiAppendchannel — on the same conversation. A gateway that handles the tool-result follow-up on a different HTTP request must keep the outboundRunSSEresponse socket alive across the two request lifecycles and route the follow-up to whichever isolate holds that socket. Every existing Floway provider is one-request-in / one-response-out and does not have this problem.Solution:
packages/platform/src/durable-http-session.ts— a new abstract runtime contract with two symmetric operations,acquire(key, init?)andrelease()/discard().initpresent = open (dial the upstream, hold the response stream).init === null= resume attempt (return the held stream to whoever wins the single-flight claim). The gateway core reaches this contract viagetDurableHttpSession()and stays runtime-neutral.Two concrete impls ship in this PR:
apps/platform-cloudflare/src/durable-http-session-do.ts— a Cloudflare Durable Object,DurableHttpSessionDO. This is Floway's first DO that owns a live outbound socket. The DO stays alive because the outbound connection keeps it alive (Cloudflare's 2026-06-19 outbound-connections-keep-DOs-alive runtime behavior — bounded by CF's 15-minute cap). An idle alarm viactx.storage.setAlarm(default 5 min) evicts abandoned sessions before then; the alarm evicts regardless of consumer attachment (matches the node impl's behavior). Chunk queues use a head-index queue with a fused concat/tail-slice so the DO does no per-chunk allocation on hot turns.apps/platform-node/src/in-process-durable-http-session.ts— a single-process in-memory impl forpnpm run dev:node. Same contract; no cross-instance semantics.The single-flight claim (so a concurrent follow-up cannot race the resume) is a D1-backed repo,
cursorSessionsin the gateway repo composition, keyed by a session id derived from(upstreamId, apiKeyId, headers, messages)(seesession-id.ts). On resume,claim()atomically locks the row forCLAIM_TTL_MS = 60_000; a miss / busy / lost-socket returns null, and the caller cold-resumes with the full transcript flattened into a fresh open (Cursor has no server-side memory of the prior turn after socket loss).New
wrangler.example.jsoncsurface driven by this stack:DURABLE_HTTP_SESSION→ classDurableHttpSessionDO.v2(new_sqlite_classes: ["DurableHttpSessionDO"]). Append-only.compatibility_flags: ["nodejs_compat"]— required because@reclaimprotocol/tls/webcrypto(used bypackages/http's userspace TLS dialer) stillimport { webcrypto } from 'crypto'.scripts/check-wrangler.tsnow enforceswrangler.jsoncmirroring the example bidirectionally, withaccount_idas the one allowlisted per-contributor exception. This is a real behavior change to the deploy gate — every value the example pins must appear in the real config, and vice versa.New D1 migrations:
0048_cursor_provider.sql— adds'cursor'to theupstreams.providerCHECK (rebuild + copy per repo pattern).0049_cursor_sessions.sql— newcursor_sessionstable for the resume single-flight.0050_cursor_drop_quotasnapshot.sql— data migration. The initial import path wrotequotaSnapshot: nullon every cursor account row; a later behavior-preserving cleanup dropped the field from the state schema and from the strict key allowlist. Without this migration, every legacy cursor row would failassertCursorUpstreamStatewithunexpected key 'quotaSnapshot'and return 500. Idempotent.Transport choice: HTTP/1.1 on
api2.cursor.sh(not HTTP/2 onapi5.*)Cursor's control-plane exposes two backend families with different HTTP versions on the wire.
api2.cursor.shservesRunSSE+BidiAppendover HTTP/1 — both HTTP/1.1 and HTTP/1.0 are accepted; Floway speaks 1.1. Theapi5family (agent.api5.cursor.shprivacy /agentn.api5.cursor.shnon-privacy) is HTTP/2-only — a forced HTTP/1.1 handshake returnsReceived HTTP/0.9; the default ALPN negotiates h2 and returns 200. Floway targetsapi2.cursor.shover HTTP/1.1 on purpose:fetchis HTTP/1.1-native and interops with Cloudflare's outbound-connections-keep-DOs-alive lifetime story. Switching to HTTP/2 would mean introducing an h2 library into the transport and re-proving DO socket lifetime under it.@floway-dev/proxyruns upstreams over operator-configured proxy chains (userspace TLS + WebSocket-tunnel over TCP, VLESS,runProxiedRequest). Every dialer speaks a raw duplex byte stream and layers HTTP/1.1 on top. HTTP/2 would require multiplexing support inside every dialer, or bypassing the proxy layer entirely (which we won't do — proxy-through-Workers is a first-class feature).api2is a header (x-ghost-mode: true|false), so we can honor it without leaving the HTTP/1.1 endpoint. Domain-based privacy on theapi5family is the only reason to consider h2 for now, and it doesn't buy anything the header path doesn't already give us.The Cursor Tab bridge dials
us-only.gcpp.cursor.sh— a separate geo-routed backend, HTTP/1.1-compatible, Connect-proto server-stream (not the RunSSE dual channel). Same transport story.How Floway's cursor provider differs from other cursor gateways
Several open-source projects have already reverse-engineered pieces of Cursor's protobuf protocol; we cross-checked wire shapes and behavior traps against them while building this provider. Floway-specific choices worth calling out:
DurableHttpSessionhad to be invented.fetchwith no proxy layer; the h2/h1 choice is invisible to them. For us it is load-bearing.UserMessage.text, emptyturns[]. Nativeturnsreference blobs the server can't fetch on a cold conversation ("Blob not found"). Prior tool rounds are framed explicitly ([Assistant] → called foo(...),[Tool result: foo] …) so the model reads them as history without needing a blob dereference.[System].AgentRunRequest.custom_system_promptis rejected by the cloud agent (unknown option '--system-prompt');conversation_state.root_prompt_messages_json(SHA-256 blob) is fetched by the server but silently ignored by the model. Inlined system text is the only method the account e2e-obeys.SelectedImage.data(field 8), notblob_id. Same trap: pureblob_idneeds a KV blob handshake the gateway does not do; inline bytes sidestep it. Prior art either drops images entirely or substitutes a text placeholder.GetBlobResultshape.KvClientMessage.get_blob_result(field 2) is a message whose field 1 carries the bytes. Placing raw bytes directly in field 2 makes Cursor read an empty blob and stall on heartbeats until the ~30 s idle timeout. Fixed inpackages/provider-cursor/src/proto/kv.ts; this is a real latent bug in some prior implementations.StreamCpp) → Zed edit-prediction. No cursor gateway we're aware of implements this. The bridge parses Zed's marker-delimited prompt (Zeta 2.1 V0318 / V0615 hashed-regions / FIM / plain), builds a StreamCpp request, and renders the response as<|marker_1|>...<|user_cursor|>...<|marker_K|><[end▁of▁sentence]>.CountTokensRPC. The liveCountTokensRPC ignores itsmodel_nameargument and adds ~1 s latency — untenable for a Tab request that must return in tens of ms. Floway uses a corpus-calibrated chars-per-token estimator with a per-model ratio.ide,CURSOR_TAB_CLIENT_VERSION). The Tab backend gates on the IDE-style client version; the CLI version string yields empty completions.AvailableModelstooltip prose (the only pre-observation source), then overlays it with the live observedConversationTokenDetails.maxTokensfrom the RunSSE stream (cached per(modelId, maxMode)with TTL). Prior implementations either hardcode context windows or read them off the tooltip only.Per-request token accounting: read this before you look at the graphs
A turn that ends on a tool call reports zero tokens. Real usage only lands on the last turn of the whole conversation. This is not a bug — it is the only shape the Cursor upstream gives us — and operators watching per-request usage graphs need to know it up front.
Cursor's cumulative token total is delivered as a
ConversationTokenDetailscheckpoint frame. That checkpoint only arrives at the very end of a run, after the model's final answer — i.e., after the last non-tool assistant turn. A turn that pauses at anexec_request(any turn that emits a tool call) returns to the gateway before its checkpoint has been written to the stream, so:usage: { prompt: 0, completion: 0, total: 0 }— Floway records the request row so the call is still counted, but every dimension is zero.usagecarries the entire run's cumulative token totals (total = usedTokens,completion = Σ TokenDeltaUpdate.tokensclamped tototal,prompt = total − completion).In per-request usage graphs this reads as: N−1 zero-usage rows followed by one row that carries all the tokens for the whole conversation. Aggregate totals stay correct; per-request medians and percentiles do not, and cannot, without Cursor exposing per-turn totals it currently does not.
Two data-plane changes in this branch back this shape:
agent-translate.ts(finalize path) — computes usage as above and always emits a trailing usage frame, zeros included, so every turn produces a countable request row.packages/gateway/src/data-plane/chat/shared/respond.ts—rememberUsagenow keeps the first zero-usage frame instead of dropping it, andrecordUsagewrites a bare request row on a non-null-but-empty usage object. Without both, the N−1 tool-round rows would be silently invisible to the request counter. (The docstring onSourceStreamState.rememberUsageexplains the invariant for future providers with the same shape.)Non-agent Cursor traffic (no tools advertised) hits only the final-turn path and looks normal in the graphs.
Cursor Tab (
/v1/completions) is separate — and its numbers are estimated, not measured. TheStreamCppbackend does not return a usage frame at all, and the liveCountTokensRPC ignores itsmodel_nameargument and adds ~1 s latency (untenable for a Tab request that has to return in tens of ms). Floway substitutes a corpus-calibrated chars-per-token estimator with a per-model ratio and records the estimate on the request row. Aggregate Cursor Tab token counts in the graphs are indicative, not authoritative — treat them as sizing signal, not billing truth.Upgrading an existing deployment
Anyone who already runs Floway on Cloudflare Workers must edit their local
wrangler.jsoncbeforepnpm run deploywill publish this branch — the deploy gate refuses to publish otherwise. The gate isscripts/check-wrangler.ts; when the config drifts it lists every missing entry by path, so the mechanical fix is straightforward, but it does need doing:DURABLE_HTTP_SESSION→ classDurableHttpSessionDOunderdurable_objects.bindings(alongside the existingBROADCAST_DO).migrations:{ "tag": "v2", "new_sqlite_classes": ["DurableHttpSessionDO"] }(DO migrations are append-only; do not modify the existingv1)."compatibility_flags": ["nodejs_compat"]at the top level (required by@reclaimprotocol/tls/webcrypto'simport { webcrypto } from 'crypto').wrangler.example.jsoncin this PR — the deploy gate now enforces bidirectional mirroring, so any real-config key the example doesn't pin (aside fromaccount_id) will also fail.D1 migrations
0048–0050apply automatically as part ofpnpm run db:migrate:remote; no manual step there.Test plan
pnpm run typecheck— greenpnpm run lint— greenpnpm run test— 4 136 tests, 0 failedwrangler dev --localboot +/api/health,/v1/modelsPOST /v1/chat/completionsstreaming against a real Cursor account (claude-sonnet-5) — end-to-end response receivedPOST /v1/completionscursor-tabplain FIM — real completion receivedPOST /v1/completionscursor-tabZeta 2.1 (V0318) marker prompt — output rendered as<|marker_1|>...<|user_cursor|>...<|marker_2|><[end▁of▁sentence]>pnpm run db:migrate:remoteon remote D1 (0048, 0049, 0050)pnpm run deploy— first-time deploys must add the newDURABLE_HTTP_SESSIONbinding + v2 DO migration +nodejs_compatto theirwrangler.jsonc