fix(api): restore SSE Content-Type for streaming responses#82
fix(api): restore SSE Content-Type for streaming responses#82
Conversation
All three streaming endpoints (/v1/chat/completions, /v1/responses,
/v1/messages) were returning Content-Type: text/plain instead of
text/event-stream, breaking strict SSE clients (browser EventSource,
EvalScope perf benchmark, any RFC 8941 implementation). Bodies were
correctly SSE-formatted but the header lied.
Root cause: when deps were bumped from elysia 1.3 → 1.4, the default
Content-Type for async-generator returns silently changed from
text/event-stream to text/plain. In 1.4 the auto SSE path triggers
only when set.headers["content-type"] (lowercase) starts with
"text/event-stream", and it then wraps every yielded chunk with
"data: ${chunk}\n\n" — but our adapters already emit pre-formatted
SSE strings, so opting in to the auto path would double-prefix data:.
Setting set.headers["Content-Type"] (mixed case) leaves both keys
live and Bun joins them as "text/event-stream, text/plain".
Fix: return a native Response wrapping a ReadableStream. Elysia's
mapResponse detects a pre-formatted Response with chunked transfer
encoding and runs handleStream with skipFormat=true, which preserves
our headers and bypasses the auto SSE wrapping
(node_modules/elysia/dist/adapter/utils.mjs:209-215).
Also wire Accept: text/event-stream content negotiation: when the
request body does not specify `stream`, an Accept header listing
text/event-stream now enables streaming. Body always wins when
explicit, so existing OpenAI-compatible behavior is preserved.
Verified with curl against a live server:
- stream:true → Content-Type: text/event-stream; charset=utf-8
- Accept: text/event-stream (no body.stream) → streaming
- stream:false + Accept: text/event-stream → JSON (body wins)
- non-streaming (default) → application/json (regression check)
Closes #81
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✅ Files skipped from review due to trivial changes (2)
📝 WalkthroughWalkthrough三个聊天/响应端点(completions、messages、responses)改为在未明确设置 ChangesSSE 流式响应改造
Sequence Diagram(s)sequenceDiagram
participant Client
participant ServerEndpoint
participant StreamProcessor
participant UpstreamModel
Client->>ServerEndpoint: POST /v1/... (Accept: text/event-stream)
ServerEndpoint->>ServerEndpoint: calls acceptsEventStream(headers)
ServerEndpoint-->>Client: if streaming -> Response(ReadableStream) with SSE headers
ServerEndpoint->>StreamProcessor: start processing (processStreamingResponse)
StreamProcessor->>UpstreamModel: request generation (streaming)
UpstreamModel-->>StreamProcessor: yields chunks
StreamProcessor-->>ServerEndpoint: encodes SSE chunks (enqueue)
ServerEndpoint-->>Client: SSE chunks over ReadableStream
UpstreamModel-->>StreamProcessor: final/done
StreamProcessor-->>Client: enqueue "data: [DONE]" (then close)
Note over StreamProcessor,ServerEndpoint: on error -> enqueue SSE "event: error" if not aborted
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Review rate limit: 4/5 reviews remaining, refill in 12 minutes. Comment |
There was a problem hiding this comment.
Code Review
This pull request implements content negotiation for Server-Sent Events (SSE) by checking the Accept header when the stream parameter is omitted. It also refactors the streaming logic in the completions, messages, and responses APIs to return a native Response wrapping a ReadableStream, ensuring proper SSE headers and avoiding framework-level double-prefixing. Feedback was provided regarding the acceptsEventStream utility, which currently does not account for the quality factor (q=0) in the Accept header.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/src/utils/api-helpers.ts`:
- Around line 54-61: The acceptsEventStream function incorrectly treats values
like "text/event-stream;q=0" as acceptable and also mis-matches types via
startsWith; update acceptsEventStream(headers: Headers) to parse each Accept
header entry into its media-type and parameters, ignore entries with q=0, and
only accept exact media type "text/event-stream" (ignore structured suffixes
like "+json"); specifically, split by commas, for each part trim and split on
";" to extract the mime-type and params, parse q parameter (default >0), and
return true only when mime-type === "text/event-stream" and q > 0 so negotiation
won’t incorrectly select SSE when body.stream is not set.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: 56e2487e-88d6-4da6-a1f2-cb63ff4fbe1b
📒 Files selected for processing (4)
backend/src/api/v1/completions.tsbackend/src/api/v1/messages.tsbackend/src/api/v1/responses.tsbackend/src/utils/api-helpers.ts
Round 1 review (#82): both gemini-code-assist and CodeRabbit flagged the same issue in `acceptsEventStream`. 1. Quality factor q=0 was not respected. `Accept: text/event-stream;q=0` means "do not accept SSE" per RFC 7231 §5.3.1, but the prior helper returned true and would incorrectly opt the client into streaming. Now parse the q parameter and only accept entries with q > 0 (default 1 when absent). 2. Structured-suffix media types were silently matched. `startsWith("text/event-stream")` would also accept `text/event-stream+json` etc. Now match the media type exactly after lower-casing and trimming. Verified with the same inputs both bots flagged: - text/event-stream;q=0 → false (was true) - text/event-stream;q=0.0 → false (was true) - text/event-stream+json → false (was true) - text/event-stream;q=0, application/json → false (was true) Existing positive cases (plain SSE, weighted lists with q>0, case/whitespace variants) still return true. Tests: api-helpers.test.ts adds 12 cases covering each branch.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/src/utils/api-helpers.ts`:
- Around line 67-72: The current q parsing uses params.find((p) =>
p.startsWith("q=")) which misses spaced forms like "q = 0" and thus treats them
as absent; update the logic in the function handling q parsing to trim each
param and match q with optional whitespace using a regex or split: e.g. find a
param where p.trim().toLowerCase().startsWith("q") and then extract the value
with p.trim().match(/^q\s*=\s*(.+)$/i) (or split on '=' after trim), convert to
Number, and return Number.isFinite(q) && q > 0; add a unit test covering
"text/event-stream ; q = 0" and "q=0" and valid positive q values to prevent
regression.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: 65bf70c4-e611-4930-ab38-d23e483372c1
📒 Files selected for processing (2)
backend/src/utils/api-helpers.test.tsbackend/src/utils/api-helpers.ts
Round 2 review (#82): CodeRabbit flagged that the round-1 q parsing used `param.startsWith("q=")` which only matches the compact form. Per RFC 7230 §3.2.6, optional whitespace is permitted around the `=` in `parameter = token "=" ( token / quoted-string )`, so a strict client may send `Accept: text/event-stream ; q = 0`. The prior fix treated such forms as "no q parameter present" and returned true, incorrectly opting the client into streaming. Replaced startsWith check with an indexOf("=") + slice + trim pass: parse name and value separately so whitespace around the delimiter no longer hides q. Empty `q=` value now correctly resolves to false (Number("") = 0, fails q > 0). Verified before/after: - text/event-stream ; q = 0 → was true, now false - text/event-stream ; q = 0.5 → was true (default-q path), now true (real q parsed) - text/event-stream;q=0 → still false (round-1 case) - text/event-stream;q= → was true (empty slice), now false Tests: api-helpers.test.ts +3 cases covering whitespace q=0, whitespace q=0.5, and empty q value. 12 → 15 cases.
Summary
/v1/chat/completions,/v1/responses,/v1/messages) were returningContent-Type: text/plaininstead oftext/event-stream. Bodies were correctly SSE-formatted but the header lied, breaking strict SSE clients (browserEventSource, EvalScope perf benchmark, any RFC 8941 implementation).Responsewrapping aReadableStream. Elysia detects pre-formatted Responses and runshandleStreamwithskipFormat=true, preserving our headers and bypassing 1.4's auto SSE wrapping.Accept: text/event-streamcontent negotiation: whenbody.streamis unset, anAcceptheader listingtext/event-streamenables streaming. Body always wins when explicit.Root cause
When deps were bumped from
elysia ^1.3.5to^1.4.16(d1f186a), the defaultContent-Typefor async-generator returns silently changed fromtext/event-streamtotext/plain. In 1.4 the auto SSE path kicks in only whenset.headers[\"content-type\"](lowercase) starts withtext/event-stream, and it then wraps every yielded chunk withdata: \${chunk}\n\n. Our adapters already emit pre-formatted SSE strings, so opting in to that auto path would double-prefixdata:. Settingset.headers[\"Content-Type\"](mixed case) leaves both keys live and Bun joins them astext/event-stream, text/plain. Returning a nativeResponseis the only path in 1.4 that lets us set the header without triggering double-wrapping.Verified against three historical versions:
stream:trueContent-Typeeebe5a2^1.3.5text/event-stream✓3223b76^1.4.16text/plain❌main(this PR)^1.4.22text/event-stream; charset=utf-8✓Test plan
curl -D -against/v1/chat/completionswithstream:true→Content-Type: text/event-stream; charset=utf-8, body is valid SSE ending indata: [DONE]\n\n/v1/messages(Anthropic) → ends withevent: message_stop/v1/responses(OpenAI Responses API) → events stream correctlyContent-Type: application/json(regression check)Accept: text/event-stream(nobody.stream) → streamingAccept: text/event-stream+stream:false→ JSON (body wins)text/event-stream;q=1, application/json;q=0.5) → streamingbun run lintandbun run checkpassCloses #81
Summary by CodeRabbit
发行说明