fix: handle Azure filter stream chunks#1982
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1a5b96f38e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1a5b96f to
1762e8f
Compare
jbeckwith-oai
left a comment
There was a problem hiding this comment.
Reviewed against the original Azure async-filter report in #1015 and the documented Azure annotation stream shape.
Validation performed:
- Reproduced the pre-fix failure at base
95b54e58: the proxiedfromReadableStream()sequence rejects withCannot read properties of undefined (reading 'content')when an empty-ID annotation omitsdelta. - Ran the same sequence at
1762e8fa: it completes successfully and preservesid,created,model, content, finish reason, and choice-level filter annotations. - Exercised Azure's documented leading prompt-annotation chunk, interleaved/trailing completion annotations, two proxied completions separated by an empty-ID annotation, and the direct
createChatCompletion()path. All passed; annotation fields remained attached to the intended choice/completion and completion boundaries remained correct. jest tests/lib/ChatCompletionStream.test.ts --runInBand: 6/6 tests and 4/4 snapshots passed.git diff --check: clean.- GitHub checks are green for build, lint, tests on Node 20/22/24/26, ecosystem tests, breaking-change detection, CodeQL, and Agents SDK regressions; OkTest reports 237/237 passed.
The changes are defensive for an Azure wire-format divergence without changing the public TypeScript API. Non-empty IDs retain existing completion-boundary and metadata behavior; empty-ID annotation chunks now preserve active completion identity while still merging choice-level moderation data. I did not find a blocking or nonblocking defect.
jbeckwith-oai
left a comment
There was a problem hiding this comment.
LGTM - the rename/destructure did make it a tad annoying to review tho
Summary
deltaChatCompletionStreamingRunner.fromReadableStream()Why
Azure's asynchronous content filter can interleave annotation-only chunks with normal chat completion chunks. Those annotations may omit
deltaand use an emptyid, which caused the streaming runner to dereferencechoice.deltaand could also make it treat the annotation as a new completion.Impact
Applications proxying Azure chat completion streams through
fromReadableStream()no longer crash or finalize early when async filter annotations arrive. Normal OpenAI and Azure streaming behavior is unchanged.Validation
pnpm exec jest tests/lib/ChatCompletionStream.test.ts --runInBandpnpm exec tsc --noEmitpnpm run lintgit diff --check