Skip to content

fix(sdk): stream Python responses without 4 KB buffering - #1296

Draft
dewanshshekhar wants to merge 1 commit into
RightNow-AI:mainfrom
dewanshshekhar:fix/python-sdk-streaming
Draft

dewanshshekhar wants to merge 1 commit into
RightNow-AI:mainfrom
dewanshshekhar:fix/python-sdk-streaming

Conversation

@dewanshshekhar

Copy link
Copy Markdown

Summary

The Python SDK waits for 4096 bytes (or EOF) before delivering small SSE responses. The same fixed-size read can split a UTF-8 character and raise UnicodeDecodeError during an otherwise valid response. Exiting on [DONE], closing the iterator early, or encountering a decode error also leaves the HTTP response open.

Changes

  • Iterate over complete response lines so events arrive while the connection remains open and UTF-8 characters are decoded together.
  • Use the response context manager to close the connection on normal completion, [DONE], consumer cancellation, and errors.
  • Add seven dependency-free regression and compatibility tests, including a real localhost HTTP server.

Testing

  • Before the fix: all five regression tests failed (four assertion failures and one UnicodeDecodeError at byte 4095).
  • After the fix: python -m unittest discover -s sdk/python -v passed all seven tests on Windows / Python 3.11.15, including CRLF, non-JSON data and HTTP-error compatibility cases.
  • cargo fmt --all -- --check passed. Python 3.8 syntax parsing and git diff --check passed.
  • cargo clippy --workspace --all-targets -- -D warnings passes
  • cargo test --workspace passes
  • Live integration tested with a local HTTP server; no model service or API key required.

The Rust build, Clippy and workspace-test attempts used --locked --offline and stopped at missing cached dependency async-trait. They are not reported as passing. This draft changes only the Python SDK and its tests; the repository's existing Rust CI remains available for validation.

Security

  • No new unsafe code
  • No secrets or API keys in diff
  • Input parsing retains the existing JSON/raw-data behavior; no new input surface is added.

Prepared with OpenAI Codex assistance. Submitted as a draft for maintainer review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant