Skip to content

fix: harden orjson import and use explicit UTF-8 file encoding - #929

Closed
Abhijeet Prasad (AbhiPrasad) wants to merge 1 commit into
mainfrom
abhi/py315-orjson-encoding-hardening
Closed

Abhijeet Prasad (AbhiPrasad) wants to merge 1 commit into
mainfrom
abhi/py315-orjson-encoding-hardening

Conversation

@AbhiPrasad

@AbhiPrasad Abhijeet Prasad (AbhiPrasad) commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Two small robustness fixes found during Python 3.15 work.

Changes

  • orjson import fallback (bt_json.py): fall back to stdlib json on any exception from import orjson, not just ImportError. Under Python 3.15's global lazy-import mode (-X lazy_imports=all / PYTHON_LAZY_IMPORTS=all), orjson 3.12.0 raises SystemError on import (an upstream bug). Bare import braintrust still succeeds because the SDK's submodules are deferred, but first real use (init_logger, bt_dumps) fails with ImportError: lazy import of 'braintrust' raised an exception during resolution. orjson 3.13.0 is not affected, but our test-litellm group pins 3.12.0 on 3.15.
  • Explicit UTF-8 file encoding: pass encoding="utf-8" to text-mode file I/O in the SDK. Python 3.15 defaults to UTF-8 (PEP 686), but 3.10–3.14 on Windows use the locale encoding. The main site is _HTTPBackgroundLogger._write_payload_to_dir: orjson output keeps non-ASCII characters, so on a non-UTF-8 locale the write raised UnicodeEncodeError and the payload dump was lost (an error is printed to stderr and the file is left empty). Also covers the btx spec loader, braintrust push, the span cache, and harbor's job/trial/manifest/instructions files. A scan of non-test code under py/src/braintrust finds no remaining sites.

Validation

  • Red: both new subprocess tests fail without the fixes (test_bt_json_falls_back_to_json_when_orjson_import_fails simulates a SystemError from the orjson import, with a 3.15-only -X lazy_imports=all variant; test_write_payload_to_dir_uses_utf8 runs with -X warn_default_encoding and warnings as errors).
  • Manual repro with real orjson 3.12.0 on 3.15.0rc2 under -X lazy_imports=all: init_logger and bt_dumps fail before, work after.
  • Manual repro on 3.14 with orjson under an ISO-8859-1 locale: the payload dump fails before ('latin-1' codec can't encode characters), is written correctly after.
  • Green:
Session Result
test_core (3.14.6) 1019 passed
test_core (3.15.0rc2) 1025 passed
test_harbor(latest) (3.14; harbor doesn't support 3.15) 42 passed
pylint passed

Running the core and harbor suites with PYTHONWARNDEFAULTENCODING=1 reports no EncodingWarning from SDK source, only from tests, noxfile.py, scripts, and vcrpy. Pre-commit passes on the changed files.

🤖 Generated with Claude Code

Co-authored by StarfolkAI (@starfolkai)[bot]

- bt_json: fall back to stdlib json on any exception from `import orjson`,
  not just ImportError. orjson 3.12.0 raises SystemError on import under
  Python 3.15's `-X lazy_imports=all`, which broke the SDK on first use.
- Pass `encoding="utf-8"` to text-mode file I/O in the SDK. Python 3.10-3.14
  on Windows default to the locale encoding, so writing orjson output
  (which keeps non-ASCII characters) to a payload dump could fail and the
  dump was lost. Also covers the btx spec loader, `braintrust push`, the
  span cache, and harbor's job/trial/manifest/instructions files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T16:41:31.280385Z f390902 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) deleted the abhi/py315-orjson-encoding-hardening branch October 13, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant