Repository navigation
fix(tests): skip aiohttp stub patches that fixed VCR.py doesn't need - #834
Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit intoOct 2, 2026
Conversation
_patch_vcr_aiohttp_stubs() works around kevin1024/vcrpy#927 and kevin1024/vcrpy#502. kevin1024/vcrpy#1055 fixes both in VCR.py and makes MockStream aiohttp's StreamReader, which takes the body. The cached content property then calls MockStream() without one, so every aiohttp replay fails with a TypeError and pylint reports E1120. Skip the content property, the no-op set_exception and the record_response replacement when MockStream is a StreamReader, which is only true with that fix. VCR.py 8.3.0 gets the same patches as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN
Member
|
hey thanks for the PR! What led you to make this change? |
Contributor
Author
hey, np! Just happened to fix vcrpy, and stumbled upon this :) . Some more context in kevin1024/vcrpy#1055 |
Abhijeet Prasad (AbhiPrasad)
approved these changes
Oct 2, 2026
Abhijeet Prasad (AbhiPrasad)
merged commit Oct 2, 2026
6dd5034
into
braintrustdata:main
83 checks passed
Pierre-Yves Le Borgne (pylaterreur)
deleted the
vcrpy-1055-aiohttp-workaround
branch
October 23, 2026 12:23
Pierre-Yves Le Borgne (pylaterreur)
added a commit
to pylaterreur/braintrust-sdk-python
that referenced
this pull request
Oct 25, 2026
vcrpy 8.5.0 includes kevin1024/vcrpy#1055, which fixes kevin1024/vcrpy#502 and kevin1024/vcrpy#927: a response streamed while recording keeps its body, a replayed response keeps a single aiohttp StreamReader as its content, so google-genai's readline() loop ends, and closing the response no longer poisons that stream. Since braintrustdata#834, conftest.py has skipped its patches for those (problems 1, 3 and 4) on such releases. Require vcrpy 8.5.0 in the test group and remove them, along with the shims only older vcrpy needed: the AsyncStreamReaderMixin placeholder, the stream_writer constructor and MockStream.iter_chunked. The gzip decoding in text() and read() stays. uv's exclude-newer cutoff moves to 2026-10-25T22:30:00Z, just after vcrpy 9.2.0's upload. uv.lock now has vcrpy 9.2.0 on Python 3.11 and newer, and 8.5.0 on Python 3.10, which vcrpy 9 doesn't support. Nothing else in uv.lock changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN
Pierre-Yves Le Borgne (pylaterreur)
added a commit
to pylaterreur/braintrust-sdk-python
that referenced
this pull request
Oct 27, 2026
vcrpy 8.5.0 includes kevin1024/vcrpy#1055, which fixes kevin1024/vcrpy#502 and kevin1024/vcrpy#927: a response streamed while recording keeps its body, a replayed response keeps a single aiohttp StreamReader as its content, so google-genai's readline() loop ends, and closing the response no longer poisons that stream. Since braintrustdata#834, conftest.py has skipped its patches for those (problems 1, 3 and 4) on such releases. Require vcrpy 8.5.0 in the test group and remove them, along with the shims only older vcrpy needed: the AsyncStreamReaderMixin placeholder, the stream_writer constructor and MockStream.iter_chunked. The gzip decoding in text() and read() stays. The weekly dependency update (braintrustdata#998) moved uv's exclude-newer cutoff past vcrpy 9.2.0's upload, so uv.lock already has vcrpy 9.2.0 on Python 3.11 and newer, and 8.5.0 on Python 3.10, which vcrpy 9 doesn't support. It only gains the new requirement, in the groups that include test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN
Abhijeet Prasad (AbhiPrasad)
pushed a commit
that referenced
this pull request
Oct 27, 2026
## Summary Follow-up to #834. VCR.py [8.5.0](https://github.com/kevin1024/vcrpy/releases/tag/v8.5.0) is out with kevin1024/vcrpy#1055, which fixes kevin1024/vcrpy#502 (an aiohttp response streamed while recording came back empty) and kevin1024/vcrpy#927 (a replayed `response.content` restarted on each access, so google-genai's `readline()` loop never ended). Since #834, `_patch_vcr_aiohttp_stubs()` in `py/src/braintrust/conftest.py` skips its patches for those on such releases, and its docstring says to delete them once `uv.lock` has one. ## Changes - Add `vcrpy>=8.5.0` to the `test` dependency group. Since #998, `uv.lock` already has VCR.py 9.2.0 on Python 3.11 and newer, and 8.5.0 on Python 3.10, which VCR.py 9 doesn't support; it only gains the new requirement. - Remove the patches for problems 1, 3 and 4 (cached `content`, no-op `MockStream.set_exception`, `record_response` replacement), and the shims that only older VCR.py needed: the `AsyncStreamReaderMixin` placeholder, the `stream_writer` constructor and `MockStream.iter_chunked`. VCR.py 8.5.0 does all of these itself. The gzip decoding in `text()` and `read()` stays. - This no longer moves the `exclude-newer` cutoff: it's rebased onto #998, whose cutoff (`2026-10-26T06:02:31Z`) already covers 9.2.0. On the old base, the cutoff the earlier push set (`2026-10-25T22:30:00Z`) also let `test_pydantic_ai_integration(latest)` install anthropic 2.3.0 next to the old `pydantic-ai==2.57.0` pin. Anthropic 2 made `BetaMessageParam` a `Union`, so `test_agent_run_anthropic_reasoning_tokens` failed with `TypeError: Cannot instantiate typing.Union` in the previous CI run; #998 moved the pin to 2.61.0, which works with it. VCR.py 9.0 has some breaking changes ([release notes](https://github.com/kevin1024/vcrpy/releases/tag/v9.0.0)). Nothing in this repo matches on the `CannotOverwriteExistingCassetteException` message or uses `filter_post_data_parameters`, and the sessions below pass with 9.2.0. ## Testing nox sessions as CI runs them (`uv run --project ./py nox -f ./py/noxfile.py -s ...`, Python 3.12, cassette replay): | Session | main (VCR.py 9.2.0 since #998) | this PR | |---|---|---| | `test_litellm(latest)` | 43 passed | 43 passed | | `test_litellm(1.74.0)` | 30 passed, 13 skipped | 30 passed, 13 skipped | | `test_dspy(latest)` | 15 passed | 15 passed | | `test_google_genai(latest)` | 44 passed, 2 skipped | 44 passed, 2 skipped | | `test_huggingface_hub(0.32.0)` | 17 passed, 2 skipped | 17 passed, 2 skipped | | `test_google_adk(latest)` | 16 passed | 16 passed | | `test_google_adk(2.6.3)` | 13 passed, 3 skipped | 13 passed, 3 skipped | | `test_google_adk(1.14.1)` | 13 passed, 3 skipped | 13 passed, 3 skipped | All six of CI's nox shards for Python 3.12 on Linux (`py/scripts/nox-matrix.py <n> 6 --python-version 3.12 --os ubuntu-24.04 --exclude-static-checks --exclude-session test_api_codegen`) pass on this branch, as on main: 90 sessions, with the same installed packages in every session. On Python 3.10, where the lock has 8.5.0, the 3.10 shards pass too (75 sessions), and `test_litellm(1.74.0)`, `test_google_genai(latest)` and `test_huggingface_hub(0.32.0)` have the same counts as above. `test_pydantic_ai_integration(latest)` passes on Python 3.11 to 3.15 (61 passed). These replays don't reach the bugs the patches were for: without the patches, `test_litellm` (both versions), `test_dspy(latest)`, `test_google_genai(latest)` and `test_google_adk(latest)` also pass on VCR.py 8.4.0. `test_google_genai` has no aiohttp, since google-genai makes it optional. With aiohttp added to `test_google_genai(latest)`, as with `google-genai[aiohttp]`, this PR on VCR.py 8.4.0 hangs in `test_basic_completion[async_stream]` (kevin1024/vcrpy#927). It passes on 8.5.0 (44 passed, 2 skipped), as main does on 8.4.0. Pre-commit hooks pass on the changed files, and `pylint --errors-only` is clean on `conftest.py`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_patch_vcr_aiohttp_stubs()inpy/src/braintrust/conftest.pyworks around kevin1024/vcrpy#927 and kevin1024/vcrpy#502. kevin1024/vcrpy#1055 (not released yet) fixes both and makesMockStreamaiohttp'sStreamReader, which takes the body. Our cachedcontentproperty then callsMockStream()without a body, so aiohttp replays fail withTypeError: MockStream.__init__() missing 1 required positional argument: 'body', and pylint reports E1120. Nothing fails yet becausepy/uv.lockpins VCR.py 8.3.0, but the first weekly lock upgrade after a release with the fix would.Changes
MockStreamis aiohttp'sStreamReader, which only happens with the fix, skip the cachedcontentproperty, the no-opset_exceptionand therecord_responsereplacement. The fix isn't released yet, so a version number can't identify it, hence the class check. Thestream_writer,iter_chunkedand gziptext()/read()patches stay.no-value-for-parameteron theMockStream()call that only older VCR.py reaches.On 8.3.0 the same attributes get the same code. With the fix,
response.contentis no longer gunzipped, which none of the 1,909 recorded responses needs.Testing
Each nox session's tests, replaying cassettes with
CI=1:test_litellmlatest, 1.74.0test_huggingface_hub(0.32.0)test_google_adklatest, 2.6.3, 1.14.1test_google_genai(latest)with aiohttppylint --errors-onlyRecording LiteLLM, google-genai and huggingface_hub calls against a local server and replaying them also works on both versions. No new test: these sessions already reproduce the failure. Pre-commit hooks pass.
🤖 Generated with Claude Code
https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN