Skip to content

fix(anthropic): end the span when a streaming messages.create call fails - #991

Closed
alexcastgonz (pcbeingused333) wants to merge 1 commit into
braintrustdata:mainfrom
pcbeingused333:fix/anthropic-sync-stream-create-error
Closed

alexcastgonz (pcbeingused333) wants to merge 1 commit into
braintrustdata:mainfrom
pcbeingused333:fix/anthropic-sync-stream-create-error

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Messages.create(stream=True) on the sync wrapped Anthropic client goes through __trace_stream, which starts the anthropic.messages.stream span and then calls the SDK with no error handling. If the request itself fails (a 4xx from the API, a connection error), the exception propagates and the span is never ended and never gets the error, so the failed call is traced as an unfinished span with no error. The async client's __create_with_stream_true already logs the error and ends the span in that case, as do the sync and async non-streaming calls.

__trace_stream now wraps the SDK call in the same try/except: on an exception it logs the error on the span, ends it and re-raises. Successful calls are unchanged.

Test: test_anthropic_sync_create_stream_logs_error_and_ends_span calls messages.create(stream=True) on the wrapped client with an invalid API key, so the API answers with a real 401, and checks that the call raises AuthenticationError and that the span has the error. The response is recorded in a cassette (cassettes/latest and cassettes/0.48.0; the repo filter removes the x-api-key header and the recording holds no credentials). It fails on main (the span has no error) and passes with the change. The other tests of the file were not run in the environment used here.

This change was made with Claude Code.

🤖 Generated with Claude Code

The sync Messages.create(stream=True) path started the span and then
called the SDK with no error handling. When the request itself failed
(for example a 4xx or a connection error), the span was never ended and
the error was never logged, so the failed call showed up as an unfinished
span without an error. The async client already logs the error and ends
the span in that case.

The sync path now does the same.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T23:44:37.944442Z a8dfdde PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these cassettes are not valid

    status:
      code: 401
      message: Unauthorized

@pcbeingused333

Copy link
Copy Markdown
Contributor Author

Thanks for the look. I'm not sure which part is invalid, so here is what the cassettes are and the options.

Both files record one real call to api.anthropic.com made with a throwaway invalid key (invalid-key-for-error-cassette), so the API answered with the 401 that the test asserts on. cassettes/0.48.0/ is a copy of cassettes/latest/, not a separate recording, so its recorded user-agent says Anthropic/Python 1.16.0.

If the objection is the 401 itself: integrations/agentscope/cassettes/latest/test_agentscope_model_call_error_propagates.yaml is the same kind of recording (an invalid key, answered with a 401). A different API error such as a 400 or a 529 can only be recorded with a valid key; this one was deliberately recorded without one.

Which would you prefer?

  1. Keep the cassette-backed 401 test as it is.
  2. Replace it with a narrow monkeypatch that makes the wrapped SDK call raise, which AGENTS.md allows for error paths a recording cannot provoke.
  3. Keep the test and have a maintainer re-record it with the setup you prefer, if the 0.48.0 copy is the problem.

I'll update the PR once you pick one.

@AbhiPrasad

Copy link
Copy Markdown
Member

No I want an actual cassette recording, the agentscope cassette is wrong and needs to be updated. I'll take care of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants