Repository navigation
Closed
Description
Sub-issue of #1001 (follow-ups from the 2026-10-26 dependency update, #998).
Do part 2 first: once tool_runner / threads go through our wrapper, the runner's streaming mode iterates create(stream=True) directly.
1. tool_runner and message threads aren't traced at all
client.beta.messages.tool_runner(...)and, on 2.2+,client.beta.messages.thread(...)produce zero spans on a wrapped client, in all four modes (sync/async, streaming or not). Reproduced live on anthropic 1.17.0 and 2.3.0.- Cause: the
Messageswrapper forwards unknown attributes to the raw object (Wrapper.__getattr__inintegrations/anthropic/_utils.py). The runner gets a method bound to the raw client, so every model call in the loop skips ourcreate/stream. - Why it matters: the tool runner is Anthropic's recommended agent loop, so users of it get no LLM spans, token metrics or tool spans.
- Fix: wrap
tool_runnerandthreadon the betaMessageswrapper.- Option A: an outer span per run, with child LLM spans by routing the runner's client back through our wrappers.
- Option B: patch the runner's per-iteration request. In 2.x the loop runs over a thread that calls
self._client.beta.messages.create(...), which makes this workable. - Carry
threadmetadata (type,previous_message_id). On follow-up turns the SDK sends only the new messages and dropssystem/tools, so span input looks incomplete unless it's linked to the earlier turn.
- Depends on item 2: once these calls go through our wrapper, the runner's streaming mode iterates
create(stream=True)directly. - Tests: cassette-backed
tool_runnertests (sync/async × stream/non-stream) with a real@beta_tool, recorded atlatest.
2. Iterating messages.create(stream=True) directly raises TypeError on wrapped clients
for event in client.messages.create(..., stream=True)fails withTypeError: 'TracedMessageStreamManager' object is not iterable. Only thewith ... as stream:form works. Reproduced on 1.17.0 and 2.3.0. The code works unwrapped and breaks once wrapped.- Cause: sync
create(stream=True)wraps the result inTracedMessageStreamManager(integrations/anthropic/tracing.py), which only implements__enter__/__exit__. - Fix: return a
TracedMessageStream(or give the manager__iter__/close()/.response) that ends the span when iteration finishes or the stream closes.- The async path returns a bare async generator with no
.response/.close(). Make it consistent at the same time. - Hardening: since anthropic 1.16 the SDK passes along SSE events it doesn't recognize.
TracedMessageStream.__process_messagecallsaccumulate_eventon every event, which raises if a non-message_startevent arrives first. Skip unknown events the way the SDK's own helper does.
- The async path returns a bare async generator with no
- Tests: red→green cassette test iterating
create(stream=True)withoutwith, sync and async.
Activity
AbhiPrasad commentedon Oct 26, 2026
released with https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.48.1
fix(anthropic): trace tool runners and threads and fix streaming (bra…