[issue-6019] [P SDK] Fix cache tokens not tracked in Bedrock Claude streaming#6038
Draft
ollieagent[bot] wants to merge 1 commit intomainfrom
Draft
[issue-6019] [P SDK] Fix cache tokens not tracked in Bedrock Claude streaming#6038ollieagent[bot] wants to merge 1 commit intomainfrom
ollieagent[bot] wants to merge 1 commit intomainfrom
Conversation
…ing usage Extract cache_creation_input_tokens and cache_read_input_tokens from the message_start chunk in ClaudeAggregator.aggregate() and pass them to anthropic_to_bedrock_usage(), so cacheWriteInputTokens and cacheReadInputTokens are correctly tracked in streaming responses. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
6 tasks
github-actions
Bot
added
python
Contributor
Python SDK Unit Tests Results (Python 3.11)1 tests 0 ✅ 7s ⏱️ For more details on these errors, see this check. Results for commit a4953bc. |
Contributor
Python SDK Unit Tests Results (Python 3.12)1 tests 0 ✅ 7s ⏱️ For more details on these errors, see this check. Results for commit a4953bc. |
Contributor
Python SDK Unit Tests Results (Python 3.14)1 tests 0 ✅ 4s ⏱️ For more details on these errors, see this check. Results for commit a4953bc. |
Contributor
Python SDK Unit Tests Results (Python 3.13)1 tests 0 ✅ 8s ⏱️ For more details on these errors, see this check. Results for commit a4953bc. |
Contributor
Python SDK Unit Tests Results (Python 3.10)1 tests 0 ✅ 8s ⏱️ For more details on these errors, see this check. Results for commit a4953bc. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
In
ClaudeAggregator.aggregate(), themessage_startchunk handler only extractedinput_tokensandoutput_tokensfrom the usage dict. The cache token fields (cache_creation_input_tokens,cache_read_input_tokens) were present in the Anthropic streaming response but were never read, soanthropic_to_bedrock_usage()always received zeros for them — causingcacheWriteInputTokensandcacheReadInputTokensto be permanently 0 in the logged span usage.The fix initializes both cache token variables before the chunk loop, extracts them from the
message_startchunk'smessage.usagedict, and includes them in the call toanthropic_to_bedrock_usage(). The non-streaming path was already correct (it passes the full usage dict from the response body).Change checklist
Issues
Testing
New unit tests in
tests/unit/integrations/bedrock/test_claude_aggregator.py:test_claude_aggregator__cache_tokens_in_message_start__included_in_usage— verifies thatcacheWriteInputTokensandcacheReadInputTokensare populated from themessage_startchunk when cache tokens are present.test_claude_aggregator__no_cache_tokens__defaults_to_zero— verifies the zero-default behavior is preserved when cache tokens are absent.To run:
cd sdks/python python -m pytest tests/unit/integrations/bedrock/test_claude_aggregator.py -vDocumentation
No documentation updates needed.