Skip to content

test: lock in Bedrock and OpenAI _parse_native_tool_call arg handling (#4972)#5898

Open
adityasingh2400 wants to merge 1 commit into
crewAIInc:mainfrom
adityasingh2400:test-parse-native-tool-call-bedrock-openai-4972
Open

test: lock in Bedrock and OpenAI _parse_native_tool_call arg handling (#4972)#5898
adityasingh2400 wants to merge 1 commit into
crewAIInc:mainfrom
adityasingh2400:test-parse-native-tool-call-bedrock-openai-4972

Conversation

@adityasingh2400
Copy link
Copy Markdown

@adityasingh2400 adityasingh2400 commented May 21, 2026

Summary

Issue #4972 reported that AWS Bedrock Converse tool calls had their arguments silently dropped because the dict branch of _parse_native_tool_call used a truthy default in func_info.get("arguments", "{}"), which short-circuited the or before the Bedrock input key was read.

The runtime fix shipped in commit 25fcf39 (fix: preserve Bedrock tool call arguments by removing truthy default), but no test locked in the behaviour, so the issue stayed open. This PR adds the regression coverage.

What is covered

A new TestParseNativeToolCall class in lib/crewai/tests/agents/test_native_tool_calling.py with parametrized cases:

  • Bedrock Converse dict shape {"name": ..., "input": {...}, "toolUseId": ...} — asserts toolUseId is used as call id, the tool name is parsed, and input round-trips as the args dict.
  • OpenAI dict shape {"id": ..., "function": {"name": ..., "arguments": "...json..."}} — asserts the id is preserved and the JSON arguments string is returned unchanged.
  • A third case asserts the parsed args dict equals the original Bedrock input (and is not the empty dict that triggered the bug).

I verified the tests catch the regression by temporarily restoring the buggy default (func_info.get("arguments", "{}")): the two Bedrock cases fail with '{}' == {...} and the OpenAI case still passes, exactly matching the bug surface in the issue.

This is a test-only PR, no source changes.

Test plan

Fixes #4972

Summary by CodeRabbit

  • Tests
    • Added a regression test suite validating parsing of native tool-call formats from multiple LLM providers, ensuring call IDs, tool/function names, and arguments are correctly preserved and extracted across provider-specific shapes.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 21, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 22f66fd7-d668-4e1c-af09-97fcdbfb8627

📥 Commits

Reviewing files that changed from the base of the PR and between dc8f655 and 3a0709b.

📒 Files selected for processing (1)
  • lib/crewai/tests/agents/test_native_tool_calling.py

📝 Walkthrough

Walkthrough

This PR adds regression test coverage for the _parse_native_tool_call method, verifying parsing of Bedrock Converse ({"name","input","toolUseId"}) and OpenAI-style ({"function":{"name","arguments"}}) tool-call dicts and ensuring Bedrock tool arguments are preserved.

Changes

Native tool-call parsing regression tests

Layer / File(s) Summary
Regression test suite for _parse_native_tool_call
lib/crewai/tests/agents/test_native_tool_calling.py
Parametrized test class TestParseNativeToolCall covers both Bedrock Converse dict shape ({"name", "input", "toolUseId"}) and OpenAI-style dict shape ({"function": {"name", "arguments"}}). Dedicated assertion ensures parsed Bedrock arguments match the original input dict and are not the empty dict that caused the regression.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A tiny test hopped into the suite,
To catch where Bedrock's inputs lost their feet.
It checks both styles, precise and small—
So args stand proud and never fall.
🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main change—adding regression tests to lock in Bedrock and OpenAI tool call argument handling in _parse_native_tool_call.
Linked Issues check ✅ Passed The PR adds comprehensive regression tests covering both Bedrock Converse dict shape and OpenAI dict shape, directly addressing issue #4972's requirements to validate argument preservation in _parse_native_tool_call.
Out of Scope Changes check ✅ Passed All changes are in-scope test additions directly related to the linked issue #4972, with no runtime code modifications or unrelated alterations.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@adityasingh2400
Copy link
Copy Markdown
Author

Hi maintainers, this PR is from a fork and the required CI workflows (Lint, Run Type Checks, Run Tests on 3.10 to 3.13) are sitting in the action_required state, which is the first-time-contributor gate. Could one of you approve the workflow runs so the required checks can complete and the PR can be merged? The change is test-only (one new file under lib/crewai/tests/agents/) and locks in the Bedrock arg-handling fix from #4972. Happy to rebase if you need anything else. Thanks.

The runtime fix from commit 25fcf39 handled the Bedrock Converse
dict shape that previously dropped tool arguments, but no test
locked the behaviour in. Add parametrized cases asserting both the
Bedrock {'name', 'input', 'toolUseId'} shape and the OpenAI
{'function': {'name', 'arguments'}} shape round-trip through
_parse_native_tool_call without losing args.

Fixes crewAIInc#4972
@adityasingh2400 adityasingh2400 force-pushed the test-parse-native-tool-call-bedrock-openai-4972 branch from dc8f655 to 3a0709b Compare May 23, 2026 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] _parse_native_tool_call drops Bedrock Converse API tool arguments — always passes empty dict

1 participant