Skip to content

Python: Require expected keys in tool argument evaluation - #8560

Merged
Evan Mattson (moonbox3) merged 2 commits into
microsoft:mainfrom
YaoxinHuang:fix/eval-null-argument-presence
Sep 21, 2026
Merged

Evan Mattson (moonbox3) merged 2 commits into
microsoft:mainfrom
YaoxinHuang:fix/eval-null-argument-presence

Conversation

@YaoxinHuang

Copy link
Copy Markdown
Contributor

Motivation & Context

tool_call_args_match accepts an omitted argument when its expected value is None, allowing an incorrect tool call to pass local evaluation.

Description & Review Guide

  • What are the major changes? Require each expected argument key to exist before comparing its value.
  • What is the impact of these changes? Missing null-valued arguments fail evaluation; explicit null and extra arguments retain their existing behavior.
  • What do you want reviewers to focus on? The distinction between missing and explicit-null keys, covered for dictionary and JSON-string arguments.

The local evaluation tests, core type checks, pre-commit checks, and package build pass. The full repository test suite was not run.

Related Issue

Fixes #8558

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation correctly distinguishes missing keys from explicit null values and includes focused regression coverage.

Review effort: Balanced
Findings: None

@moonbox3
Evan Mattson (moonbox3) added this pull request to the merge queue Sep 21, 2026
Merged via the queue into microsoft:main with commit 2565daf Sep 21, 2026
49 of 52 checks passed

This branch was previously deployed

1 inactive deployment
github-app-auth — c68ab467 Deployed Sep 20, 2026 by YaoxinHuang via team_check #4877
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: tool_call_args_match accepts missing null-valued arguments

3 participants