[Gareth.Y (linjiaye929)]

Summary

Forward the existing error_score_handler parameter from EvalAsync() to _EvalCommon, as Eval() already does.

Previously, passing a custom handler to the async public API silently discarded it. Even for one evaluation row, a task or scorer failure produced no fallback scores and never called the handler. The synchronous API returned the configured fallback score for the same input.

This is a one-line forwarding fix. Public signatures, default values, runner behavior and exception handling are unchanged.

Regression coverage

18 parameterized cases exercise the real public Eval and EvalAsync APIs across:

• task failure, scorer failure and successful evaluation;
• handlers returning fallback scores, handlers returning None, and no custom handler.

Tests verify callback invocation, scores and retained task/scorer error semantics. Successful evaluations do not invoke the handler.

Validation

Python 3.12.14, existing local test environment, offline socket guard, no_send_logs=True, cache disabled for the new regressions:

• New regressions on the unchanged base: 4 failed, 14 passed.
• After the fix: 18 passed.
• src/braintrust/test_framework.py: 139 passed, 2 skipped, independently repeated with the same result. The skips concern optional Pydantic and Python 3.14 lazy annotations.
• Runtime assertions in type_tests/test_eval_generics.py and type_tests/test_dict_scorers.py: 17 passed. This is not a static type-check result.
• Ruff check/format, git diff --check, and the configured pre-commit checks for these files pass (two unrelated file-filtered hooks skipped).
• All test runs recorded zero attempted network connections. No live provider calls or cassette recording.
• Full nox core/provider/Python-version CI matrix and pyright/mypy/pylint were not run.

Related work and scope

Related to #909, which fixes per-row state and handler execution inside the runner. This PR only fixes the public EvalAsync argument forwarding, reproducible with one row; it does not modify or duplicate that runner work.

AI assistance was used for analysis, implementation and the listed local verification. No human approval or hosted CI success is claimed.

Hosted checks

At head 12484d1684c1f3946f91a4780b95fbf3b3cb87b8, checks and integration-tests are both completed with action_required, each with zero jobs. There are zero check runs. Hosted tests have not executed; local results above are not a claim of passing hosted CI.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-14T20:23:48.794740Z	12484d1	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review" or "Codex (@codex) security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[Abhijeet Prasad (AbhiPrasad)]

released with https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.46.1