[sentry-junior]

Description

A customer reported that model costs are missing for OpenAI Agents spans. Relay prices each span using gen_ai.response.model, falling back to gen_ai.request.model, plus the token counts. This PR fixes two cases where the chat span reports the wrong model:

1. Streamed Chat Completions report the requested name as the response model. When OpenAIChatCompletionsModel (or LitellmModel) streams, the Agents SDK builds the final Response with model=self.model. That is the requested name, such as an Azure deployment name like prod-gpt4o. We copied it into gen_ai.response.model, so Relay found no price and set no cost. The model that actually responded (e.g. gpt-4o-2024-08-06) only shows up on the provider chunks. We now wrap the chunk stream that _fetch_response(stream=True) returns, record chunk.model, and prefer that value. The wrapper is a pass-through proxy: it delegates every attribute to the original stream.

2. RunConfig.model was ignored for gen_ai.request.model and the span name. get_model() gives RunConfig.model priority over Agent.model, but the chat span read agent.model. A run with Agent(model="gpt-4") and RunConfig(model="gpt-4.1-mini") was reported as chat gpt-4. The chat span now uses the name of the model instance that the runner resolved in _get_model. That name also has provider prefixes like litellm/ already removed.

Also: the response model captured from _fetch_response used to be stored on the shared Agent object (agent._sentry_response_model). It now lives in a closure on the model copy made for each turn, so concurrent runs that share one Agent don't read each other's state.

Before → after, with openai-agents 0.22.3 and a mocked Azure client:

chatcmpl stream, deployment "prod-gpt4o":   response.model 'prod-gpt4o'  ->  'gpt-4o-2024-08-06'
RunConfig(model="gpt-4.1-mini") over "gpt-4o": request.model 'gpt-4o'    ->  'gpt-4.1-mini'

Not fixed here (an upstream default, not an SDK bug): for clients that aren't the official OpenAI endpoint (Azure, OpenAI-compatible proxies), openai-agents doesn't send stream_options.include_usage when streaming Chat Completions. The provider then returns no usage, so there are no tokens and no cost. Users need ModelSettings(include_usage=True).

Tests

• test_ai_client_span_request_model_uses_run_config_model
• test_ai_client_span_response_model_from_streamed_chat_completions. Skipped on openai-agents < 0.7.0, because its Chat Completions streaming already fails against the pinned openai types on master.

Both new tests fail on master and pass with this change. I ran the full tests/integrations/openai_agents suite locally on Python 3.12 against openai-agents 0.22.3, 0.14.8, 0.7.0 and 0.0.19, and all pass (on 0.0.19 the new streaming test is skipped). ruff check/ruff format are clean, and mypy reports no errors in sentry_sdk/integrations/openai_agents.

via matej.

--

View Junior Session [Sentry]

[matejminar]

we can close this, I just asked Junior to do some digging while in meetings, didn't expect it to open a PR

[github-actions]

Codecov Results 📊

✅ 129617 passed | ❌ 3 failed | ⏭️ 7232 skipped | Total: 136852 | Pass Rate: 94.71% | Execution Time: 442m 16s

❌ Failed Tests

test_additional_parameters[True-False]

File: tests.integrations.litellm.test_litellm
Suite: py3.10-litellm-v1.96.0
Error: assert 0 == 1 + where 0 = len([])

Stack Trace

tests/integrations/litellm/test_litellm.py:2274: in test_additional_parameters
    assert len(chat_spans) == 1
E   assert 0 == 1
E    +  where 0 = len([])

test_multiple_agents_asyncio[True-True]

File: tests.integrations.openai_agents.test_openai_agents
Suite: py3.14t-openai_agents-v0.7.0
Error: IndexError: list index out of range

Stack Trace

tests/integrations/openai_agents/test_openai_agents.py:5246: in test_multiple_agents_asyncio
    assert spans[8]["name"] == "test_agent workflow"
           ^^^^^^^^
E   IndexError: list index out of range

✅ Patch coverage is 97.83%. Project has 2559 uncovered lines.
✅ Project coverage is 90.36%. Comparing base (564c6c1) to head (ac8e2a8).

Files with missing lines (2)

File	Patch %	Lines
sentry_sdk/integrations/openai_agents/utils.py	85.71%	⚠️ 1 Missing and 1 partials
sentry_sdk/integrations/openai_agents/patches/models.py	100.00%	⚠️ 1 partials

Coverage diff

@@            Coverage Diff             @@
##        master       #PR       +/-##
==========================================
+ Coverage    90.34%    90.36%    +0.02%
==========================================
  Files          199       199         —
  Lines        26524     26548       +24
  Branches      9864      9862        -2
==========================================
+ Hits         23962     23989       +27
- Misses        2562      2559        -3
- Partials      1495      1494        -1

---

Generated by Codecov Action