fix(openai): handle None arguments in legacy function_call streaming merge#1753
Open
roli-lpci wants to merge 2 commits into
Open
fix(openai): handle None arguments in legacy function_call streaming merge#1753roli-lpci wants to merge 2 commits into
roli-lpci wants to merge 2 commits into
Conversation
…se#1751 arity _extract_streamed_openai_response now returns a 5-tuple (adds service_tier, upstream langfuse#1751) instead of 4. Update this test's unpack to match; the bug assertions (model/completion/metadata content) are unchanged.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
_extract_streamed_openai_response(langfuse/openai.py) merges the deprecatedfunction_callstreaming delta with a bare
+=:getattr(obj, "arguments", "")only returns the""default when the attribute is absent. SomeOpenAI-compatible proxies send a first delta with
function_call.argumentsexplicitly set toNone(not missing) — a real payload shape, not a hypothetical one. In that casecurr["arguments"]isNone, andNone += strraisesTypeError, killing the merge for thewhole streamed completion. The
TypeErroris currently swallowed by a bareexcept Exception: passfurther up the call stack, so today this fails silently: the generation ends with no outputand no error a caller would notice.
This is the same crash class PR #1339 already fixed for the newer
tool_callsarray — but thatfix only touched the
tool_callsbranch a few lines below; the sibling legacyfunction_callbranch was never patched. The
tool_callsbranch already guards the accumulator side and skipsNoneincrements:The fix brings the same protection to the legacy branch, adopting that branch's
(... or "") + ...accumulator guard and coalescing the incoming delta inline so aNone(ormissing)
argumentson either side can't break the concatenation:This guards both sides: the accumulator being
None, and the incoming delta'sargumentsbeingNoneor missing. When both sides are normal strings it is equivalent to the original+=— nobehavior change on the happy path. Two-line functional change to a single merge branch; does not
touch the
tool_callsbranch or any other streaming path.No existing issue or open PR describes this legacy-
function_call-branch crash (see the searchunder Verification below); happy to open a tracking issue first if that's preferred for bug-fix PRs.
Type of change
Verification
List the main commands you ran:
Test added:
test_streaming_chat_completion_handles_legacy_function_call_none_arguments, mirroringthe existing
test_streaming_chat_completion_preserves_tool_calls_after_contentpattern but forthe legacy branch, with a first delta carrying
function_call.arguments=None.Checklist
code_review.md..env.templateif needed. (No public API or documentedbehavior changed; two-line internal guard.)
regeneration path.
Greptile Summary
This PR fixes a silent
TypeErrorcrash in the deprecatedfunction_callstreaming-merge branch of_extract_streamed_openai_responsewhen an OpenAI-compatible proxy sends a delta withargumentsexplicitly set toNone. The fix mirrors the identical guard already applied to thetool_callsbranch in PR #1339 that was never extended to this legacy sibling branch.langfuse/openai.py: Replaces the barecurr["arguments"] += getattr(...)accumulation with a null-safe pattern(curr.get("arguments") or "") + (getattr(...) or ""), protecting both the accumulator and the incoming delta from beingNone.tests/unit/test_openai.py: Adds a regression test that replays the two-delta payload shape (first delta witharguments=None, second with the real JSON string) and asserts correct final output.Confidence Score: 5/5
Safe to merge — the change is a two-line targeted guard that cannot regress the happy path (string + string produces the same result as before), and the new test directly validates the failing case.
The fix is a minimal null-coalescing guard that mirrors an already-proven pattern from the tool_calls branch. The initialization path at line 807 could still store None for arguments on a single-delta stream, but that pre-existing edge produces None output rather than a crash and is out of scope for this PR. No other correctness issues found.
No files require special attention.
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant Proxy as OpenAI-compatible Proxy participant Extract as _extract_streamed_openai_response participant Accum as function_call accumulator Proxy->>Extract: "chunk 1: function_call.arguments = None" Extract->>Accum: "if not curr → initialize {name, arguments: None}" Proxy->>Extract: "chunk 2: function_call.arguments = '{"city":"Berlin"}'" Note over Extract,Accum: BEFORE fix: None += str → TypeError (swallowed silently) Note over Extract,Accum: AFTER fix: (None or "") + (str or "") → '{"city":"Berlin"}' Extract->>Accum: "curr["arguments"] = "" + '{"city":"Berlin"}'" Accum-->>Extract: "{name: "get_weather", arguments: '{"city":"Berlin"}'}"%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant Proxy as OpenAI-compatible Proxy participant Extract as _extract_streamed_openai_response participant Accum as function_call accumulator Proxy->>Extract: "chunk 1: function_call.arguments = None" Extract->>Accum: "if not curr → initialize {name, arguments: None}" Proxy->>Extract: "chunk 2: function_call.arguments = '{"city":"Berlin"}'" Note over Extract,Accum: BEFORE fix: None += str → TypeError (swallowed silently) Note over Extract,Accum: AFTER fix: (None or "") + (str or "") → '{"city":"Berlin"}' Extract->>Accum: "curr["arguments"] = "" + '{"city":"Berlin"}'" Accum-->>Extract: "{name: "get_weather", arguments: '{"city":"Berlin"}'}"Reviews (1): Last reviewed commit: "test(openai): rebase legacy function_cal..." | Re-trigger Greptile