[BUG CLIENT]: Z.ai GLM 5.3 streaming responses are (sometimes) malformed in various ways

[smathot]

Python -VV

Python 3.11.13 | packaged by conda-forge | (main, Jun  4 2025, 14:48:23) [GCC 13.3.0]

Pip Freeze

aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiosignal==1.4.0
alabaster==1.0.0
alembic==1.16.5
annotated-types==0.7.0
anthropic==0.86.0
anyio==4.10.0
arrow==1.3.0
asttokens==3.0.0
attrs==25.3.0
babel==2.17.0
backoff==2.2.1
bcrypt==4.3.0
beautifulsoup4==4.13.5
bibtexparser==1.4.4
blinker==1.9.0
build==1.3.0
cachelib==0.13.0
cachetools==5.5.2
certifi==2025.8.3
cffi==1.17.1
charset-normalizer==3.4.3
chromadb==1.0.20
click==8.2.1
coloredlogs==15.0.1
comm==0.2.3
cryptography==45.0.7
dataclasses-json==0.6.7
datamatrix==1.0.16
ddgs==9.16.0
debugpy==1.8.16
decorator==5.2.1
Deprecated==1.2.18
distro==1.9.0
docstring_parser==0.17.0
docutils==0.21.2
durationpy==0.10
et_xmlfile==2.0.0
eval_type_backport==0.2.2
executing==2.2.1
fake-useragent==2.2.0
filelock==3.19.1
Flask==3.1.2
flask-cors==6.0.1
Flask-Login==0.6.3
Flask-Migrate==4.1.0
Flask-Scss==0.5
Flask-Session==0.8.0
Flask-SQLAlchemy==3.1.1
Flask-WTF==1.2.2
flatbuffers==25.2.10
free_proxy==1.1.3
frozenlist==1.7.0
fsspec==2025.9.0
google-auth==2.40.3
google-genai==1.37.0
googleapis-common-protos==1.70.0
greenlet==3.2.4
grpcio==1.74.0
h11==0.16.0
hf-xet==1.1.9
httpcore==1.0.9
httptools==0.6.4
httpx==0.28.1
httpx-sse==0.4.1
huggingface-hub==0.34.4
humanfriendly==10.0
idna==3.10
imagesize==1.4.1
importlib_metadata==8.7.0
importlib_resources==6.5.2
iniconfig==2.1.0
invoke==2.2.0
ipykernel==6.30.1
ipython==9.5.0
ipython_pygments_lexers==1.1.1
itsdangerous==2.2.0
jedi==0.19.2
Jinja2==3.1.6
jiter==0.10.0
jq==1.10.0
json-tricks==3.17.3
jsonpatch==1.33
jsonpath-python==1.1.5
jsonpointer==3.0.0
jsonschema==4.25.1
jsonschema-specifications==2025.4.1
jupyter_client==8.6.3
jupyter_core==5.8.1
kubernetes==33.1.0
langchain==0.3.27
langchain-anthropic==0.3.19
langchain-community==0.3.29
langchain-core==0.3.75
langchain-mistralai==0.2.11
langchain-openai==0.3.32
langchain-text-splitters==0.3.11
langsmith==0.4.23
lxml==6.0.1
Mako==1.3.10
Markdown==3.8.2
markdown-it-py==4.0.0
MarkupSafe==3.0.2
marshmallow==3.26.1
matplotlib-inline==0.1.7
mdurl==0.1.2
mistralai==2.1.3
mmh3==5.2.0
mpmath==1.3.0
msgspec==0.19.0
multidict==6.6.4
mypy_extensions==1.1.0
nest-asyncio==1.6.0
numpy==2.3.2
oauthlib==3.3.1
onnxruntime==1.22.1
openai==2.29.0
openpyxl==3.1.5
opentelemetry-api==1.39.1
opentelemetry-exporter-otlp-proto-common==1.39.1
opentelemetry-exporter-otlp-proto-grpc==1.39.1
opentelemetry-exporter-otlp-proto-http==1.39.1
opentelemetry-proto==1.39.1
opentelemetry-sdk==1.39.1
opentelemetry-semantic-conventions==0.60b1
orjson==3.11.3
outcome==1.3.0.post0
overrides==7.7.0
packaging==25.0
parso==0.8.5
pexpect==4.9.0
pillow==12.1.0
platformdirs==4.4.0
pluggy==1.6.0
posthog==5.4.0
prettytable==3.16.0
primp==2.0.1
prompt_toolkit==3.0.52
propcache==0.3.2
protobuf==6.32.0
psutil==7.0.0
ptyprocess==0.7.0
pure_eval==0.2.3
pyalex==0.20
pyasn1==0.6.1
pyasn1_modules==0.4.2
pybase64==1.4.2
pycparser==2.22
pydantic==2.11.7
pydantic-settings==2.10.1
pydantic_core==2.33.2
Pygments==2.19.2
PyJWT==2.13.0
pylatexenc==2.11
pyOpenSSL==25.1.0
pyparsing==3.2.3
PyPika==0.48.9
pyproject_hooks==1.2.0
pyScss==1.4.0
PySocks==1.7.1
pytest==8.4.2
python-dateutil==2.9.0.post0
python-dotenv==1.1.1
PyYAML==6.0.2
pyzmq==27.1.0
redis==6.4.0
referencing==0.36.2
regex==2025.9.1
requests==2.32.5
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
rich==14.1.0
roman-numerals-py==3.1.0
rpds-py==0.27.1
rsa==4.9.1
scholarly==1.7.11
scipy==1.16.1
selenium==4.35.0
shellingham==1.5.4
sigmund @ file:///home/sebastiaan/git/sigmundai
six==1.17.0
sniffio==1.3.1
snowballstemmer==3.0.1
sortedcontainers==2.4.0
soupsieve==2.8
Sphinx==8.2.3
sphinx-rtd-theme==3.0.2
sphinxcontrib-applehelp==2.0.0
sphinxcontrib-devhelp==2.0.0
sphinxcontrib-htmlhelp==2.1.0
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==2.0.0
sphinxcontrib-serializinghtml==2.0.0
SQLAlchemy==2.0.43
stack-data==0.6.3
stripe==12.5.0
sympy==1.14.0
tenacity==9.1.2
tiktoken==0.11.0
tokenizers==0.22.0
tomlkit==0.13.3
tornado==6.5.2
tqdm==4.67.1
traitlets==5.14.3
trio==0.30.0
trio-websocket==0.12.2
typer==0.17.3
types-python-dateutil==2.9.0.20250822
typing-inspect==0.9.0
typing-inspection==0.4.1
typing_extensions==4.14.1
urllib3==2.5.0
uvicorn==0.35.0
uvloop==0.21.0
watchfiles==1.1.0
wcwidth==0.2.13
websocket-client==1.8.0
websockets==15.0.1
Werkzeug==3.1.3
wrapt==1.17.3
wsproto==1.2.0
WTForms==3.2.1
yarl==1.20.1
zai-sdk==0.2.3
zipp==3.23.0
zstandard==0.24.0

Reproduction Steps

The issues are hard to reproduce exactly because they occur intermittently in the context of SigmundAI during complex conversations. But I've encountered the following issues quite often. I ran into them with Z.ai GLM 5.3, but I haven't tested it with other models, so I'm not 100% sure they are specific to GLM 5.3.

Below example CompletionEvent objects that illustrate the issues. These are directly printed to stdout during streaming for debugging purposes.

I noted:

• Empty responses
• Responses with only thinking content
• Tool calls to function names that are empty strings
• Tool calls to function names that are malformed

Empty responses

Here the response is entirely empty.

CompletionEvent(data=CompletionChunk(id='cb0c8e5e2ade4b5580a976fbfc76713b', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role='assistant', content='', tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896503, usage=None))
CompletionEvent(data=CompletionChunk(id='cb0c8e5e2ade4b5580a976fbfc76713b', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role='assistant', content='', tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896504, usage=None))
CompletionEvent(data=CompletionChunk(id='cb0c8e5e2ade4b5580a976fbfc76713b', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content='', tool_calls=Unset()), finish_reason='stop')], object='chat.completion.chunk', created=1789896504, usage=UsageInfo(prompt_tokens=16715, completion_tokens=49, total_tokens=16764, prompt_audio_seconds=Unset(), prompt_tokens_details={'cached_tokens': 0})))

Responses with only thinking content

Here there is only thinking (and empty) content.

CompletionEvent(data=CompletionChunk(id='8c5a4cf998b34348b7152cd8ad5104f3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role='assistant', content='', tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896473, usage=None))
CompletionEvent(data=CompletionChunk(id='8c5a4cf998b34348b7152cd8ad5104f3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role='assistant', content='', tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896473, usage=None))
CompletionEvent(data=CompletionChunk(id='8c5a4cf998b34348b7152cd8ad5104f3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content=[ThinkChunk(thinking=[TextChunk(text='Now', type='text')], type='thinking', closed=True)], tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896473, usage=None))
CompletionEvent(data=CompletionChunk(id='8c5a4cf998b34348b7152cd8ad5104f3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content=[ThinkChunk(thinking=[TextChunk(text=' I have a good understanding of', type='text')], type='thinking', closed=True)], tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896473, usage=None))
CompletionEvent(data=CompletionChunk(id='8c5a4cf998b34348b7152cd8ad5104f3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content=[ThinkChunk(thinking=[TextChunk(text=' the codebase. The', type='text')], type='thinking', closed=True)], tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896473, usage=None))
(... much more thinking content)
CompletionEvent(data=CompletionChunk(id='8c5a4cf998b34348b7152cd8ad5104f3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content=[ThinkChunk(thinking=[TextChunk(text=' now.', type='text')], type='thinking', closed=True)], tool_calls=Unset()), finish_reason=None)], object='chat.completion.chunk', created=1789896473, usage=None))
CompletionEvent(data=CompletionChunk(id='8c5a4cf998b34348b7152cd8ad5104f3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content='', tool_calls=Unset()), finish_reason='stop')], object='chat.completion.chunk', created=1789896473, usage=UsageInfo(prompt_tokens=11813, completion_tokens=780, total_tokens=12593, prompt_audio_seconds=Unset(), prompt_tokens_details={'cached_tokens': 10688})))

Tool calls to empty strings

Here there is a tool call to a function with a name that is an empty string. The arguments field is also malformed, containing only '}'. Other than that, the response doesn't contain anything.

CompletionEvent(data=CompletionChunk(id='092acf180c544b57a364ef231348eed3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content='', tool_calls=[ToolCall(function=FunctionCall(name='', arguments='"}'), id='null', type='function', index=0)]), finish_reason=None)], object='chat.completion.chunk', created=1789896145, usage=None))
CompletionEvent(data=CompletionChunk(id='092acf180c544b57a364ef231348eed3', model='zai-glm-latest', choices=[CompletionResponseStreamChoice(index=0, delta=DeltaMessage(role=Unset(), content='', tool_calls=Unset()), finish_reason='tool_calls')], object='chat.completion.chunk', created=1789896145, usage=UsageInfo(prompt_tokens=425, completion_tokens=3245, total_tokens=3670, prompt_audio_seconds=Unset(), prompt_tokens_details={'cached_tokens': 128})))

Tool calls to malformed function names

I haven't been able to reproduce this for this bug report, but I'm sure it happens. In this case, the FunctionCall object looks something like below. Here, it seems that the name has been incorrectly parsed, leaving an HTML tag as part of the name.

FunctionCall(name='list_workspace_files\n</arg_value>', arguments='{}')

Expected Behavior

I expect well-formed non-empty respones.

Additional Context

No response

Suggested Solutions

No response

[Classic298]

Reproducing this, with the trigger condition and what we know about reasoning_effort.

We hit the "only thinking content" variant (thinking chunks stream, then finish_reason='stop', zero text) repeatedly in an agent harness where GLM 5.3 composes large replies from tool results. Same signature as the report: long thinking phase, then a clean stop with nothing visible.

The trigger in our case: a request where the last message is a tool role result (a full file read, ~20KB) and the model is asked to review/compose from it, with tools present. It fails intermittently in bursts: two separate windows today had repeated silent stops on this exact shape (including inside live agent sessions), while a batch of 12 identical requests outside those windows all completed normally. So it is not per-request deterministic, but the shape is always the same when it hits: 1000+ thinking deltas, then stop with no text. One extra data point on top of the report: the final usage chunk in our failing streams says completion_tokens: 0 while ~23KB of thinking was streamed, so the thinking is not even counted as completion tokens in those turns.

On reasoning_effort: we observed silent stops with "low" set, so lowering the effort does not rule it out. Turns with "high" set were the only ones we did not observe failing, but given the burst pattern we cannot claim that as more than a candidate mitigation. For this model the API accepts low, high, max — a 400 on other values lists exactly those three, while the 422 from schema validation lists a different set (none, minimal, low, medium, high, xhigh, max), so the two validation layers disagree about the enum.

Also matching the report's point 3: we see tool-call deltas arrive in fragments where the first fragment carries id + name + empty arguments and continuation fragments carry name: "" with argument slices. That part is standard OpenAI-style streaming, but a naive accumulator that re-reads name on every fragment gets the empty-string name the report shows — worth ruling out on the serving side whether the fragmented name field is intended, since that is what produces empty-name tool calls in clients that don't take the name only from the first fragment.