Repository navigation
voice-agents-python: the TypeScript runtime holds frames during a resume, fails lost tool calls, contains a throwing handler - #4
Conversation
…lost tool calls, contains a throwing handler
|
Superseded by the render on voice-agents/render-9269369, which carries this change and the lane's later ones. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| if self.ended: | ||
| raise SessionEnded(self.id) |
There was a problem hiding this comment.
🟡 (optional) Audio agents will now log a "a handler failed" traceback at the end of most calls. Any code after send_audio in the handler is also skipped. send_audio now raises SessionEnded once ended is true. Audio already queued in the session inbox still reaches on_audio after the reader has set end or lost. The repo's own conformance agent (conformance/main.py:63) does not catch it. Fix: handle the end of the call in every send_audio caller. Either drop audio silently after the end, as the resume hold already does, or keep the raise and catch SessionEnded in the conformance on_audio and in the README guidance.
Why this was flagged
The trigger is an audio session whose session.end arrives, or whose session is lost, while caller audio frames are still queued in session._inbox. Each 20 ms frame is its own item, so a backlog is common. agent.py:305-307 puts the SessionEnd in the inbox and then calls session._finish, which sets end at once. _lost at agent.py:350-353 does the same through _abandon, which sets lost. _run_session (agent.py:393-394) then delivers the earlier audio items to on_audio. The conformance agent's on_audio calls session.send_audio(samples) with no try (conformance/main.py:62-63), so the new check at session.py:94-95 raises SessionEnded. _safely (agent.py:418-419) logs it with log.exception("a handler failed"). On the base branch send_audio never raised SessionEnded. The bytes went to send_bytes, which swallows a closed socket. So existing handlers, including this repo's conformance agent, which catches SessionEnded only around say/tool, never expected it. Nothing in the diff updates those callers or the README's send_audio(samples) description.
Verification: Trigger: in an audio session, session.end arrives or the session is lost while caller audio frames are still queued in session._inbox. The new guard at the top of send_audio (session.py:94-95) is if self.ended: raise SessionEnded(self.id). On the base, send_audio had no ended check. sentdm_voice_agents/conformance/main.py:62-63 has no except SessionEnded.
| session._resuming = False | ||
| connection.sessions.pop(session.id, None) | ||
| if frame.error.code == "connection_full": |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: Pre-existing: after a connection_full refusal, agent audio can still go to another caller. The new resume guard does not cover this case. _error clears _resuming but leaves session._connection on the full connection, so send_audio and _send still transmit there with the old stream number. Fix: keep the session marked detached (for example _resuming stays True, or a separate flag) from the refusal until session.resumed arrives on the new connection, so both audio and frames are held.
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
The trigger is Sent answering a session.resume with code connection_full. agent.py:337 sets session._resuming = False and agent.py:341 opens a new connection, but session._connection still points at the connection that refused it (set at agent.py:279). Until the new connection is ready, session.py:98 no longer drops audio. send_audio sends audio_frame(self.stream, ...) on the full connection. self.stream belongs to the old, lost connection and can match another caller's stream there. That is the hazard the new comment at session.py:99-100 describes. Counted frames are also sent there at session.py:146 instead of waiting for the replay. The base branch has the same behaviour. The diff's guard only covers the time a resume is unanswered and misses this window.
Verification: Pre-existing. _error (agent.py:337-342) sets session._resuming = False and calls self._open([session], 0.0). It never detaches session._connection, so the session still points at the full connection. During that gap, the new guard in send_audio (session.py:98, if self._resuming: return) does not fire. On the base, send_audio had no _resuming check at all.
Rendered by the Voice Agents lane in sent-dm-sdk-pipeline from sent-dm-voice-protocol at
v0.2.0, after https://github.com/sentdm/sent-dm-sdk-pipeline/commit/b62925299e2c1785890cb4dccd14822b712a4cf5.What a reviewer checks: this repository's CI, the conformance job included, is green on this PR; the generated frames module and the runtime changed as the protocol or the template did; nothing release-please owns moved. Merging makes release-please open the release.