feat(realtime): bundle initial_state into livekit_join

[VerioN1]

What

Send the initial reference image/prompt inside the livekit_join message as initial_state, instead of as a separate set_image/prompt request fired after livekit_room_info arrives.

Wire change:

// before: two messages
{ "type": "livekit_join" }
{ "type": "set_image", "image_data": "…" }   // sent after room info

// after: one message
{ "type": "livekit_join", "initial_state": { "type": "set_image", "image_data": "…" } }

The inference server now gets the reference image/prompt with the join — no extra client round-trip waiting on livekit_room_info.

How

• LiveKitJoinMessage gains an optional initial_state field.
• buildInitialStateRequest() produces a single reusable { message, matchAck } from the caller's initialState.
• request() gains a write? flag so the ack can be armed without re-sending the payload. The initial-state ack is still awaited after room info, so the readiness gate behaves exactly as before — only the message's position on the wire moved.
• Removed the now-unused sendInitialState/sendInitialStateTracked two-step path.

Scope

Isolated to the wire format: signaling-channel.ts, the LiveKitJoinMessage type, and the affected unit tests. No error-verbosity, playground, or media-channel changes from the source branch were carried over.

Test

vitest run unit — 210 passed. tsc --noEmit and biome check clean.

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes the realtime signaling wire contract and handshake timing; server must understand bundled initial_state, though client-side connect gating behavior is intended to stay the same.

Overview
Realtime join now sends reference image/prompt inside livekit_join as initial_state (or null as a capability marker), instead of a follow-up set_image / prompt after livekit_room_info.

buildInitialStateRequest() maps caller InitialState to the nested wire message; request() gains optional write: false so the client can wait for the ack without resending. Early set_image_ack / prompt_ack messages are buffered until after room info arms the waiter, preserving queue-timeout behavior and the existing initialStateAck readiness gate.

^{Reviewed by Cursor Bugbot for commit 521d5cb. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Observability marks failed ack success

Low Severity

In flushInitialState, endPhase("initial-state-handshake", { success: true }) runs before checking ack.success. When the server returns a failed set_image_ack or prompt_ack, telemetry still records a successful initial-state handshake even though the method then throws.

 

^{Reviewed by Cursor Bugbot for commit fc7cc3e. Configure here.}

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@decartai/sdk@155

commit: 521d5cb

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f49f547. Configure here.}

[cursor]

Stale buffered acks poison requests

Medium Severity

While connected is still false, every set_image_ack and prompt_ack is appended to bufferedAcks, but leftover entries are not cleared after the initial-state handshake. Later request() calls (including setImage) resolve from the first buffered match and can skip writeMessage, so a duplicate or unrelated early ack can make a later image send appear successful without hitting the wire.

Additional Locations (1)

• packages/sdk/src/realtime/signaling-channel.ts#L341-L346

 

^{Reviewed by Cursor Bugbot for commit f49f547. Configure here.}