Skip to content

server: split Inkling 2 replies into reasoning, text and tool calls - #4124

Open
ajensenwaud wants to merge 1 commit into
ashhart:mainfrom
ajensenwaud:ink-reply
Open

ajensenwaud wants to merge 1 commit into
ashhart:mainfrom
ajensenwaud:ink-reply

Conversation

@ajensenwaud

@ajensenwaud ajensenwaud commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

Inkling 2 replies are typed blocks of reasoning, text and tool calls. The server now splits them by the state table of vLLM's Inkling parser, gives each tool call the request's keyed id, and holds a streamed block header back until the block names its kind.

Without the last two, agents looped: every call came back as call_0, and streamed tool names built up in the history until the model wrote them as prose. Only tokenizers with Inkling's markers take this path.

Checks on this head

  • Head 883d011da; lean checker 168 on the base, 168 on this head, 0 new.
  • zig build test test-golden: 10,925 of 11,016 tests, 91 skipped on main c94e65d33; rebased onto cfc8510ce, rechecked there: the cluster membership tests pass five clean runs.
  • zig fmt --check build.zig zig and git diff --check: clean. CI: pending.
  • The three route tests fail with the fix reverted and pass with it.

Receipt

n/a: no family on main serves Inkling 2. Kimi writes its own call ids, so it keeps the old path.

Served probe, two GB10s

A two-turn agent chat with one shell tool, served by the family stack with this parser on 19 November:

  • The whole and streamed replies return the same call, id call_3841ff1feb3b791e5b054bbb, and no content.
  • The second turn returns a new id, call_04810c8943136edb2aebd90b.
  • Before the fix, every turn returned call_0 and the stream sent run_shell as text.

@ashhart

ashhart commented Nov 20, 2026

Copy link
Copy Markdown
Owner

Thanks @ajensenwaud for the Inkling 2 series (#4124 to #4128). Before we review it, one housekeeping ask that applies to all five: the commits' authors use your GitHub noreply address, but their committer is a personal email address. Our landing checks only accept noreply identities in a commit's history, so please rewrite the committer on each branch, for example with git rebase --exec 'git commit --amend --no-edit --reset-author' origin/main after setting user.email to your noreply address, then force-push. Nothing else needs to change; we will start the reviews as soon as the heads move.

  • Synthena

@ajensenwaud

Copy link
Copy Markdown
Contributor Author

Done: all five heads are rewritten with the noreply address as committer and force-pushed. The kernel PRs also now use the common CUDA helpers, which fixes the cuda_common_check failure; the SASS is unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants