Repository navigation
server: split Inkling 2 replies into reasoning, text and tool calls - #4124
ajensenwaud wants to merge 1 commit into
Conversation
|
Thanks @ajensenwaud for the Inkling 2 series (#4124 to #4128). Before we review it, one housekeeping ask that applies to all five: the commits' authors use your GitHub noreply address, but their committer is a personal email address. Our landing checks only accept noreply identities in a commit's history, so please rewrite the committer on each branch, for example with
|
f6ce80c to
2aec728
Compare
|
Done: all five heads are rewritten with the noreply address as committer and force-pushed. The kernel PRs also now use the common CUDA helpers, which fixes the |
2aec728 to
883d011
Compare
What this changes
Inkling 2 replies are typed blocks of reasoning, text and tool calls. The server now splits them by the state table of vLLM's Inkling parser, gives each tool call the request's keyed id, and holds a streamed block header back until the block names its kind.
Without the last two, agents looped: every call came back as
call_0, and streamed tool names built up in the history until the model wrote them as prose. Only tokenizers with Inkling's markers take this path.cfc8510ce.atc overlap --json: incomplete (ATC's limits). Conflicts only inCHANGELOG.mdand one import line inzig/src/server/root.zig, with idol: the octree and Gaussian decoders on Metal, to the 3DGS PLY (stacked on #1766) #1773, idol: the octree and Gaussian decoders on CUDA (stacked on #1773, #1767; not yet run on a GPU) #1858, lanes: LFM3's row draw and Flash Next's host reference draw use core's sampling_row (#2460 item 3, step 2) #2679 and server: --vision-image-share fixed gives every image the same share of the visual-token budget (#2510) #2717.Checks on this head
883d011da; lean checker 168 on the base, 168 on this head, 0 new.zig build test test-golden: 10,925 of 11,016 tests, 91 skipped on mainc94e65d33; rebased ontocfc8510ce, rechecked there: the cluster membership tests pass five clean runs.zig fmt --check build.zig zigandgit diff --check: clean. CI: pending.Receipt
n/a: no family on main serves Inkling 2. Kimi writes its own call ids, so it keeps the old path.
Served probe, two GB10s
A two-turn agent chat with one shell tool, served by the family stack with this parser on 19 November:
call_3841ff1feb3b791e5b054bbb, and no content.call_04810c8943136edb2aebd90b.call_0and the stream sentrun_shellas text.