Repository navigation
[PR #25] generate: clamp a stage's budget to the room left in the context instead of refusing the request - #4
Merged
cynodesmus merged 1 commit intoNov 19, 2026
Conversation
…fusing the request; report a truncated track in its replay JSON
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ServeurpersoCom#25
A request whose prompt, score and frame budget do not fit the context together is refused before the long stage:
[AR] FATAL: prefix N + budget M + end exceeds context C. With a duration used as a ceiling ("a limit, not a target", ServeurpersoCom#17)this turns a setting that only bounds the song into a hard error: a 10-minute ceiling fails on a context reduced with
--max-seq, although the song itself would have ended minutes earlier.minimaxmusic.cpp already handles the same case by shrinking the budget (
[AR] Frame budget clamped to N by the KV cache),and #3 describes the result as wanted: the stage "ends by truncation, but the process can continue". This PR does the same here:
yue2_generate: a budget the context cannot hold after the prefix shrinks to the room left and the stage goes on;the stream is marked truncated when it reaches that budget. The request is still refused when the room is below the
stage's
min_tokens(the prompt and the score alone do not leave a song).[AR] Frame budget clamped to N by the KV cache (prompt M tokens)(Score budget ...for the score stage).yue-server: a track whose stage hit its budget before the end token (the duration's or the context's) carries"truncated": truein its JSON replay part, so a client can tell the person the song was cut rather than finished.The request parser ignores unknown keys, so resubmitting that part is unchanged.
Tested on an RTX 4090, Q8_0, Windows, CUDA 13 (on our fork's branch, the same change on top of this master):
--max-seq 5120 --no-fa, a planned score, prompt 1582 tokens,duration240:Frame budget clamped to 3537 by the KV cache, a 141.5 s song in 3 acoustic chunks,"truncated": truein its JSON part (before: the request was refused);--max-seq 8192,duration600:Frame budget clamped to 6656 by the KV cache, the song ended by itself at 152 s;duration60, the same seeds: the same semantic tokens as before the change, no clamp line.A CPU build of this branch compiles clean (MSVC, Ninja).