Skip to content

generate: clamp a stage's budget to the room left in the context instead of refusing the request - #25

Open
timoncool wants to merge 1 commit into
ServeurpersoCom:masterfrom
timoncool:semantic-budget-kv
Open

timoncool wants to merge 1 commit into
ServeurpersoCom:masterfrom
timoncool:semantic-budget-kv

Conversation

@timoncool

Copy link
Copy Markdown

A request whose prompt, score and frame budget do not fit the context together is refused before the long stage:
[AR] FATAL: prefix N + budget M + end exceeds context C. With a duration used as a ceiling ("a limit, not a target", #17)
this turns a setting that only bounds the song into a hard error: a 10-minute ceiling fails on a context reduced with
--max-seq, although the song itself would have ended minutes earlier.

minimaxmusic.cpp already handles the same case by shrinking the budget ([AR] Frame budget clamped to N by the KV cache),
and #3 describes the result as wanted: the stage "ends by truncation, but the process can continue". This PR does the same here:

  • yue2_generate: a budget the context cannot hold after the prefix shrinks to the room left and the stage goes on;
    the stream is marked truncated when it reaches that budget. The request is still refused when the room is below the
    stage's min_tokens (the prompt and the score alone do not leave a song).
  • The log names it: [AR] Frame budget clamped to N by the KV cache (prompt M tokens) (Score budget ... for the score stage).
  • yue-server: a track whose stage hit its budget before the end token (the duration's or the context's) carries
    "truncated": true in its JSON replay part, so a client can tell the person the song was cut rather than finished.
    The request parser ignores unknown keys, so resubmitting that part is unchanged.
  • README and ARCHITECTURE describe both.

Tested on an RTX 4090, Q8_0, Windows, CUDA 13 (on our fork's branch, the same change on top of this master):

  • --max-seq 5120 --no-fa, a planned score, prompt 1582 tokens, duration 240: Frame budget clamped to 3537 by the KV cache, a 141.5 s song in 3 acoustic chunks, "truncated": true in its JSON part (before: the request was refused);
  • --max-seq 8192, duration 600: Frame budget clamped to 6656 by the KV cache, the song ended by itself at 152 s;
  • the full context, duration 60, the same seeds: the same semantic tokens as before the change, no clamp line.

A CPU build of this branch compiles clean (MSVC, Ninja).

…fusing the request; report a truncated track in its replay JSON
cynodesmus added a commit to cynodesmus/yue2.cpp that referenced this pull request Nov 19, 2026
[PR ServeurpersoCom#25] generate: clamp a stage's budget to the room left in the context instead of refusing the request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant