Skip to content

server: a batch's songs are given as each is ready - #7

Merged
cynodesmus merged 1 commit into
cynodesmus:pr-28-upstream-batch-tracks-readyfrom
timoncool:upstream-batch-tracks-ready
Nov 19, 2026
Merged

cynodesmus merged 1 commit into
cynodesmus:pr-28-upstream-batch-tracks-readyfrom
timoncool:upstream-batch-tracks-ready

Conversation

@cynodesmus

Copy link
Copy Markdown
Owner

ServeurpersoCom#28
A request for several songs (lm_batch_size > 1) decoded every song only after the last one's latents were
solved, so a client waited for the whole batch before it could play the first song.

Now each song is decoded as soon as its latents stand, inside the per-song loop of pipeline_generate (the NAR
is released before the VAE is taken, so a store that keeps one group on the card swaps them per song).
pipeline_generate takes an optional track_ready(void *, size_t track, Yue2Song &) callback; the server
uses it to keep each finished track (replay request, encoded audio, latents) on the job:

  • GET /job?id=N answers {"status":"running","tracks_ready":K};
  • GET /job?id=N&result=1&track=K returns track K alone, in the same multipart as the whole result
    (404 while it is not ready; once the job is done the whole result holds every track and the single ones are
    dropped);
  • GET /job?id=N&result=1 is unchanged.

Callers that pass no callback behave as before, apart from the VAE now running between songs instead of after
all of them.

Tested in YuE2 Studio (Windows, RTX 4090, the same change on our fork): a two-song request at 30 s each — the
first song reached the library through track=0 and played while the second was still rendering.

Ported onto master by hand from our fork, where it runs in YuE2 Studio 4.1.0; not built on master on our side.

Each song of a request is decoded as soon as its latents stand, instead of
after the last song: GET /job names tracks_ready while the job runs, and
GET /job?id=N&result=1&track=K fetches one finished track in the same
multipart as the whole result. pipeline_generate takes a track_ready
callback for it.
@cynodesmus
cynodesmus merged commit b41d67b into cynodesmus:pr-28-upstream-batch-tracks-ready Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants