Repository navigation
render() is async on a worker thread; the player shows before the audio (1.0.0) - #44
Merged
Merged
Conversation
A long render (a piano piece, a NEUTRINO vocal) no longer blocks the caller's event loop: render() hands the work to a small pool of worker threads (src/render-pool.ts, src/render-worker.ts), registered by the Node entry, and resolves with the same bit-identical result. The browser entry renders in place. renderNow() is the synchronous core the worker runs (internal). - Samples go to the worker as data (loadSamples' sources can list theirs). - Errors keep their class across the thread (instanceof works for callers). - Only RenderOptions' own fields cross; a field added there is a type error until the pool passes it on. - Breaking: every render() call needs await. Tests, smoke, docs, README and the playground worker updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
manage()'s renderScore takes `deferAudio`: with a player, it waits at most DEFER_WAIT_MS (1 s) for the audio, and otherwise answers at once with a `pending` PlayerData (score, lanes, cues, lyrics, sheet music; no audio) while the render goes on. awaitDeferredPlayer(id) gives the finished PlayerData, the reason it failed, or "gone" (kept 30 min after it finishes, in memory). The GUI plugin sets it when the host's context says `deferredAudio: true` (its view has a runtime). The view shows a pending card at once with play disabled, collects the audio through the host's dispatch (a request answered by the same execute, never sent by the LLM), saves it with onUpdateResult, and then plays, autoplaying when asked (fresh from when the audio was ready). A render that fails after the card appeared is shown and, if fresh, told to the LLM once. Hosts that do not set it get the finished card as before. The demo page gains a pending card and a stand-in runtime; checked in a browser. Both packages go to 1.0.0 (render() returns a Promise). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ire on time - A pending card that unmounts before its audio arrives leaves the reply to the mount that shows the card now: it no longer saves, autoplays (which would mark the card autoplayed with no audio element) or reports a failure. - A finished deferred render is dropped DEFERRED_KEEP_MS after it settles by an unref'd timer, instead of only when another render or request came along. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Where render() runs in place (Node 22.12 running src/, no type stripping, so no worker) and ffmpeg is absent, a short render finished before even a 0 ms timer, so the test expecting a pending player got the finished one. waitMs 0 now means "do not wait": the answer is always pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tests/deferred-renders.test.ts: resolve the pending render without a placeholder function (sonarjs/no-extra-arguments). - gui-plugin tests: the two that render a 42 s score get 60 s; on a CI runner the render and its ffmpeg encode took over the 5 s default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t fallback - CLAUDE.md: only the GUI plugin may lean on its host (opt-in, as deferredAudio); as a library (MulmoCast calls render()) nothing may. manage()'s carrier options and awaitDeferredPlayer are allowed as API: they are not score features. - Deferred renders: at most the newest 20 finished ones are kept, a collected one only a minute more; renders still running are always kept. - The pool says once (process warning JINGLESCRIPT_NO_RENDER_WORKER) when no worker can run and render() works on the calling thread. - manage() checks NEUTRINO and the voice before rendering (checkSinger), so a missing install reaches the LLM even when the audio would be deferred. - Sung renders run one at a time in the pool (NEUTRINO uses every core). - renderNow expands the score once; render() no longer expands it first. - Tests on the deferred path no longer depend on how fast the machine is. Not changed: samples are still copied to the worker per render (tens of MB at most for a grand piano piece, small next to the render; a cache would have to tell apart callers that give different data under one key). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…appeared A deferred render that failed more than FRESH_MS after renderScore (an uncached vocal, a render queued behind others) was never reported, so an LLM waiting for a message (notifyWhenDone) waited forever. failedMessage now judges freshness by when the card appeared, not when the render failed: a card replayed from history still says nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
render()no longer blocks the caller, and the GUI plugin shows its card before a long render's audio is ready. Both packages go to 1.0.0, becauserender()now returns a Promise (breaking).1.
render()returns a Promise and renders on a worker thread (Node)src/render-pool.ts/src/render-worker.ts: a small pool of worker threads (at most 2), reused between renders and unref'd while idle. The Node entry,renderToFilesandmanageregister it. NEUTRINO runs inside the worker, so an MCP server or GUI host keeps answering during a long piano piece or an uncached vocal.renderNow, the internal synchronous core).loadSamples' sources can list what they hold. A caller's ownSampleSourceis asked for exactly the keyssamplesNeededreports.JingleScriptError,SamplesNotLoadedError,Neutrino*Error, …), soinstanceofstill works in the CLI andmanage.RenderOptions' own fields cross the thread. A field added there is a type error until the pool passes it on, and anything else on the options object (e.g.renderToFiles'onDownloadcallback) stays behind.JINGLESCRIPT_NO_RENDER_WORKER): a bundle that leftrender-worker.jsbehind, orsrc/on a Node that cannot strip types. The browser entry always renders in place. Sung renders run one at a time (NEUTRINO uses every core).render()call needsawait. Tests, smoke, README, docs, eval and the playground worker are updated.2. The card before the audio (GUI plugin)
manage()'sdeferAudiooption, used only withplayer: renderScore waits at mostDEFER_WAIT_MS(1 s) for the audio. A short jingle comes back finished, as before. Otherwise it answers at once with apendingPlayerData (score, lanes, cues, lyrics, sheet music; no audio).awaitDeferredPlayer(id)gives the finished one, why it failed, orgone(src/deferred-renders.ts). A finished render is kept in memory until a view collects it (then 1 min more, for another mount of the card), at most 30 min, and only the newest 20. NEUTRINO and its voice are checked before rendering (checkSinger), so a missing install always reaches the LLM.The plugin turns this on only when the host passes
deferredAudio: truein its execute context. The view then:dispatch, as a{ kind: "collectRenderedAudio", id }request thatpluginCore.executeanswers before treating the call as a tool call;onUpdateResult;If the audio fails after the card appeared, the card says why and, for a fresh render, tells the LLM once. Hosts that don't set
deferredAudiobehave as before.gui-chat-protocol/vueis now external in the plugin build, so the view injects the host's runtime by the protocol's ownPLUGIN_RUNTIME_KEY.CLAUDE.md: only the GUI plugin may lean on its host, and only as an opt-in. Used as a library (MulmoCast calls
render()), nothing may depend on the host.To enable this, MulmoTerminal needs two changes (not in this PR; see gui-plugin README): wrap the view with
wrapWithPluginRuntime, and addmanageJingleScript: { deferredAudio: true }toEXTRA_CONTEXT_BY_TOOL.Testing
scripts/smoke.sh(pack, install, render via CLI, library, browser entry, MCP) passes.Related: #43 (NEUTRINO render cache cleanup, follow-up from #42).
🤖 Generated with Claude Code
work in jinglescript