Singing via the user's NEUTRINO, selected in the score by snakajima · Pull Request #42 · receptron/jinglescript

snakajima · 2026-10-26T17:47:12Z

Summary

Singing through the user's own NEUTRINO install, chosen entirely by the score.

A track with "sing": { "engine": "neutrino", "voice": "reina", "level": 4 } is sung by NEUTRINO inside the ordinary render() / CLI / MCP. No new public API.
NEUTRINO is located, never bundled: NEUTRINO_DIR env var > NEUTRINO_DIR in ./.env > /Applications/NEUTRINO.
Missing NEUTRINO → NeutrinoMissingError; missing voice (default REINA) → NeutrinoVoiceMissingError listing installed voices. No silent fallback. Browser build throws SingerUnavailableError.
level (LU over accompaniment, default 4) sets vocal loudness; vocal is rendered at the mix's sample rate (fixes pitch/tempo drift).
Removes the built-in ONNX singer, its training pipeline, and the onnxruntime-node dependency.
CLAUDE.md: new Design section (extensions via the score, not new APIs; external engines located, errors instead of fallback).
Docs: docs/singing-neutrino.md, README Singing section, .env.example.

NEUTRINO output must not be used for AI training or voice conversion (NEUTRINO EULA). No NEUTRINO files are included.

Tests

733 passed, 100 skipped; typecheck, lint, prettier and smoke pass. NEUTRINO tests use a fake install.

src/singer/neutrino.ts finds NEUTRINO (NEUTRINO_DIR, /Applications/NEUTRINO, <cache>/neutrino), lists its voices, writes the sung track as MusicXML (tempo 120, lead-in bar, legato under 0.1 s, ties over barlines), runs musicXMLtoLabel and neutrino in a cache work dir, trims and resamples the 48 kHz WAV, and caches renders by score+voice+options+version. Falls back to a built-in singer with a warning. Nothing of NEUTRINO is bundled or downloaded. Demo: scripts/sing-neutrino-demo.ts; docs/singing-neutrino.md (Japanese, EULA notes).

…accompaniment The vocal was resampled to 44.1 kHz and mixed into a 48 kHz render, so it ran 8.8% fast and sharp. Default to the render's rate; add vocalLevel (default 4 LU over the other tracks, track gain on top) and vocalGainDb(). The demo starts with a beat of rest so the first consonant is not cut.

…inger - Remove the built-in singer (model/vocoder/frames/post/sing, sing-demo, training/singing, singing-voice-plan, onnxruntime-node peer dep, fallbackModel) and its tests. - A track's `sing: { engine: "neutrino", voice, level }` is rendered by the ordinary render()/CLI/MCP (Node registers NEUTRINO; the browser throws SingerUnavailableError). No separate API. - NEUTRINO_DIR from env, then ./.env, default /Applications/NEUTRINO; .env.example, .env ignored. - NeutrinoMissingError (path + how to set it), NeutrinoVoiceMissingError (lists installed voices); default voice REINA, no fallback. - Docs (ja) with a full score example, README, CLAUDE.md design principle, tests.

snakajima

Reviewed and run locally on 663bddd against a real NEUTRINO install (/Applications/NEUTRINO: MERROW, NAKUMO, REINA, RUNO, SOMA).

What I ran

scripts/sing-neutrino-demo.ts with REINA and MERROW: both sing.
- REINA: -14.0 LUFS, -1.7 dBTP, 1.2 dB limiting.
- MERROW: -14.0 LUFS, -2.2 dBTP; 34.7 s for the first, uncached render.
A five-note solo: with the first beat at 0.5 s, the voice starts at 0.44 s (the consonant comes before the beat). No NaN, DC ≈ 0.
typecheck passes, and the 21 singer/prerendered/browser tests pass.
The user listened to the MP3s; I can't judge the sound myself.

The design fits the new CLAUDE.md: singing is chosen only by sing in the score, NEUTRINO is located rather than bundled, and a missing voice is an error listing the installed ones (checked). The MusicXML conversion is good: notes are split and tied at barlines, and gaps under 0.1 s are sung legato. Measuring the vocal and the accompaniment separately, by prerendering silence, is a neat approach.

Inline comments below. The first two are the ones I'd fix before merging.

snakajima · 2026-10-26T17:56:45Z

+/** The index of the track with `sing`, or undefined; an error for more than one. */
+export function sungTrack(score: Score): number | undefined {
+  const sung = score.tracks.flatMap((track, index) => (track.sing ? [index] : []));
+  if (sung.length > 1) throw new Error(`Only one track can be sung; tracks ${sung.join(", ")} all have "sing".`);


check passes scores that render then fails on. Reproduced with the CLI: a score with sing on two tracks gets ok: 2 s, 2 notes from jinglescript check, then render crashes with this plain Error and a stack trace (it is not in userFacingMessage).

Non-kana lyrics aren't validated either. {"lyric": "la"} goes all the way to NEUTRINO and fails with neutrino failed: Command failed: … and Failed to load phrase data.

Both belong in expandScore issues (a JingleScriptError with a JSON path and a hint), so an LLM can fix the score at check, before NEUTRINO runs. kana.ts already has what's needed for the lyric check. The same check could decide whether several morae on one note ("かえる") are allowed. That is currently passed through and sung (no error).

snakajima · 2026-10-26T17:56:45Z

+
+/** NEUTRINO's PCM (mono, `sampleRate`, from score time 0: the lead-in removed). */
+export function alignNeutrinoPcm(data: Float32Array, rate: number, sampleRate: number): Float32Array {
+  const skip = Math.round(NEUTRINO_LEAD_IN * rate);


A note at at: 0 loses its consonant and starts with a click. NEUTRINO places the consonant before the beat, and this trims the lead-in at exactly 2 s. A score of さくらさ starting at at: 0 renders with |x| already 0.26 within the first 20 ms, with no fade. The demo avoids this with a beat of rest, but an LLM will write at: 0. Either apply a few ms of fade-in at the cut, or warn in check when a sung note starts before the consonant fits (~0.1 s).

snakajima · 2026-10-26T17:56:45Z

+
+const defaultExec: NeutrinoExec = (file, args, env, cwd) => {
+  try {
+    execFileSync(file, [...args], { env, cwd, maxBuffer: 64 * 1024 * 1024, stdio: ["ignore", "ignore", "pipe"] });


execFileSync blocks the whole process while NEUTRINO runs: about 35 s for a first MERROW render here, during which the MCP server answers nothing. render() is sync, so this is hard to avoid for now, but it should be stated in docs/singing-neutrino.md (and ideally in the MCP tool description).

snakajima · 2026-10-26T17:56:45Z

+ * dB to scale the vocal `pcm` of `track` by so it sits `level` LU over the other tracks (plus the
+ * track's own gain); 0 when either side is silent.
+ */
+export function vocalGainDb(score: Score, track: number, pcm: Float32Array, options: RenderOptions, level: number): number {


Minor: levelling renders the whole score two more times (vocal only, accompaniment only), on top of the final render. That's fine for jingles, but noticeable with grandpiano samples or long scores; worth a comment.

snakajima · 2026-10-26T17:56:45Z

+  const root = join(options.cacheDir ?? defaultCacheDir(), "neutrino-renders");
+  const key = neutrinoCacheKey(xml, voice, options, install.version);
+  const file = join(root, `${key}.wav`);
+  if (existsSync(file)) return { wav: readFileSync(file), cached: true };


Minor: neutrino-renders/ only grows. Each edit of a sung score leaves another WAV. Consider pruning by age/size, or documenting that the folder is safe to delete.

snakajima · 2026-10-26T17:56:45Z

+ * The NEUTRINO folder: NEUTRINO_DIR from the environment, else from `envFile` (.env in the current
+ * directory), else DEFAULT_NEUTRINO_DIR. The .env file is only read, never loaded into process.env.
+ */
+export function neutrinoDir(env: NodeJS.ProcessEnv = process.env, envFile = join(process.cwd(), ".env")): string {


Minor: NEUTRINO_DIR comes from .env in the current directory and points at binaries that get executed. Rendering a score with sing inside an untrusted checkout runs whatever its .env points to. A sentence in the docs would do.

snakajima · 2026-10-26T17:56:45Z

+const TEMPO = 120;
+const DIVISIONS = 480;
+const MEASURE = DIVISIONS * 4;
+/** A bar of rest before the first note so its consonant can start early; trimmed from the output. */


Nit: this JSDoc belongs to NEUTRINO_LEAD_IN (line 31), but it now sits stacked on top of LEGATO_GAP's.

snakajima · 2026-10-26T17:56:45Z

  start: number;
  end: number;
  midi: number;
  /** One mora of hiragana or katakana, "ー"/"_" to hold the previous vowel, or "" for a rest. */


Nit: in timelineOf, "" right after a sung note is sung as "ー" (holding the vowel), not a rest; it is a rest only when nothing is sounding. The comment should say so (the schema's lyric description too, if it says the same).

- check/parse reject >1 sung track and non-kana / multi-mora lyrics on a sung track (JingleScriptError with path + hint); render validates before NEUTRINO runs - warn in check when a sung note starts before 0.1 s; 5 ms fade-in at the lead-in cut - kana: accept くぁ/ぐぁ - prune neutrino-renders (30 days / 200 files); document it is safe to delete - document blocking execFileSync (docs, MCP tool, schema), levelling's extra renders, .env NEUTRINO_DIR risk - fix NEUTRINO_LEAD_IN JSDoc placement and the "" lyric (hold vs rest) comment/schema

snakajima added 3 commits October 26, 2026 07:16

snakajima commented Oct 26, 2026

View reviewed changes

snakajima merged commit cff7ce4 into main Oct 26, 2026
10 checks passed

This was referenced Oct 26, 2026

NEUTRINO render cache: cache hits don't refresh age; stale work-* folders never cleaned #43

Open

render() is async on a worker thread; the player shows before the audio (1.0.0) #44

Merged

Navigation Menu

Repository navigation

Singing via the user's NEUTRINO, selected in the score - #42

Singing via the user's NEUTRINO, selected in the score#42
snakajima merged 4 commits into
mainfrom
singer-neutrino

snakajima commented Oct 26, 2026

Uh oh!

snakajima left a comment

Uh oh!

snakajima Oct 26, 2026

Uh oh!

snakajima Oct 26, 2026

Uh oh!

snakajima Oct 26, 2026

Uh oh!

snakajima Oct 26, 2026

Uh oh!

snakajima Oct 26, 2026

Uh oh!

snakajima Oct 26, 2026

Uh oh!

snakajima Oct 26, 2026

Uh oh!

snakajima Oct 26, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

snakajima commented Oct 26, 2026

Summary

Tests

Uh oh!

snakajima left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant