Pocket TTS small form: the K-quant lane, a 75 MB English file, storywish on one voice, the parrot example - #3992
Conversation
…ight format with no kernel behind it - the named groups (the backbone's attention projections and FFN matrices, the frame input projection, the speaker projection, the flow head, the codec) round through ggml's own quantizer (the built llama.cpp's libggml-base through ctypes, the exact blocks a K-quant plane would carry) and back to f32 before they store in the file's usual Q8_0 or f16 form, so the engine's existing lanes and the 200-sentence rig score the format's loss; a width the format's block does not divide is left alone and reported (the codec's convolutions and the head's 32-wide input projection stay under a 256-block format). The files are local experiments and need a --name; pocket.fake carries the spec. The ladder it ran on the English file (alba, 200 sentences, the q8 baseline 3.91 / 4.328): the backbone at Q6_K 4.09 / 4.327, the FFN alone at Q4_K 4.13 / 4.301, all four matrices at Q4_K 3.91 / 4.295, at Q4_0 3.73 / 4.284, at IQ4_XS 4.36 / 4.326 with seven percent more audio, at Q3_K 4.50 / 4.205; on the Q4_K backbone the head at Q8_0 4.09 / 4.281, at Q4_K 3.91 / 4.259, the codec transformers at Q4_K 3.68 / 4.309. The ruling: backbone Q4_K, head Q8_0, codec transformers Q4_K. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ble) and the strided group (the codec's strided, transposed and resampling convolutions the file keeps f16, rounded per output channel over their cin x k taps; the served convs stay in the codec group), and the f16 rungs on the ruled configuration (backbone Q4_K, head Q8_0, codec transformers Q4_K: 3.86 / 4.267 against the q8 file's 3.91 / 4.328): the embedding table at Q8_0 4.27 / 4.271, at Q4_K 4.00 / 4.262; the strided convolutions at Q8_0 3.86 / 4.257, at Q4_0 3.73 / 4.127 - the one rung the waveform side refuses Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nsformers' matrices, codecconv the served 32-wide convolutions, so the recipe the rig scored (the transformers at Q4_K, the served convolutions at Q8_0, the strided ones at Q8_0, the embedding at Q4_K) mints as one file; three sentences in alba and in bill_boerst through it and through the q8 file were played side by side Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ec encoder out, and storywish reads through one: convert_pocket.py stores each roster voice as voice_latents.<name> (the clip through the package's own codec encoder, 16 KB a voice where a clip was a megabyte), takes --voices for the roster and --no-cloning to leave the encoder's 43 tensors out with pocket.cloning = false; the loader reads either roster form (the older clip form still encodes on first use, and a clip in a file without the encoder is refused by name), builds a stored voice's state through voice_state_from_latents (the clip path's second half), reports cloning = false in caps() for such a file and refuses tts_register_voice on it. test_pocket_no_encoder holds the shipped shape (pocket-tts-en-bill-q8.gguf, one voice, 121 MB at q8: caps, one spoken line, the refusal); the pocket suite 10/10. Storywish reads through that file - no packs, no phoneme step, bill_boerst - on the desktop (its smoke 8/8 tells and reads the story) and in the shell's argument; models.json gains a files list for a GGUF that is its own served form, which mint_models.py copies as it does the packs; the examples folder's architecture doc, the card and the deploy's comment say Pocket. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nes, and the settled recipe mints as a real file. convert_pocket.py --kq writes the backbone's and the codec transformers' matrices and the text embedding as Q4_K through ggml's own quantizer and the flow head as Q8_0, the rest as the q8 form does; TtsLinear gains a K-quant plane pair (kq_fmt names the kernel-layer format, kq_repacked whether the active backend's layout holds them; the serializer, the blob walk and the teardown grew with it), linear_rows_decode takes the engine's K-quant GEMV over a row requantized to the Q8_K form, linear_rows the batched kq GEMM where the backend carries the tile and the GEMV per row elsewhere, linear_vec routes a quantized vector layer to the decode entry, and the loader keeps the three lanes switchable: unpinned, a Q4_K tensor comes in through gguf_transcode_q4k as its own planes; pinned q8 or f32 it dequantizes into that lane, and a Q8_0 head matrix takes its blocks on the q8 lane. test_pocket_kq_file holds the lanes to each other on the real file (the kq lane against the q8 lane of the same file 2.6e-2 rms relative; the f16 reference 1.0e-1, the format's loss the rig gates) and the exact lane speaking; the rig on pocket-tts-en-kq.gguf's native lane reads WER 3.86 / UTMOS 4.295 / RTF 0.049 against the published q8 file's 3.91 / 4.328 / 0.051. Storywish reads through pocket-tts-en-stuart-kq.gguf (stuart_bell, no encoder, 65 MB); the pocket suite 11/11, its smoke 8/8, the wasm64 page rebuilt and its set restaged (a story read at RTF 0.66 on the portable tier, the codec decoder 60 percent of it); the architecture doc's file section carries the three lanes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s back-forward cache: Back to another page and Forward brought storywish back with its workers and the audio output frozen mid-frame and out of step, and the first sound was whatever the output ring held; the shells of storywish and storyteller start over on a persisted pageshow (the gate again, a click, a clean start), and the examples folder's architecture doc says why Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nd talk, Silero VAD ends the take two seconds after you go quiet (or the stop button, or the model's 60 s cap), Pocket TTS clones the voice from the take on the speech thread, and the text in the box (Frost's "Stopping by Woods on a Snowy Evening" to begin with, or whatever you type) is read aloud in it on the say button; recording again replaces the voice, and the page says in plain words that the recording stays in the tab. main.das is storywish's shape: one source for the desktop run and wasm64, archived Ask/Answer records over two streams (the take's PCM rides the record; the thread answers a say with its chunk count before the first clip, so the frame thread can tell the last clip from a pause), polled keys and a polled mouse against the buttons' rectangles (the cursor scaled from window points to framebuffer pixels), the microphone at the model's own 24 kHz drained per frame with a 16 kHz copy for the VAD. --clip clones from a file instead of the microphone: test_parrot.das spawns the example under --smoke with the tree's own jfk_ask_not.wav and reads the witness lines (cloned, said in voice you, read out); registered in the stocked suite and the tts + audio areas (the tts plan pins 11 -> 12). models.json carries the Pocket file with its codec encoder (pocket-tts-en-kq.gguf, 75 MB) and a new `tree` list for a file the repository itself ships (silero_vad.bin by repo-relative path and sha256), which mint_models.py copies from the checkout; a set with no image carries the version the deploy expects. The examples folder's rule doc admits the tree form, its architecture doc gets the charter and a section on the take, the deploy loops and the site's shell test cover the third example, the card sits on examples.html with the parrot poster. Found by the browser preview: dasAudio's AudioWorklet non-blocking patch left the CAPTURE descriptor at the requested sample rate while a capture AudioContext runs at the browser's 48 kHz, so a 24 kHz take arrived as 48 kHz frames counted as 24 kHz - twice as long and an octave down (a 9 s take cloned as 16 s). The patch now reads the context's own sampleRate into both descriptors and the device layer resamples; block 6 covers a pristine 0.11.25 header, block 6b a tree the previous script already patched, the guard keys on the new marker, and the two paths converge byte for byte. The patch script is a configure dependency of dasAudio's CMakeLists now, so a change to it reaches an already-fetched header. Verified in Chrome on the staged page with a synthetic microphone (the JFK clip as a MediaStream): the take clones as 8.4 s and the poem reads in the clone at RTF 0.65. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…the model card's Files table, its use and provenance paragraphs and the licence row carry pocket-tts-en-kq.gguf (75 MB, the encoder and 19 voices, the browser examples' file) and pocket-tts-en-stuart-kq.gguf (65 MB, one voice, no encoder), publish_tts_hf.py lists them (a dry run stages both with their sidecars, hashes verified against the card). Ledger row 124 now records the small-format ladder that settled the recipe and what is left below Q4_K; row 128 is Ogg Opus in load_audio_mono (miniaudio's Ogg arm is Vorbis, the voices-dir help says "ogg"). The dead second block-alignment guard in gguf_transcode_q5_0_to_q8 is gone (the line above it already checks expect_n). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…a box below the pen position while the glyph quads rise above it, so a click on the visible text missed; a button now carries its label's own box (quads_dim at the text size) and answers inside it with a margin. In the browser the surface is the document viewport and the page's stage sits below the nav, so the picture is letterboxed; Emscripten maps a click through the canvas element's box with one ratio per axis, which is exact only when that box is the picture - the shell sizes the canvas element to the letterboxed box (max-width / max-height on the replaced element) instead of stretching it over the stage with object-fit. Verified on the staged page with mouse events at the labels: record, stop and say each fire. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…owser's microphone request and writes its outcome over the picture's corner - the device the browser handed the program, or the refusal's reason - and a page a browser opened by address rather than on localhost or https, which has no microphone API at all, gets a stand-in that refuses with that reason instead of the program's request throwing in a callback. A take with no speech in it now reports the level it saw: silence throughout points at the microphone being refused or off, a low peak at a quiet room. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…9630af1cba2f6f7946efa7b8ea194b9b093dc9 carries pocket-tts-en-kq.gguf and pocket-tts-en-stuart-kq.gguf beside the set, TTS_HF is pinned there, and the model-set table gains their rows with their recipes (the --kq form, and its one-voice no-encoder twin); the fetch verifier reads both back at the pinned commit with the table's size and sha256, and the server's catalog test counts the served set at fourteen. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s into take.das, a pure module the test file reaches without a window: the continuous 24 kHz to 16 kHz resampler the VAD reads (a chunked feed equals the one-shot feed), the take-end rules (the button, two seconds of quiet after speech, six seconds with no frames at all), the clip window padded a quarter second around the speech and capped at sixty seconds, and the text box's typing rules; main.das drains the mic before it stops it, logs the recorder's overflow count, refuses a take while a clone runs, decodes a --clip before the window opens so a bad file refuses in one second, and cancels a running say when a take starts or the window closes. The engine: linear_rows_add_bias gets its scalar tail (a width off the float4 grid read past the row), linear_take_kq releases the f32 weight it replaced, kq_gemv_row quantizes its row in place (the module-global scratch row is gone), read_linear's K-quant branch is one guard and the load log names the kq arm, the repack stage is decided once per load, and read_roster, the frame-step check, ensure_voice_state and pocket_encode_latents on a file without the encoder panic by name; gguf_transcode_q8_0 refuses an unaligned width. convert_pocket.py: the speaker group matches flow_lm.speaker_proj_weight, --fake refuses a group named twice or a spec with no '=', the report reads the group each tensor was rounded under and refuses a --fake group that matched nothing, a --kq file's general.name says Q4_K; test_convert_pocket.py holds nine of those on numpy alone, wired into the extended checks. mint_models.py refuses an empty list and a zero-file stage. The tests: test_tts_blocks holds the kq rows and decode kernels to the leaf per row at 512x96 and 768x64 in both the repacked and the disk order, the bias tail at 256x6, an added poison, and the width refusal; test_tts_pocket splits the kq cell into the lanes (kq against q8 with a poisoned expectation, kq against the exact lane, the head over frames against the dump, the stored roster's latents against the f16 file's own encoder) and the clone over the roster (nineteen voices stay, alba speaks); test_parrot holds the resampler, the take rules and the text box model-free and the bad --clip refusal in sixty seconds. The rule documents answer the round: REVIEW.md's charter clause names a weight format and a serving lane, REVIEW_TTS.md splits the [hot_path] rule from the text front-end ban and drops the nolint clauses, REVIEW_PLACEMENT.md gives the caller-guard rule its own line, tests/REVIEW.md spells its loader list out, performance/REVIEW.md's companion rule reads in both directions, examples/dasLLAMA/REVIEW.md's canvas and bfcache rules say what the shell does; the architecture docs carry the latents charter, the three lanes and the take rules. PERF_LEDGER.md takes the small form's row (RSS and RTF, the kq file against the q8 file, both lanes of the kq file through the rig). The server's /catalog document says which speech files read the front-end packs: its tts list carries the two packs the route loads (the browser's American-only twin rides no ladder) and every served model with needs_packs - the engine's family test on the file once it is here, true until then - so the control page wires a Pocket file on disk with no packs, and a phoneme family waits for its packs; before this the card demanded every companion pack, the American twin included, before it offered any model, which the fixtures captured before those rows landed never showed. The four catalog fixtures are re-captured over the fourteen-row served set, the page's speech ladder has a spec for both arms, and the catalog test reads the list back over a planted file. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
There are a few correctness/safety issues in the new tests and mint_models.py (notably unvalidated tree paths) that should be addressed before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates the dasLLAMA TTS stack and dasllama.io/browser examples to support Pocket TTS “small-form” artifacts (K-quant lane + smaller GGUFs), adjusts the server /catalog semantics so the UI can enable speech when packs aren’t needed, and adds a new in-browser voice-cloning example (“parrot”) alongside a miniaudio AudioWorklet capture-rate fix.
Changes:
- Add Pocket TTS K-quant (“kq”) lane support, including new validation tests and updated docs/model specs for the new
pocket-tts-*-kq.ggufforms. - Extend
/catalog(and control page ladder logic) withneeds_packsper TTS row and broaden fixtures/tests accordingly. - Add the
parrotbrowser example and update site/pages pipeline to build and publish it; fix AudioWorklet capture sample rate handling in the miniaudio patch.
File summaries
| File | Description |
|---|---|
| utils/dasllama-server/tests/tts.spec.js | Adds UI test coverage for needs_packs behavior in the TTS offer ladder. |
| utils/dasllama-server/tests/fixtures/catalog_idle.json | Updates captured /catalog idle fixture with needs_packs and new Pocket files. |
| utils/dasllama-server/tests/fixtures/catalog_empty.json | Updates captured /catalog empty fixture with needs_packs and new Pocket files. |
| utils/dasllama-server/tests/fixtures/catalog_downloading.json | Updates captured /catalog downloading fixture with needs_packs and new Pocket files. |
| utils/dasllama-server/tests/fixtures/catalog_done.json | Updates captured /catalog done fixture with needs_packs and new Pocket files. |
| utils/dasllama-server/test_model_catalog.das | Updates /catalog JSON generation tests and adds TTS list validation for needs_packs. |
| utils/dasllama-server/README.md | Documents updated /catalog TTS list semantics (needs_packs). |
| utils/dasllama-server/openai_server.das | Wires /catalog to pass tts_needs_packs into catalog JSON generation. |
| utils/dasllama-server/model_catalog.das | Adds TTS_FRONT_END_PACKS and extends catalog_state_json to emit needs_packs. |
| utils/dasllama-server/control.html | Updates the TTS download/enable ladder logic to respect needs_packs. |
| site-dasllama/test_metadata.py | Registers the new parrot example shell for metadata tests. |
| site-dasllama/REVIEW.md | Tightens copy-fix rule to include dated news/story entries. |
| site-dasllama/index.html | Updates homepage copy and adds a Parrot news entry block. |
| site-dasllama/feed.xml | Adds Parrot entry and refreshes Storywish TTS copy. |
| site-dasllama/examples.html | Adds Parrot card and updates Storywish description. |
| site-dasllama/_news/2026-09-10-storywish.md | Updates Storywish news copy to Pocket TTS small-form. |
| site-dasllama/_news/2026-09-10-parrot.md | Adds Parrot news post. |
| REVIEW_COMMON.md | Tweaks wording of rule-length constraint. |
| modules/dasLLAMA/tests/test_tts_pocket.das | Adds KQ-file tests and a “no encoder” Pocket file test; updates expectations. |
| modules/dasLLAMA/tests/test_tts_blocks.das | Adds K-quant lane numerical tests vs leaf implementation. |
| modules/dasLLAMA/tests/test_storywish.das | Updates Storywish smoke gating to Pocket one-voice file (no packs). |
| modules/dasLLAMA/tests/test_run_suites.das | Updates planned TTS file count expectations. |
| modules/dasLLAMA/tests/test_parrot.das | Adds Parrot model-free tests + smoke test rail. |
| modules/dasLLAMA/tests/run.das | Adds Parrot to suite roster and area plans. |
| modules/dasLLAMA/tests/REVIEW.md | Clarifies checklist language around loader families and fixtures. |
| modules/dasLLAMA/tests/CLAUDE.md | Updates census entry for Pocket small forms and new gates. |
| modules/dasLLAMA/tests/_example_rail.das | Notes Parrot smoke test as another consumer. |
| modules/dasLLAMA/REVIEW.md | Adjusts routed checklist references and wording around TTS + placement. |
| modules/dasLLAMA/REVIEW.das | Updates image layout stamp hash. |
| modules/dasLLAMA/REVIEW_UPSTREAM.md | Renames and refines “upstream/reference-build” naming checklist. |
| modules/dasLLAMA/REVIEW_TTS.md | Updates TTS checklist rules to cover K-quant lane and Pocket converter changes. |
| modules/dasLLAMA/REVIEW_PLACEMENT.md | Refines placement rules around duplication/guards/side-effect requires. |
| modules/dasLLAMA/performance/REVIEW.md | Tightens companion-artifact rules for model spec rows. |
| modules/dasLLAMA/performance/model_specs.das | Adds new Pocket small-form specs/recipes and updates HF pin. |
| modules/dasLLAMA/PERF_LEDGER.md | Records performance/quality comparison for small-form Pocket files. |
| modules/dasLLAMA/harness/tts_model_card.md | Documents new Pocket small-form files and licensing row consolidation. |
| modules/dasLLAMA/harness/test_convert_pocket.py | Adds pure-python unit tests for Pocket conversion predicates. |
| modules/dasLLAMA/harness/publish_tts_hf.py | Updates published file list to include new Pocket small forms. |
| modules/dasLLAMA/followup_general.md | Updates ladder/followup rows for Pocket small-form and next steps. |
| modules/dasLLAMA/ENVIRONMENT.md | Updates DISPLAY variable description to include Parrot smoke test. |
| modules/dasLLAMA/dasllama/dasllama_tts.das | Adjusts caps/voice resolution logic (Pocket cloning flag + voice lookup). |
| modules/dasLLAMA/dasllama/dasllama_tts_blocks.das | Implements K-quant lane support in TtsLinear + rows/decode paths. |
| modules/dasLLAMA/dasllama/dasllama_gguf.das | Tightens Q8_0 transcode slice alignment checks. |
| modules/dasLLAMA/dasllama/dasllama_env.das | Updates DISPLAY docstring to include Parrot. |
| modules/dasLLAMA/ARCHITECTURE_TTS.md | Notes K-quant planes lane alongside f32/q8 in TTS architecture. |
| modules/dasLLAMA/ARCHITECTURE_POCKET.md | Documents Pocket roster-as-latents and K-quant lane behavior. |
| modules/dasAudio/patches/miniaudio_memory64.cmake | Updates patch idempotency marker and fixes capture sample rate wiring. |
| modules/dasAudio/CMakeLists.txt | Forces reconfigure when the patch script changes (configure dependency). |
| examples/dasLLAMA/wasm/test_mint_models.py | Adds test coverage for new files and tree staging modes. |
| examples/dasLLAMA/wasm/mint_models.py | Adds files/tree staging and supports “no images” sets. |
| examples/dasLLAMA/storywish/web_shell.html | Updates Storywish shell to fetch Pocket file (no packs) and BFCache reload. |
| examples/dasLLAMA/storywish/models.json | Switches Storywish model set to include Pocket one-voice GGUF as a file. |
| examples/dasLLAMA/storywish/main.das | Updates Storywish CLI/docs/defaults for Pocket one-voice GGUF. |
| examples/dasLLAMA/storywish/.das_package | Updates package description to Pocket TTS. |
| examples/dasLLAMA/storyteller/web_shell.html | Adds BFCache reload handler. |
| examples/dasLLAMA/REVIEW.md | Adds BFCache reload rule and clarifies polling rules (mouse/buttons too). |
| examples/dasLLAMA/parrot/take.das | Adds pure helpers for Parrot take/VAD/text box/button hit testing. |
| examples/dasLLAMA/parrot/models.json | Adds Parrot model set (Pocket file + checked-in Silero VAD via tree). |
| examples/dasLLAMA/parrot/.das_package | Adds Parrot package/release manifest. |
| examples/dasLLAMA/ARCHITECTURE.md | Documents Parrot and expands browser-example mechanisms (BFCache, mouse). |
| doc/source/reference/tutorials/dasLLAMA_16_text_to_speech.rst | Updates tutorial text for Pocket no-encoder and K-quant lane. |
| .github/workflows/pages.yml | Builds/publishes Parrot wasm example alongside Storyteller/Storywish. |
| .github/workflows/extended_checks.yml | Adds unit test run for Pocket converter pure predicates. |
Review details
- Files reviewed: 67/68 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…: any absent row of the capture with needs_packs overridden, not a Pocket file - the page reads only the key, so the arm holds whatever the capture lists first Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
It spans core kernel/lane changes, server/UI behavior, deploy tooling, and multiple test/doc updates, so it warrants final human review despite only minor issues found.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
utils/dasllama-server/README.md:313
- This route description says the
/catalogttslist is ordered “the two front-end packs … then every served speech GGUF”, but the server emits models interleaved with their companion pack rows (e.g., kitten-nano before packs incatalog_idle.json). Consider rewording to avoid documenting a specific ordering unless the server enforces it.
utils/dasllama-server/model_catalog.das:353 - The docstring claims the
/catalogJSON’sttslist is ordered as “the two front-end packs, then every served speech model”, butserve_tts_set()emits each model followed by its companion packs (and thecatalog_idle.jsonfixture reflects that). Either reordertts_itemsto match the documented order, or adjust this comment to avoid promising an order that isn’t guaranteed.
- Files reviewed: 67/68 changed files
- Comments generated: 0 new
- Review effort level: Lite
…e way the news step installs markdown - the darwin runner's python carries neither Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
utils/dasllama-server/control.html currently treats a missing needs_packs field as “no packs needed”, which can incorrectly offer “enable speech” when packs are absent unless the field is explicitly present and correct.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
utils/dasllama-server/control.html:3261
- In
ttsOfferArm, the condition(!m.needs_packs || !missingPacks.length)treats a missing/undefinedneeds_packsfield as “does not need packs” (because!undefinedis true). If a catalog document ever omitsneeds_packsfor a phoneme-family model while packs are absent (e.g., older fixture/doc or a partial response), the UI would incorrectly offer “enable speech” even though packs are required. Defaulting to “needs packs unless explicitly false” keeps behavior conservative and matches the ladder description.
- Files reviewed: 67/68 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
It spans kernel/runtime execution lanes, model-format rules, server/catalog wiring, CI workflows, and user-facing site/example behavior, making it high-impact despite strong test additions.
Review details
- Files reviewed: 67/68 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🟡 Changes recommended
The /catalog + server pack-presence logic appears inconsistent with the engine’s documented “full pack vs US-only pack” fallback, which can incorrectly block enabling/serving phoneme-family TTS on some valid model directories.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 67/68 changed files
- Comments generated: 2
- Review effort level: Lite
| continue if (item.pack && !has_value(TTS_FRONT_END_PACKS, item.entry.name)) | ||
| let p = path_join(dir, item.entry.name) | ||
| let present = stat(p).is_valid | ||
| let packs = !item.pack && (!present || invoke(needs_packs, p)) | ||
| tts_items |> push(JV((file = item.entry.name, bytes = item.entry.bytes, pack = item.pack, |
| let TTS_LANE_DEFAULT = "q8" | ||
| var g_tts_lane = TTS_LANE_DEFAULT // the weight lane the worker is ASKED for; its ready event echoes the pin it took | ||
| var g_tts_voices_dir = "" // clips a cloning model adds to its voices at boot, each under its file's stem | ||
| let TTS_VOICE_CLIP_EXTENSIONS : array<string> <- [".wav", ".flac", ".mp3", ".ogg"] // what the decode rail reads | ||
| let TTS_FRONT_END_PACKS : array<string> <- ["tts_g2p.bin", "tts_postag.bin"] // load_tts_model reads both from the GGUF's directory | ||
| var g_shutdown_requested = false |
Behavior change: the storywish page on dasllama.io fetches a 65 MB speech file in place of the 152 MB one; the deploy mints it.
Why. The Pocket TTS English file was 152 MB at Q8_0: too large for a browser download and for a game embedding, and cloning needed the desktop. The rig showed the model holds its quality at 4 bits.
What changes.
convert_pocket.py --kqwrites the recipe as a real file, stores the voices as latent frames, and can leave the encoder out.parrotexample clones your voice from a take and reads the typed text in it./catalogsays which speech files need the front-end packs.Observable behavior.
Where to look.
dasllama_tts_blocks.das(linear_take_kq,linear_rows_kq),dasllama_pocket.das(read_linear),examples/dasLLAMA/parrot/take.das, the miniaudio patch.Validation, claims, ledger
Validation
The kq lane against the q8 lane on the real file: rms-relative 2.6e-2 on the teacher-forced frames (
test_pocket_kq_file), the exact lane speaks. A listen test of three sentences in two voices against the q8 file heard no difference.The rig (
harness/tts_rig.py, alba, 200 sentences, parakeet WER / UTMOS, M1 Max, JIT, the box's tune profile) on every lane the kernel change reaches, at the tip:--q8pin--f32pinThe q8 file's 200 WAVs at the tip are byte-identical to the run before this change's kernel edits, so the q8 lane's figures are the earlier run's; the f16 file's f32 row is the earlier run's too.
Process size and speed,
harness/tts_synth.dasover the first 60 sentences, resident set sampled every half second, two reps each: the q8 file holds 2.54 GB at RTF 0.051, the kq file 2.70 GB at RTF 0.0435 (PERF_LEDGER.md, the small-form section; the kq lane's 0.33 GB first-synthesis step is followup 129).The server's four
/catalogfixtures are re-captured over the served set (the American g2p pack and the six Pocket files of the last two PRs had not been captured either). The true document turned five speech-card specs red: the card demanded every pack row, so a box with the two packs was told to fetch the American twin and a Pocket file could not be wired - theneeds_packskey and the page's ladder rule above are that fix, with a spec for both arms;npx playwright testinutils/dasllama-server/testsat the tip.The browser pages were run locally from the wasm64 build: storywish reads at RTF 0.66 on the portable tier; parrot's capture, VAD, clone and say were driven end to end with a synthetic microphone (the JFK clip as a MediaStream, since the automation browser never gets the machine's microphone), then with a real microphone through an ssh port forward.
The miniaudio patch was applied to a pristine 0.11.25 header and to an already-patched one; the two results are byte-identical and a re-run is a no-op.
The full preflight ran once; its fast tier was red on two docs over the 300-line cap and on the image-layout stamp (the batch turned
linear_prepare'sdeleteintorelease_weight, the same teardown - no image byte moved, so the stamp is re-hashed andIMAGE_VERSIONstays 36). Both fixed, the two gates re-run green, then the six lanes the red tier skipped ran once each, and the module's model-free and stocked suites plus the image suite's kitten arm ran on this box. Model-free: 74 files, 2 red -test_vulkan_dec_tailandtest_vulkan_tieron "the class rails must engage on a live device", the M1 Max's MoltenVK arm that was red the same way before the last Pocket PR; this branch touches no Vulkan file.Claims - stated, not tested
parrot: cloned N s of speech.treestaging inmint_models.pyruns on the pages runner; a break stages nosilero_vad.binand the parrot page fails at its model load.Not done
followup_general.mdrow 124 records the ladder and what is open.load_audio_mono: row 128.