Repository navigation
Conversation
…torer (.work/restore.md, train/restore/) Prior art checked (FINALLY's recipe, Sidon MIT as the teacher, AnyEnhance, UniPASE, Vocos BWE, URGENT 2026), the open-license data that exists for English, Russian and singing and what does not, the design: one pass in the STFT, ConvNeXt over frames with self-attention for the room's tail, a change of log magnitude and a phase, trained as a regressor then adversarially with WavLM's features in the loss; every loss through a band limit at the target's own bandwidth, so 24 and 16 kHz corpora train beside full-band ones in one run. train/restore: the on-the-fly degradation chain (rooms measured or synthetic, noise, microphone, compressor, clipping, band limit, codecs through ffmpeg, word length), manifests with each file's bandwidth, the generator and discriminators, losses, the run with resume and validation, ONNX export in neural-separate's contract, chunked inference, scoring on bench/speech.py's measures plus LSD, and a CPU smoke test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUVeWzmg33J7gc9R3MWMYh
…ath), pesq's build on 3.13 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUVeWzmg33J7gc9R3MWMYh
…pen data, scan, train, export); measured here: no GPU, no route to the corpora, the full model past 15 GB on CPU Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUVeWzmg33J7gc9R3MWMYh
…ore and restore() (data by license, the teacher pass and its gate, stage A go/no-go, the run, trouble, evaluation sets and baselines, export and hosting, the plugin's contract, the op); data.py pairs writes examples to listen to Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUVeWzmg33J7gc9R3MWMYh
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The groundwork for
restore(), an offline alternative to Adobe Podcast Enhance Speech: a voice recorded in a room, in noise, on a phone or through a codec, speech or singing, English or Russian, back as a studio take at 48 kHz. No weights yet: this is the plan, the training machinery (verified end to end on CPU) and the runbook to train and ship it as@audio/neural-restorein the audiojs ecosystem..work/restore.md: prior art checked (FINALLY's recipe, Sidon as the teacher, AnyEnhance, UniPASE, Vocos BWE, URGENT 2026), the open-license data that exists for English, Russian and singing and what does not, the design and its reasons, compute, evaluation, shipping, risks.train/restore/: the on-the-fly degradation chain (rooms measured or synthetic, noise, microphone, compressor, clipping, band limit, MP3/AAC/Opus/GSM through ffmpeg, word length); manifests with each file's measured bandwidth; a one-pass generator in the STFT (ConvNeXt over frames with self-attention, a change of log magnitude plus a phase, ~18M) and HiFi-GAN / UnivNet discriminators; multi-resolution STFT, log-mel, WavLM-feature and least-squares GAN losses, every one through a band limit at the target's own bandwidth so 16 and 24 kHz corpora train beside full-band ones in one run; training with resume and validation; ONNX export inneural-separate's[1, 2, 1025, T]contract; chunked inference; scoring onbench/speech.py's measures plus LSD;run.shas one command on a GPU machine;test.pysmoke test.train/restore/HANDOUT.md: the step-by-step runbook from an empty GPU box to the published plugin and the op, with go / no-go numbers..work/todo.md: sections 3 and 4 of the enhance plan updated to point here.Verified
python train/restore/test.pyon CPU (torch 2.14): bandwidth detection on 24 kHz, 48 kHz and low-passed files; the chain with all four codecs; a training run with resume; ONNX export within 123 dB of torch at two lengths; chunked inference on a 25 s file matching the input's length; the WavLM loss path by shape on an untrained model of the same class.Not done here, and why: no GPU, no network route to any corpus, and 15 GB of RAM in the session. Measured: the full model is killed at batch 4 on CPU; a toy adversarial step takes 30 s. The run belongs on a rented 24 GB card (handout, section 1).
Not in this PR
The op itself (
fn/restore.js), the plugin, and weights: they follow the first trained model, per the handout's sections 5–7.🤖 Generated with Claude Code
https://claude.ai/code/session_01MUVeWzmg33J7gc9R3MWMYh
Generated by Claude Code