A multilingual greeting wake-word spotter built on the O-Series liquid core.
Always-on keyword spotting on a battery: the model listens continuously, holds a fixed-size state no matter how long the microphone has been open, and fires when it hears a greeting — "hello", "hola", "bonjour", "你好" — in any of the languages it was trained on.
mic → log-mel (fixed, no weights) → liquid core (multi-timescale τ) → wake / not-wake
↑ carried state: 5,120 bytes, constant
This section comes first on purpose.
Validated (runs today, in CI)
- 32 tests pass: streaming
step()is numerically identical to the batchedforward(), carried state stays constant over 500 frames, τ spans the configured range, the export CLI fits its budget, and the budget gate really fails the build when it cannot be met. - ONNX export: 5.03 MB fp32, 1.27 MB int8, max |onnx − torch| = 3.7e-09.
Measured, reproducible with
python export_onnx.py --int8. - Carried state: 5,120 bytes per stream, independent of stream length — this is a property of the recurrence, verified by test, not a measurement that could drift.
Measured — the architecture works, the multilingual framing does not (yet)
- Run 2 (2026-08-01, English only, 301 wake clips): FRR 0.146 at FAR 0.046 on held-out test. Best dev accuracy 0.919 against a 0.780 never-fire baseline — the first configuration here that beats "always say no".
- Run 1 (2026-07-31, 9 languages, 91 wake clips): FRR 0.909 at FAR 0.049, never firing at all for six of the nine. Dev 0.846 against a 0.984 never-fire baseline — worse than saying no unconditionally.
- Run 7 (2026-08-18, 10 languages, full MSWC): dev acc 0.885
(balanced 0.849), but per-language FRR is bounded by the worst language
(1.000, Italian, 2 positive clips) — most languages have 2–13 positive
clips against 400 negatives, so per-language numbers are noise there.
English remains the only language with meaningful data (FRR 0.098 at FAR
0.133, 41 clips). Checkpoint + ONNX published as a research artifact:
release
research-2026-08-18. - Same code, same config, same hardware; only the label set changed. Miss rate fell to roughly one third, which settles what Run 2 was built to settle: the architecture is not the blocker. Two variables moved (3.3× the positives, one acoustic target) and this pair of runs does not separate them.
- Run 8 (v2, 2026-09-29, 10 languages, GPU A100) — improved recipe
(hidden 896, 40 epochs, 400 negatives, multiclass OR inference): dev acc
0.838, dev balanced 0.8815 (Run 7: 0.849), worst-language dev FRR
0.600 (sv-SE; Run 7: 1.000). On the held-out test split no operating
point reaches FAR ≤ 1% — the production bar is still unmet. Same honest
verdict as Run 7: per-language numbers at 2–13 positive clips are noise;
the multilingual claim remains unsupported at this data scale. The gap is
negative-sample diversity, not the model. Checkpoint:
o1sound_multiclass_v2.pt(9.8 MB, on the training server; HF export pending the FAR fix). - Run 9 (v3, 2026-10-04, Speech Commands 负样本扩充) — 真实非唤醒音频
集扩充（9706 条 SC 提取、训练用 3000 条 + MSWC 全语言负池）：dev acc
0.8754、dev balanced 0.8913（Run 8: 0.8815）、最差语言 FRR 0.500
（Run 8: 0.600）。测试集（MSWC + SC 留出负样本，167 正/1859 负）：
FAR ≤ 1% 门槛已跨过（2026-10-05）——阈值扫描从 99 点网格
（
linspace(0.01, 0.99, 99)）改为精确扫描（以全部唯一分数为候选阈值） 后：thr 0.9906 → FAR 0.97%（18/1859）≤ 1% ✓。此前网格报告"最小可达 1.02%"是网格量化卡住了 0.99 与真实边界之间的最后一条假接受，不是模型 瓶颈。完整取舍曲线亦测出：thr 0.9994 时 FAR 可至 0（0/1859，FRR 0.81）。 ⚠ 诚实记录：全量 9706 条负样本重训（60 epochs）是负结果——dev balanced 崩到 0.7319（类别失衡 ~1:10 压垮召回）。最佳配置是 v3（3000 条平衡采样）。 剩余边界：最差语言 FRR 仍受 2–3 条测试样本的语言约束（pt/ru = 1.000）， 多语言 claim 的数据量问题独立于本次 FAR 修复。 代码：--extra-negatives-dir+--extra-negatives（train/eval 两侧）+eval.py精确扫描。权重：o1sound_neg_v1.pt（最佳）在服务器； 评估产物results/test_neg_v1_exact.json。 - Not production quality. Deployed wake words run single-digit FRR at a false-accept rate quoted per hour, not per clip. And the multilingual claim is still unsupported — Run 1 is evidence against it at this data scale. The planned fix is a multi-class head OR'd at inference, not more languages poured into one binary class. Reasoning in RESULTS.md.
Research weights — published, not production-grade
- A trained checkpoint ships as a GitHub Release
(
research-2026-08-18):o1sound.pt(4.8 MB) + exportedo1sound.onnx(fp32 5.04 MB) + per-language metrics. It is a research artifact: English shows a usable signal, multilingual is open (see Run 7 above). Not a production wake-word model.export_onnx.pywithout--ckptexports an untrained graph: the size is real, the behaviour is noise. It says so when you run it. - The greeting list in
o1sound/keywords.pycovers 20 languages, but a language only counts once its MSWC directory has actually been downloaded and trained on.train.pyprints a warning naming any language in the spec that is missing from disk, because "20 languages" and "20 languages we trained on" are different claims. - Latency is unmeasured on real hardware. The Python
step()loop runs at ~0.5 ms/frame on a desktop CPU, which says nothing about an MCU or a DSP. No power figure exists either. - The 7 MB budget was a target given up front, not a constraint the architecture strained against — at hidden=640 it lands at 5.03 MB with room to spare.
What would make the claims real
python scripts/fetch_mswc.py --languages en,de,fr,es,it,pt,pl,ru,tr,id --out data/mswc
python train.py --root data/mswc --epochs 20 --out checkpoints/o1sound.pt
python eval.py --ckpt checkpoints/o1sound.pt --root data/mswc --split test --out results/test.json
python export_onnx.py --ckpt checkpoints/o1sound.pt --out dist/o1sound.onnx --int8Real-time wake detection from a microphone (or a WAV file) through the same
streaming step() path the ONNX graph uses:
pip install sounddevice
python scripts/demo_stream.py # microphone, threshold 0.9
python scripts/demo_stream.py --wav x.wav # replay a 16kHz wav instead
python scripts/demo_stream.py --threshold 0.95
python scripts/demo_stream.py --list-devicesCarried state is 5,120 bytes per stream and constant — watch it fire on "hello / hallo / hola". Honest boundary: this is the research checkpoint (multilingual Run 7); on out-of-domain audio (e.g. pure tones) it can false-trigger — it is not a production wake-word engine.
eval.py reports false-reject rate at a fixed false-accept budget, per
language, and prints the worst language separately. That worst number — not
the mean, not accuracy — is what bounds any multilingual claim made about this
model.
The streaming model is plain C in freertos/ — log-mel + liquid
core + head, constant 5,120-byte state, one frame at a time. Host-side parity
gate: the C logits match PyTorch frame-by-frame within 1e-3 of the logit span
(python tests/test_c_parity.py). A FreeRTOS task wiring (I2S ring buffer →
step → debounced wake) ships as freertos/freertos_demo.c. The 480-point DFT
in the reference kernel is O(N²); the production MCU path is a mixed-radix FFT
(see freertos/README.md). Nothing here is measured on real hardware yet.
The OR-head fix planned in RESULTS.md ran end-to-end on 10 languages
(en/de/fr/es/pt/pl/ru/cs/fa/sv-SE): one class per greeting + "other", wake
decision = OR over greeting classes. Dev balanced accuracy 0.84 (vs 0.50 =
chance); test-set FRR 0.35 @ FAR 0.046 overall — the multilingual claim
remains unsupported: worst-language FRR is 1.000 (ru, 2 test clips), and the
tail languages have 2–15 clips. Same conclusion as Runs 1/7: the architecture
is not the blocker, per-language data is. Checkpoint + per-language metrics in
the research release notes.
Each channel carries its own learnable time constant τ, parameterised as
softplus(log_tau) + tau_min and initialised geometrically across
10–240 ms. Short-τ channels track the current phoneme; long-τ channels hold the
envelope of the whole word. That spread is what separates a wake phrase from a
phonetically close neighbour without stacking depth — which matters when the
budget is a few megabytes and the model never sleeps.
Two consequences for the target device:
- Streaming is O(1). The state is a fixed
(batch, hidden)tensor per layer. It does not grow with how long the microphone has been open, so there is no drift in memory or latency across a long session. step()is what ships. The exported ONNX graph is the single-frame step — one frame in, logits and the next state out — not a fixed-length window.
pip install -e .
# .opus clips additionally need: pip install soundfileMeasured, n_classes=2:
| hidden | layers | params | fp32 | int8 | carried state |
|---|---|---|---|---|---|
| 384 | 2 | 483,984 | 1.94 MB | 0.48 MB | 3,072 B |
| 512 | 2 | 841,872 | 3.37 MB | 0.84 MB | 4,096 B |
| 640 | 2 | 1,298,064 | 5.19 MB | 1.30 MB | 5,120 B |
| 512 | 3 | 1,367,184 | 5.47 MB | 1.37 MB | 6,144 B |
| 768 | 2 | 1,852,560 | 7.41 MB | 1.85 MB | 6,144 B |
hidden=640, layers=2 is the default: the largest 2-layer configuration that
still clears 7 MB in fp32 once ONNX graph overhead is counted (5.03 MB on disk).
o1sound/features.py log-mel frontend (torch.stft, no torchaudio, ONNX-clean)
o1sound/model.py liquid core + streaming step()
o1sound/data/mswc.py MSWC loader, per-language splits, binary wake/not-wake
o1sound/keywords.py greeting word per language
train.py training loop, per-language FRR each epoch
eval.py FRR at a fixed FAR budget, per language
export_onnx.py streaming ONNX + int8 + hard size gate
- AwareLiquid/M1 — the MT-LNN / O-Series research line this core comes from.
- awareliquid.ai — benchmarks and retractions.
MIT.