From 6a65ec5f90815aa227f4118c0da67d3dc05ab097 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 11:32:15 +0000 Subject: [PATCH 1/2] M3: nn_suppressor on tap::dsp::nn; pitch search accumulates in Sample MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The wake-word plan's M3: the dense and GRU arithmetic now lives in DspTap (tap::dsp::nn, tap/DspTap#15) and nn_suppressor consumes basic_dense and basic_gru. The weight arrays are moved into the kernels at construction (one copy in memory). The kernels' loop order is exactly the one this header carried, so the promotion is bit-identical in both profiles on the shipping v2 model against the pre-refactor binary; the six parity cases pass at the M2 depths, the M2 battery passes unchanged, and the nn_suppressor ratchet moved -0.12 % / -0.10 % on m55 and -0.19 % / -0.11 % on m33 (16 k / 48 k) with every other scenario unchanged to the instruction. submodules/dsptap points at the M3 tree; repoint at the identical tree on main once the DspTap PR merges. Follow-up to the M2 finding on the Cortex-M33 leg: the speech predictor's pitch search (lpc.h) accumulated its normalized correlation in double for every lag from 32 to 400 over a 1024-sample window — about a million software double operations per 64-sample block on a core without FP64, and two thirds of the tonal PEM headline's 1085 s under qemu mps2-an505 (the test harness's deliberate double room simulation was the other third: 772 s with it switched to float). The search now accumulates in Sample. The double profile is bit-for-bit unchanged; the float rows' measured numbers are unchanged to the MSG bisection's 0.5 dB quantum (kalman-loop tonal ASG +7.81 dB before and after, PEM tonal +10.84 -> +11.13, ERLE and misalignment identical). The tonal headline now takes 31 s on the M33 emulator and the full float selection 174 s, so the MUTAP_ON_TARGET_SOFT_FP64 exclusion is removed: the M33 leg runs exactly the M55's selection. docs/wake-word-plan.md carries the M3 record and the corrected M2 finding; HANDOFF.md points at M4. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01JuUg1ZBxm3fyBqWgQv6H1G --- .github/workflows/ci.yml | 8 +- HANDOFF.md | 2 +- cmake/arm-cortex-m33-mps2.cmake | 4 - docs/wake-word-plan.md | 56 ++++++++++---- include/mutap/lpc.h | 36 ++++++--- include/mutap/nn_suppressor.h | 130 +++++++++++++------------------- submodules/dsptap | 2 +- tests/CMakeLists.txt | 5 -- tests/bare_metal_main.cpp | 20 ++--- 9 files changed, 131 insertions(+), 132 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 35ceeb0..760b1a7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -184,10 +184,10 @@ jobs: # Cortex-M33 (Raspberry Pi Pico 2 W / RP2350 class: single-precision FPU, # no FP64, no MVE) on QEMU's MPS2+ AN505 model — the wake-word plan's named # embedded target. Same Armv8-M startup as the M55 leg, Ooura float32 FFT - # (no Helium here), the M55's float-profile selection minus the long float - # PEM scenarios (their test harness simulates the room in double: soft-float - # here, ~17 min for one of them — see tests/bare_metal_main.cpp). Ported - # from RatioTap's cortex-m33-qemu job. + # (no Helium here), the same float-profile selection as the M55 leg (about + # 3 minutes of emulation once the speech predictor's pitch search stopped + # accumulating in double — see tests/bare_metal_main.cpp). Ported from + # RatioTap's cortex-m33-qemu job. cortex-m33-qemu: name: Cortex-M33 cross (QEMU) runs-on: ubuntu-latest diff --git a/HANDOFF.md b/HANDOFF.md index 0f2420c..26d0ec2 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -756,6 +756,6 @@ Resolved since rev 1: ~~license~~ (MIT), ~~core language~~ (header-only C++20), Resolved since rev 3: ~~Max external naming~~ — settled (Rev 4): **`mutap.afc~`** (rename from the `mutap.defeed~` placeholder) and **`mutap.aec~`** for the new echo canceller, an acronym pair matching the literature. The rename executes in Stage 2 of "The next effort" above. Still open: -- **Wake-word detection — in progress (M0–M2 done).** A background briefing, a staged implementation proposal (rev 2) and the adversarial audit that produced it landed in [`docs/wake-word-briefing.md`](docs/wake-word-briefing.md), [`docs/wake-word-plan.md`](docs/wake-word-plan.md) and [`docs/wake-word-audit.md`](docs/wake-word-audit.md). The proposal reuses the `nn_suppressor` / `tools/ml` patterns rather than importing a runtime; the audit found those patterns sound but the learned path's *oracles* missing (the suppressor is double-only everywhere, never on-target, never instruction-counted, parity not in CI), so rev 2 adds an M2 that builds them before the kernels are promoted — a milestone worth doing whether or not the spotter ships. The named M33 target is the Raspberry Pi Pico 2 W (RP2350: single-precision FPU, so float32 is its profile; the M33 QEMU rig is ported from RatioTap in M2, and M7 adds a `pico2w` CI job that builds the board example against a pinned Pico SDK, uploads the UF2 and asserts its footprint — detection itself is a bench step with a loudspeaker-playback protocol and committed numbers, since QEMU has no RP2350 model), and the docs plan includes a user guide to training a phrase whose commands CI runs on a toy corpus. **M0 is decided (4 September 2026):** repository MuTap, charter widened; host rate fixed at 16 kHz in `kws.h` with conversion as an `@resample` option on the Max external, backed by a new DspTap `decimate.h` (2/3/6) and composed with RatioTap for 44.1 kHz, since `poly~` is powers-of-two only and neither RatioTap nor SampleRateTap covers 48 → 16; release shape runtime-first, no bundled phrase, the training guide as the primary document; development phrase `marvin` from Speech Commands for M5 bring-up, a synthesized four-syllable phrase from M4, never shipped; TTS voices lineage-verified from the Piper model cards — `en_US-libritts-high` (from scratch, CC BY 4.0, 904 speakers), `en_US-kristin-medium`, `en_GB-cori-high` (from scratch, public domain), `en_US-john-medium` (from Kristin), with every Lessac-derived voice (most of the English set, `libritts_r` and `vctk` included) and the sample generator's bundled `.pt` generator excluded; training on an Apple Silicon Mac via `--device mps`, under an hour per run on the development set. **M1 is done** on DspTap's `claude/mutap-wake-word-plan-2i63pe`: `log_mel.h` and `decimate.h` with their typed batteries, the numpy reference generator (`tools/reference/make_frontend_reference.py` — the family's single numpy copy of the formulas), C ABI and bridge; every tolerance in the plan's M1 record is a measured number. **M2 is done** on MuTap's `claude/mutap-wake-word-plan-2i63pe`: the learned suppressor has its oracles — typed float/double tests with a −120 dB cross-precision pin, a float32 chain gate, a Python↔C++ parity CI job in both profiles on random and exported weights, a Cortex-M33 QEMU leg (mps2-an505, Ooura float32 FFT) with the float suppressor suite on-target on every leg, and `nn_suppressor` icount scenarios with baselines on m55, m33 and hexagon. The oracles found and fixed a Nyquist-bin contract defect at 48 kHz (C++ notched bin N/2, numpy did not; fixed on both sides). Next: M3, promoting the dense/GRU kernels into DspTap's `tap::dsp::nn`. +- **Wake-word detection — in progress (M0–M3 done).** A background briefing, a staged implementation proposal (rev 2) and the adversarial audit that produced it landed in [`docs/wake-word-briefing.md`](docs/wake-word-briefing.md), [`docs/wake-word-plan.md`](docs/wake-word-plan.md) and [`docs/wake-word-audit.md`](docs/wake-word-audit.md). The proposal reuses the `nn_suppressor` / `tools/ml` patterns rather than importing a runtime; the audit found those patterns sound but the learned path's *oracles* missing (the suppressor is double-only everywhere, never on-target, never instruction-counted, parity not in CI), so rev 2 adds an M2 that builds them before the kernels are promoted — a milestone worth doing whether or not the spotter ships. The named M33 target is the Raspberry Pi Pico 2 W (RP2350: single-precision FPU, so float32 is its profile; the M33 QEMU rig is ported from RatioTap in M2, and M7 adds a `pico2w` CI job that builds the board example against a pinned Pico SDK, uploads the UF2 and asserts its footprint — detection itself is a bench step with a loudspeaker-playback protocol and committed numbers, since QEMU has no RP2350 model), and the docs plan includes a user guide to training a phrase whose commands CI runs on a toy corpus. **M0 is decided (4 September 2026):** repository MuTap, charter widened; host rate fixed at 16 kHz in `kws.h` with conversion as an `@resample` option on the Max external, backed by a new DspTap `decimate.h` (2/3/6) and composed with RatioTap for 44.1 kHz, since `poly~` is powers-of-two only and neither RatioTap nor SampleRateTap covers 48 → 16; release shape runtime-first, no bundled phrase, the training guide as the primary document; development phrase `marvin` from Speech Commands for M5 bring-up, a synthesized four-syllable phrase from M4, never shipped; TTS voices lineage-verified from the Piper model cards — `en_US-libritts-high` (from scratch, CC BY 4.0, 904 speakers), `en_US-kristin-medium`, `en_GB-cori-high` (from scratch, public domain), `en_US-john-medium` (from Kristin), with every Lessac-derived voice (most of the English set, `libritts_r` and `vctk` included) and the sample generator's bundled `.pt` generator excluded; training on an Apple Silicon Mac via `--device mps`, under an hour per run on the development set. **M1 is done** on DspTap's `claude/mutap-wake-word-plan-2i63pe`: `log_mel.h` and `decimate.h` with their typed batteries, the numpy reference generator (`tools/reference/make_frontend_reference.py` — the family's single numpy copy of the formulas), C ABI and bridge; every tolerance in the plan's M1 record is a measured number. **M2 is done** on MuTap's `claude/mutap-wake-word-plan-2i63pe`: the learned suppressor has its oracles — typed float/double tests with a −120 dB cross-precision pin, a float32 chain gate, a Python↔C++ parity CI job in both profiles on random and exported weights, a Cortex-M33 QEMU leg (mps2-an505, Ooura float32 FFT) with the float suppressor suite on-target on every leg, and `nn_suppressor` icount scenarios with baselines on m55, m33 and hexagon. The oracles found and fixed a Nyquist-bin contract defect at 48 kHz (C++ notched bin N/2, numpy did not; fixed on both sides). **M3 is done**: the dense/GRU kernels are DspTap's `tap::dsp::nn` (tap/DspTap#15) and `nn_suppressor` consumes them, bit-identically in both profiles on the shipping v2 model, with the M2 battery, the six parity cases and the ratchet unchanged; MuTap's DspTap pin points at the M3 tree (repoint at `main` after the DspTap PR merges) and MuTap-Max's pin follows once this merges. Alongside it, the M33 finding's root cause was fixed in the library: the speech predictor's pitch search now accumulates in `Sample` (the double profile is bit-for-bit unchanged, the float rows unchanged to the MSG bisection's quantum), the tonal PEM headline dropped from 1085 s to 31 s on the M33 emulator, and the M33 leg runs the full float selection (174 s) — the M2 exclusion is gone. Next: M4, the corpus and dataset builder, on the Apple Silicon Mac. - **Default engine in the external** — `@kalman` off (classic NLMS) is the shipping default purely on seniority; the measured case for flipping it is in `tests/test_fd_kalman.cpp` and book chapter 1. Decide after real-room listening. - **RIR fixtures, the measured half** — the fixture pipeline is built and three physically-modeled rooms (image-source, documented geometry) are committed baselines with regression tests. What remains yours: which MEASURED rooms join them — an academic dataset room (MYRiAD is the PEM-AFROW group's own database; openAIR is the other usual source; check each room's license allows redistribution in an MIT repo) and/or your own swept-sine measurements. Either way it is one command per room: `python3 tools/fixtures/make_rir_fixtures.py --from-wav room.wav myroom --source ""`, then a test with a freshly measured threshold. (The dataset hosts are unreachable from the remote dev container's network policy, so the WAVs have to enter via a commit.) diff --git a/cmake/arm-cortex-m33-mps2.cmake b/cmake/arm-cortex-m33-mps2.cmake index eac3120..bd806f8 100644 --- a/cmake/arm-cortex-m33-mps2.cmake +++ b/cmake/arm-cortex-m33-mps2.cmake @@ -41,7 +41,3 @@ set(TAP_DSP_FFT_CMSIS OFF CACHE BOOL "No MVE on the Cortex-M33: Ooura float32 FF # One-shot CTest mode (no argv on bare metal; see tests/CMakeLists.txt). set(MUTAP_BARE_METAL ON) -# Single-precision FPU only: the on-target selection drops the long float -# PEM scenarios, whose test harness simulates the room in double (soft-float -# here; ~17 min for one of them under QEMU). See tests/bare_metal_main.cpp. -set(MUTAP_ON_TARGET_SOFT_FP64 ON) diff --git a/docs/wake-word-plan.md b/docs/wake-word-plan.md index 0f785f1..a1b894d 100644 --- a/docs/wake-word-plan.md +++ b/docs/wake-word-plan.md @@ -1,8 +1,8 @@ # Building `mutap.wake~` — implementation proposal *Proposal, rev 2 — 4 September 2026; **in progress**: M0 decided, M1 done -(DspTap) and M2 done (MuTap), each with a dated record of measured numbers -under its milestone in §6. Background and +(DspTap), M2 done (MuTap) and M3 done (DspTap · MuTap), each with a dated +record of measured numbers under its milestone in §6. Background and corpus survey in [`wake-word-briefing.md`](wake-word-briefing.md). Rev 2 carries every amendment from the [adversarial audit](wake-word-audit.md) of rev 1; the audit refers to rev 1's milestone numbers, and the mapping is given @@ -389,18 +389,25 @@ nothing in the float profile touches double. The M33 leg is ported from RatioTap (`cmake/arm-cortex-m33-mps2.cmake`, `platform/mps2_an505.ld`, the shared `armv8m_startup.c`, a `cortex-m33-qemu` CI job with the Ooura float32 FFT pinned since there is no MVE) and the on-target filter on every leg carries the -float `nn_suppressor` suite, the cross-precision pin and the chain test. One -honest limit of the M33 leg, found by running it: the long float PEM -scenarios are driven by a test harness that simulates the room in double on -purpose (the closed-loop convolution and the MSG bisection), and the speech -predictor's pitch search accumulates in double — hardware on the M55, -software on the M33 — so the tonal PEM headline alone took 1030 s under qemu -mps2-an505 against 84 s on mps3-an547. The M33 selection -(`MUTAP_ON_TARGET_SOFT_FP64`) drops those four scenarios, which the M55 and -Hexagon legs and every host still run. That is also the first concrete cost -figure for "double on the RP2350" in this plan, and the reason §5's rule that -nothing on the wake-word path touches double is a budget rule, not a style -rule. Layer 4 +float `nn_suppressor` suite, the cross-precision pin and the chain test. The +M33 leg also produced the first concrete cost figure for "double on the +RP2350" in this plan: the tonal PEM headline took 1085 s under qemu +mps2-an505 against 84 s on mps3-an547 and the selection timed out, so M2 +first shipped with the four long float PEM scenarios excluded there. An +experiment then separated the two suspects — with the test harness's +deliberate double room simulation switched to float the scenario still took +772 s, so the harness was a third of the cost and the library two thirds: the +speech predictor's pitch search accumulated its normalized correlation in +double for every lag from 32 to 400 over a 1024-sample window, about a +million software double operations per 64-sample block on that core. The +follow-up landed with M3: the search accumulates in `Sample` (`lpc.h`), the +double profile is bit-for-bit unchanged, the float rows' measured numbers +are unchanged to the bisection's 0.5 dB quantum (kalman-loop tonal ASG ++7.81 dB before and after; PEM tonal +10.84 → +11.13; ERLE and misalignment +identical), and the full float selection now runs on the M33 in 174 s (the +tonal headline 31 s), so the exclusion is gone and the M33 leg runs exactly +the M55's selection. That is the reason §5's rule that nothing on the +wake-word path touches double is a budget rule, not a style rule. Layer 4 of `bench/icount` is the suppressor at both trained geometries with xorshift weights; baselines seeded locally on m55 and m33 (the local ratchet reproduces every committed m55 baseline to 0.00 %, so local seeding is trustworthy; CI @@ -440,6 +447,27 @@ scenario is within the drift gate on both targets; DspTap's typed battery for `tap::dsp::nn` pins layout, gate order and accumulator precision; MuTap and MuTap-Max pins bumped. +**Done, 5 September 2026** (DspTap and MuTap branches +`claude/mutap-wake-word-plan-2i63pe`; tap/DspTap#15). DspTap's seventh +primitive, `include/tap/dsp/nn.h`: `basic_dense` (row-major +`[out x in]`, linear / tanh / sigmoid) and `basic_gru` (PyTorch +`nn.GRU` convention, gates r, z, n), contract version 1, weights stored as +float32 and converted to `Sample` at the point of use, bias-first ascending +accumulation in `Sample`, moved-in owned weights, noexcept processing. +Measured: the GRU against an independent long-double restatement summing in +the opposite order 2.2e-16 double / 1.0e-7 float (pinned 1e-14 / 1e-6); +float vs double at the suppressor geometry 6.3e-7 state / 2.5e-7 gains +(pinned 3e-6 / 1e-6); 156 DspTap tests, warnings as errors, clang-tidy +clean. MuTap's `nn_suppressor` consumes the kernels (its weight arrays are +moved into them at construction, one copy in memory) and the promotion is +**bit-identical** in both profiles on the shipping v2 model against the +pre-refactor binary; the six parity cases pass at the M2 depths; the M2 +battery passes unchanged; the `nn_suppressor` ratchet moved −0.12 % / −0.10 % +on m55 and −0.19 % / −0.11 % on m33 (16 k / 48 k), inside the gate, with +every other scenario unchanged to the instruction. Pins: MuTap's DspTap pin +points at the M3 tree (repointed at the identical tree on `main` once +tap/DspTap#15 merges); MuTap-Max's MuTap pin follows once MuTap's PR merges. + ### M4 — Corpus, splits and dataset builder *(tools/ml/kws)* `kws_features.py` as source of truth for the feature *values* (the M1 numpy diff --git a/include/mutap/lpc.h b/include/mutap/lpc.h index b46db9c..78aeeaf 100644 --- a/include/mutap/lpc.h +++ b/include/mutap/lpc.h @@ -400,30 +400,42 @@ namespace tap::mu { if (n < 2 * m_cfg.min_lag) { return; } + // Numeric contract: the correlation sums accumulate in Sample. + // The float profile therefore contains no double arithmetic + // here (on a single-precision core such as the Cortex-M33 this + // search was the canceller's dominant soft-float cost: ~1 M + // double operations per 64-sample block at the default lag + // range); a normalized correlation over at most + // analysis_capacity samples loses ~1e-6 relative in float, + // which is far below the voicing decision's resolution. The + // double profile is unchanged. const size_t max_lag = (m_cfg.max_lag < n / 2) ? m_cfg.max_lag : n / 2; - double best = 0.0; + Sample best = Sample(0); for (size_t lag = m_cfg.min_lag; lag <= max_lag; ++lag) { - double cross = 0.0; - double e_now = 0.0; - double e_lag = 0.0; + Sample cross = Sample(0); + Sample e_now = Sample(0); + Sample e_lag = Sample(0); for (size_t i = lag; i < n; ++i) { - const double x0 = static_cast(m_residual[i]); - const double x1 = static_cast(m_residual[i - lag]); + const Sample x0 = m_residual[i]; + const Sample x1 = m_residual[i - lag]; cross += x0 * x1; e_now += x0 * x0; e_lag += x1 * x1; } - if (e_now <= 0.0 || e_lag <= 0.0) { + if (!(e_now > Sample(0)) || !(e_lag > Sample(0))) { continue; } - const double rho = cross / std::sqrt(e_now * e_lag); + const Sample denom = std::sqrt(e_now * e_lag); + if (!(denom > Sample(0))) { // product underflow on a near-silent residual + continue; + } + const Sample rho = cross / denom; if (rho > best) { best = rho; - if (rho > static_cast(m_cfg.voicing_threshold)) { - const double beta = cross / e_lag; + if (rho > m_cfg.voicing_threshold) { + const Sample beta = cross / e_lag; m_lag = lag; - m_beta = static_cast(std::clamp(beta, -static_cast(m_cfg.max_gain), - static_cast(m_cfg.max_gain))); + m_beta = std::clamp(beta, -m_cfg.max_gain, m_cfg.max_gain); } } } diff --git a/include/mutap/nn_suppressor.h b/include/mutap/nn_suppressor.h index 566008a..65a036b 100644 --- a/include/mutap/nn_suppressor.h +++ b/include/mutap/nn_suppressor.h @@ -20,6 +20,7 @@ #include #include "mutap/fft.h" +#include "tap/dsp/nn.h" namespace tap::mu { @@ -88,10 +89,13 @@ namespace tap::mu { /// dense_out) and every band energy accumulates in Sample. Double is /// the golden model; the float profile contains NO double arithmetic, /// so it runs natively on parts without FP64 (the Cortex-M33 of the - /// RP2350 included) and never falls into soft-float. The float-tracks- - /// double depth is pinned by tests/test_nn_suppressor.cpp - /// (NnSuppressorCrossPrecision) and the promotion of these kernels into - /// DspTap must keep it unchanged. The gain path is E-only: the + /// RP2350 included) and never falls into soft-float. Since the + /// wake-word plan's M3 the network layers are DspTap's tap::dsp::nn + /// (basic_dense / basic_gru, contract version 1, whose loop order is + /// exactly the one this header carried before, so the promotion was + /// bit-identical); the float-tracks-double depth is pinned by + /// tests/test_nn_suppressor.cpp (NnSuppressorCrossPrecision) and the + /// Python parity job. The gain path is E-only: the /// echo estimate Yhat feeds the FEATURES, never the signal path, so /// the worst a bad prediction can do is mis-gain a band — the /// structural safety that motivates learning gains instead of a @@ -118,14 +122,21 @@ namespace tap::mu { Sample floor_bias = Sample(4); /// One-pole smoothing of the echo_explained() accumulators. Sample explained_smoothing = Sample(0.95); - /// The trained model. + /// The trained model. Its weight arrays are moved into the + /// inference kernels at construction (one copy in memory). nn_suppressor_weights weights; }; explicit nn_suppressor(config cfg) : m_cfg(validated(std::move(cfg))) , m_g(m_cfg.weights.geometry) - , m_fft(m_g.frame()) { + , m_fft(m_g.frame()) + , m_dense_in(std::move(m_cfg.weights.dense_in_w), std::move(m_cfg.weights.dense_in_b), m_g.dense, + m_g.features(), tap::dsp::nn::activation::tanh) + , m_gru(std::move(m_cfg.weights.gru_w_ih), std::move(m_cfg.weights.gru_w_hh), + std::move(m_cfg.weights.gru_b_ih), std::move(m_cfg.weights.gru_b_hh), m_g.gru, m_g.dense) + , m_dense_out(std::move(m_cfg.weights.dense_out_w), std::move(m_cfg.weights.dense_out_b), m_g.bands, + m_g.gru, tap::dsp::nn::activation::sigmoid) { build_bands(); const size_t f = m_g.frame(); m_window.resize(f); @@ -142,10 +153,7 @@ namespace tap::mu { m_time.resize(f); m_bin_pow.assign(m_g.bins(), Sample(0)); m_feat.assign(m_g.features(), Sample(0)); - m_dense.assign(m_g.dense, Sample(0)); - m_gates.assign(3 * m_g.gru, Sample(0)); - m_gates_h.assign(3 * m_g.gru, Sample(0)); - m_state.assign(m_g.gru, Sample(0)); + m_hidden.assign(m_g.dense, Sample(0)); m_gains.assign(m_g.bands, Sample(0)); m_bin_gain.assign(m_g.bins(), Sample(0)); m_floor.assign(m_g.bins(), Sample(0)); @@ -170,7 +178,7 @@ namespace tap::mu { std::fill(m_prev_e.begin(), m_prev_e.end(), Sample(0)); std::fill(m_prev_yhat.begin(), m_prev_yhat.end(), Sample(0)); std::fill(m_overlap.begin(), m_overlap.end(), Sample(0)); - std::fill(m_state.begin(), m_state.end(), Sample(0)); + m_gru.reset(); std::fill(m_gains.begin(), m_gains.end(), Sample(0)); std::fill(m_bin_gain.begin(), m_bin_gain.end(), Sample(0)); std::fill(m_floor.begin(), m_floor.end(), Sample(0)); @@ -256,8 +264,6 @@ namespace tap::mu { return cfg; } - static Sample sigmoid(Sample x) noexcept { return Sample(1) / (Sample(1) + std::exp(-x)); } - /// ERB-spaced triangular band weights (features.py band_matrix()). void build_bands() { const size_t bins = m_g.bins(); @@ -406,74 +412,42 @@ namespace tap::mu { } } - /// dense(tanh) -> GRU (PyTorch r,z,n convention) -> dense(sigmoid). + /// dense(tanh) -> GRU (PyTorch r,z,n convention) -> dense(sigmoid), + /// on tap::dsp::nn's kernels. void infer() noexcept { - const nn_suppressor_weights& w = m_cfg.weights; - const size_t nf = m_g.features(); - const size_t nd = m_g.dense; - const size_t ng = m_g.gru; - for (size_t i = 0; i < nd; ++i) { - Sample acc = static_cast(w.dense_in_b[i]); - for (size_t j = 0; j < nf; ++j) { - acc += static_cast(w.dense_in_w[i * nf + j]) * m_feat[j]; - } - m_dense[i] = std::tanh(acc); - } - for (size_t i = 0; i < 3 * ng; ++i) { - Sample acc = static_cast(w.gru_b_ih[i]); - for (size_t j = 0; j < nd; ++j) { - acc += static_cast(w.gru_w_ih[i * nd + j]) * m_dense[j]; - } - m_gates[i] = acc; - acc = static_cast(w.gru_b_hh[i]); - for (size_t j = 0; j < ng; ++j) { - acc += static_cast(w.gru_w_hh[i * ng + j]) * m_state[j]; - } - m_gates_h[i] = acc; - } - for (size_t i = 0; i < ng; ++i) { - const Sample r = sigmoid(m_gates[i] + m_gates_h[i]); - const Sample z = sigmoid(m_gates[ng + i] + m_gates_h[ng + i]); - const Sample n = std::tanh(m_gates[2 * ng + i] + r * m_gates_h[2 * ng + i]); - m_state[i] = (Sample(1) - z) * n + z * m_state[i]; - } - for (size_t b = 0; b < m_g.bands; ++b) { - Sample acc = static_cast(w.dense_out_b[b]); - for (size_t j = 0; j < ng; ++j) { - acc += static_cast(w.dense_out_w[b * ng + j]) * m_state[j]; - } - m_gains[b] = sigmoid(acc); - } + m_dense_in.apply(m_feat.data(), m_hidden.data()); + m_gru.step(m_hidden.data()); + m_dense_out.apply(m_gru.state(), m_gains.data()); } - config m_cfg; - nn_geometry m_g; - basic_real_fft m_fft; - std::vector m_window; - std::vector m_bmat; ///< [bands x bins] triangular weights - std::vector m_bnorm; ///< per-bin 1/sum(band weights) - std::vector m_prev_e; - std::vector m_prev_yhat; - std::vector m_overlap; - std::vector m_spec_e; - std::vector m_spec_y; - std::vector m_time; - std::vector m_bin_pow; ///< current frame per-bin |E|^2 - std::vector m_scratch_pow = std::vector(m_g.bins()); - std::vector m_feat; - std::vector m_dense; - std::vector m_gates; - std::vector m_gates_h; - std::vector m_state; - std::vector m_gains; - std::vector m_bin_gain; - std::vector m_floor; ///< smoothed per-bin E PSD - std::vector m_min_cur; ///< running half-window minimum - std::vector m_min_prev; ///< completed half-window minimum - size_t m_min_count = 0; - Sample m_syy = Sample(0); - Sample m_sdd = Sample(0); - std::uint32_t m_rng = 0x2545F491U; + config m_cfg; + nn_geometry m_g; + basic_real_fft m_fft; + tap::dsp::nn::basic_dense m_dense_in; + tap::dsp::nn::basic_gru m_gru; + tap::dsp::nn::basic_dense m_dense_out; + std::vector m_window; + std::vector m_bmat; ///< [bands x bins] triangular weights + std::vector m_bnorm; ///< per-bin 1/sum(band weights) + std::vector m_prev_e; + std::vector m_prev_yhat; + std::vector m_overlap; + std::vector m_spec_e; + std::vector m_spec_y; + std::vector m_time; + std::vector m_bin_pow; ///< current frame per-bin |E|^2 + std::vector m_scratch_pow = std::vector(m_g.bins()); + std::vector m_feat; + std::vector m_hidden; ///< dense_in output, the GRU's input + std::vector m_gains; + std::vector m_bin_gain; + std::vector m_floor; ///< smoothed per-bin E PSD + std::vector m_min_cur; ///< running half-window minimum + std::vector m_min_prev; ///< completed half-window minimum + size_t m_min_count = 0; + Sample m_syy = Sample(0); + Sample m_sdd = Sample(0); + std::uint32_t m_rng = 0x2545F491U; }; /// Parse weights from an in-memory MUNN image (the format diff --git a/submodules/dsptap b/submodules/dsptap index 41215c6..4e9a29a 160000 --- a/submodules/dsptap +++ b/submodules/dsptap @@ -1 +1 @@ -Subproject commit 41215c6590f6c72e2e30b54ce373b7bc0e7967bd +Subproject commit 4e9a29a65a948002633b0a9ce4a434075db56d28 diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 26cb911..af2d493 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -56,11 +56,6 @@ if(MUTAP_BARE_METAL) # not reliably propagate exit codes through the emulator. target_sources(mutap_tests PRIVATE bare_metal_main.cpp) target_link_libraries(mutap_tests PRIVATE GTest::gtest) - if(MUTAP_ON_TARGET_SOFT_FP64) - # No FP64 hardware (Cortex-M33): bare_metal_main.cpp drops the long - # float PEM scenarios, whose test harness simulates the room in double. - target_compile_definitions(mutap_tests PRIVATE MUTAP_ON_TARGET_SOFT_FP64=1) - endif() add_test(NAME mutap_tests_emulated COMMAND mutap_tests) set_tests_properties(mutap_tests_emulated PROPERTIES PASS_REGULAR_EXPRESSION "MUTAP_TESTS_COMPLETE rc=0" diff --git a/tests/bare_metal_main.cpp b/tests/bare_metal_main.cpp index c81367f..40c5017 100644 --- a/tests/bare_metal_main.cpp +++ b/tests/bare_metal_main.cpp @@ -13,16 +13,13 @@ // target-independent math already covered on every host platform — and the // bisection-heavy ASG measurements beyond the float ones kept. // -// MUTAP_ON_TARGET_SOFT_FP64 (the Cortex-M33 leg: single-precision FPU, no -// FP64) additionally drops the long float PEM scenarios. Their cost is not -// the canceller: the closed-loop and echo harnesses simulate the room in -// double on purpose (tests/support/closed_loop.h convolves the feedback -// path in double per sample; the MSG bisection measures in double), and the -// speech predictor's pitch search accumulates in double (lpc.h). That is -// hardware on the M55 and software on the M33: the tonal PEM headline alone -// measured 1030 s under qemu mps2-an505 against 84 s on mps3-an547. Those -// scenarios stay covered on the M55 and Hexagon legs and every host; the -// M33 leg exists for the pure-float32 paths and the learned suppressor. +// The same selection runs on the Cortex-M33 leg (single-precision FPU, no +// FP64). It once could not: the speech predictor's pitch search accumulated +// in double, which is hardware on the M55 and software on the M33, and the +// tonal PEM headline alone took 1085 s under qemu mps2-an505 (84 s on +// mps3-an547) — the selection timed out. With the search accumulating in +// Sample (lpc.h) the whole selection runs in ~3 minutes there; the test +// harnesses' deliberate double room simulation costs little by comparison. // SPDX-License-Identifier: MIT // Copyright 2026 MuTap contributors #include @@ -40,10 +37,7 @@ int main() { "PemAfcConfigValidation.*:PemAfcRtContract.*:" "closed_loop_test/0.*:" "AdaptationControlConfigValidation.*:" -#ifndef MUTAP_ON_TARGET_SOFT_FP64 - // The long float PEM scenarios (double harness), see the header comment. "kalman_loop_test/0.*:pem_afc_test/0.*:burst_test/0.*:aec_test/0.*:" -#endif "nn_suppressor_test/0.*:NnSuppressorCrossPrecision.*:NnChainFloat32.*"; ::testing::InitGoogleTest(); const int rc = RUN_ALL_TESTS(); From 1c2d6d449cec71d6ba77e0b8369e3ab13d56821d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 5 Sep 2026 12:53:09 +0000 Subject: [PATCH 2/2] Repoint the DspTap pin at the merged tap::dsp::nn tree on main tap/DspTap#15 merged by squash; this is the identical tree, reachable after branch cleanup. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01JuUg1ZBxm3fyBqWgQv6H1G --- submodules/dsptap | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/submodules/dsptap b/submodules/dsptap index 4e9a29a..58210ae 160000 --- a/submodules/dsptap +++ b/submodules/dsptap @@ -1 +1 @@ -Subproject commit 4e9a29a65a948002633b0a9ce4a434075db56d28 +Subproject commit 58210aeb4fde2d798c9d61f28154fdb3e091b95f