fix: don't cast Silero VAD to fp16 (MPSGraph abort with --fp16 --vad)

[anubissbe]

Fixes the MPSGraph abort described in #31.

Problem

enable_vad() calls vad_->to_half() whenever the transcriber is fp16. SileroVAD::to_half() casts the VAD weights but leaves its context buffer, LSTM state and audio input in fp32, so the first inference builds a mixed-dtype graph and MPSGraph aborts the process:

'mps.add' op requires the same element type for all operands and results
%3 = "mps.add"(%arg0, %arg1) : (tensor<1x512xf32>, tensor<512xf16>) -> tensor<*xf32>
failed assertion `Error: MLIR pass manager failed'

use_fp16_ in SileroVAD is written but never read, so the cast has nothing backing it.

Change

Stop casting the VAD to fp16 at the three enable_vad() call sites (ParakeetTranscriber, TDTTranscriber, DiarizedTranscriber). The VAD still moves to GPU; only the dtype cast is dropped.

This is deliberately the conservative fix rather than making fp16 VAD work: the VAD is a ~1.2 MB preprocessor whose output is segment boundaries, not tensors entering the ASR graph, so fp32 costs essentially nothing. Making it genuinely fp16 would require the context buffer, LSTM state, input window and the typed_data<float>() result read in process_chunk() all made dtype-aware, plus confirmation that the Silero LSTM is stable in fp16 — which felt like your call to make.

Verification

On macOS 27.0 / Apple Silicon, Metal build, model parakeet-tdt-0.6b-v3 (tdt-600m):

Case	Before	After
--fp16 --vad, request use_vad: true	process aborts	transcribes correctly
--fp16 --vad, request use_vad: false	ok	ok
--fp16, no VAD	ok	ok
fp32 + VAD	ok	ok

Transcript output is unchanged in every case that already worked, and VAD segmentation is identical to the fp32 path (verified on a 20.4 s clip: 6 speech segments, 19.7 s speech / 20.4 s total, same word output).

clang-format reports no new violations on the touched files.

---

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved voice activity detection compatibility when transcription uses FP16.
  • Prevented processing failures caused by mixed FP16 and FP32 data during GPU-based transcription and diarization.
  • VAD now intentionally maintains stable FP32 state (including its internal buffers) while still running on the GPU when enabled.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d1935ba6-a069-41ac-b830-73d7d8409fde

📥 Commits

Reviewing files that changed from the base of the PR and between f66ed42 and bb82249.

📒 Files selected for processing (1)

• src/api/diarize.cpp

🚧 Files skipped from review as they are similar to previous changes (1)

• src/api/diarize.cpp

---

📝 Walkthrough

Walkthrough

VAD setup no longer converts Silero VAD to FP16 in standard, TDT, or diarized transcription paths. VAD remains FP32, while GPU placement continues when enabled.

Changes

VAD precision handling

Layer / File(s)	Summary
Preserve FP32 VAD setup include/parakeet/api/transcribe.hpp, src/api/diarize.cpp	Standard, TDT, and diarized transcribers no longer cast VAD to FP16; GPU placement remains conditional on GPU configuration.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

• Frikallo/parakeet.cpp issue 31 — Removes the VAD FP16 casting associated with the MPSGraph abort described in the issue.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: removing the Silero VAD fp16 cast to fix an MPSGraph abort.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/api/diarize.cpp`:
- Around line 78-79: Update the comment near the VAD casting logic to reference
Transcriber::enable_vad instead of ParakeetTranscriber::enable_vad, preserving
the rest of the explanation unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b8d604ee-5dba-47e2-931e-05da4762d7a3

📥 Commits

Reviewing files that changed from the base of the PR and between a212eea and f66ed42.

📒 Files selected for processing (2)

• include/parakeet/api/transcribe.hpp
• src/api/diarize.cpp