fail to load nemotron-3.5-asr-streaming-0.6b

[sirolf99]

fail to load nemotron-3.5-asr-streaming-0.6b-q4_k.gguf
even after

./parakeet-cli quantize nemotron-3.5-asr-streaming-0.6b-q4_k.gguf nemo-35-out.gguf q4_k
quantize: nemotron-3.5-asr-streaming-0.6b-q4_k.gguf -> nemo-35-out.gguf [q4_k]
quantized 0 tensor(s), copied 681 verbatim, 0 allowlisted kept F32 (block).

(linux vulkan)

[dthuerck]

I've had the same problem. Turns out, NeMo 2.7.3 (the newest release of NVIDIA's NeMo Speech library on pypi). However, this release is from April '26 and thus predates the ASR model (see https://github.com/NVIDIA-NeMo/Speech). I've had success with using that repo's main branch instead of the 2.7.3 release. Attaching a script my agent wrote (ofc I did the review); it outputs a converted q8 model that works successfully with this repo.

@mudler Could we use upstream NeMo for the converter in the project? I'd be happy to craft a PR.

convert_with_nemo_main.sh