Skip to content

About

Speech recognition on the Apple Neural Engine: local transcription that leaves the GPU free

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

ane-speech

Speech recognition on the Apple Neural Engine

Local transcription that leaves the GPU free.

PyPI CI License: MIT Hugging Face Apple Silicon Swift Core AI

Install · Models · Command line · Python · How it works

ane-speech runs open speech-recognition models on the Neural Engine of Apple silicon Macs. Every layer runs there, so transcription stays quick while the GPU is busy with something else: another model, a render, a game. Nothing leaves your Mac.

It grew out of mlx-speech, whose models it converts, and powers the on-device recognition in TNT.

Install

uv tool install ane-speech     # the command line
pip install ane-speech         # the Python library, command line included

You need a Mac with Apple silicon and a macOS release that includes Core AI; on an older macOS, ane-speech says so when it starts.

Models

Name Model Best for
confucius4_r2t2 (default) Confucius4-R2T2 Live transcription: text appears while you speak
qwen3_asr_1_7b Qwen3-ASR 1.7B Whole recordings: one accurate pass once you stop
qwen3_forced_aligner_0_6b Qwen3-ForcedAligner 0.6B Timestamps: when each word of a known transcript was spoken

The recognizers are multilingual, take a hint of names and terms to expect, and match their MLX originals word for word apart from rare near-ties; a recording or live take holds up to about a minute of speech. The aligner times Chinese and English, character by character and word by word; it runs from a source checkout (Building from source), while the PyPI release carries the two recognizers. Each model downloads from Hugging Face the first time you use it.

Command line

ane-speech transcribe meeting.wav --checkpoint qwen3_asr_1_7b
ane-speech transcribe meeting.wav --checkpoint qwen3_asr_1_7b --language Chinese --context "Acme, Kubernetes"
ane-speech stream meeting.wav       # R2T2, played as if spoken live, text printed as it settles
ane-speech align meeting.wav --text "the exact transcript"   # from source: start, end, and word per line
ane-speech pull qwen3_asr_1_7b      # download ahead of time

Python

import ane_speech

asr = ane_speech.load("qwen3_asr_1_7b")
result = asr.generate("meeting.wav")         # a file, or 16 kHz mono float32 samples
print(result.language, result.text)

Live, from a microphone:

live = ane_speech.load("confucius4_r2t2")
session = live.stream_session(chunk_ms=480)
for pcm in microphone:                        # 16 kHz mono float32, any length
    print(session.feed(pcm).committed)        # final text, which only grows
print(session.finalize().text)

Timestamps for a transcript, such as the one above, from a source checkout:

aligner = ane_speech.load("qwen3_forced_aligner_0_6b")
for text, start_s, end_s in aligner.generate("meeting.wav", result.text):
    print(f"{start_s:.2f}  {end_s:.2f}  {text}")

A loaded model runs in a helper process of its own and exits with yours. The method names follow mlx-speech, so code moves between the two.

Disk space

Each model takes space twice, about its own size each time: the download, in the Hugging Face cache, and a copy compiled for your Mac's Neural Engine. The first load compiles it, which takes a while once, and a macOS update compiles it again. To remove both:

rm -rf ~/.cache/huggingface/hub/models--appautomaton--*-ane
rm -rf ~/Library/Caches/coreai-cache/*/ane-speech

How it works

open checkpoint  ->  trace in MLX  ->  convert to Core AI  ->  run on the Neural Engine

Each model is converted once, in Python: its MLX graph is traced, rebuilt for the Neural Engine in fp16, and checked against the original. From then on a small Swift runtime does the work, reading only the converted files, with no PyTorch. The Python package is a thin layer over that runtime, and other apps can use the runtime the same way through ane-speech serve, which answers JSON requests on stdin; TNT does.

Building from source

git clone https://github.com/appautomaton/ane-speech.git
cd ane-speech
swift build -c release
uv run ane-speech transcribe speech.wav    # models still come from Hugging Face
uv run ane-speech align speech.wav --text "the exact transcript"

AGENTS.md covers converting models yourself, the checks behind them, and releases; a coding agent can follow it as written.

License

The code is MIT. The models keep the licenses of their sources: Qwen3-ASR under Apache-2.0, Confucius4-R2T2 under the NetEase Youdao Model Use License Agreement.

Acknowledgements

The Qwen team for Qwen3-ASR, NetEase Youdao for Confucius4-R2T2, and mlx-speech for the MLX implementations this project builds on.

About

Speech recognition on the Apple Neural Engine: local transcription that leaves the GPU free

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages