Help Support TensorFold · @tensorfold_dev on X
TensorFold serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. The engine reads checkpoints, tokenizes requests and runs Metal or CUDA kernels directly. Serving needs no Python or MLX installation.
Building with a coding agent? Have it load the contributor skill and follow CONTRIBUTING.md.
Start with Qwen3.5-2B, a small 4-bit checkpoint. The commands below download its pinned revision, then serve it as
local-model. Model weights have their own licenses.
Install with Homebrew, then start the server:
brew install ashhart/tensorfold/tensorfold
tensorfold pull mlx-community/Qwen3.5-2B-MLX-4bit@93760be4f1f69842a46bc13dbdc0f19e291392a3
tensorfold serve mlx-community/Qwen3.5-2B-MLX-4bit --name local-model --context 8192 --no-thinkingNeeds glibc 2.28 or newer and NVIDIA driver R580 or newer (CUDA 13.0). The run guide lists the archive's requirements; Windows has its own guide.
V=1.3.0
case "$(uname -m)" in
aarch64|arm64) PLATFORM=linux-aarch64 ;;
x86_64) PLATFORM=linux-x86_64 ;;
*) echo 'No release archive for this CPU' >&2; exit 1 ;;
esac
curl -fLO "https://github.com/ashhart/TensorFold/releases/download/v$V/tensorfold-$V-$PLATFORM.tar.gz"
curl -fLO "https://github.com/ashhart/TensorFold/releases/download/v$V/tensorfold-$V-$PLATFORM.tar.gz.sha256"
sha256sum -c "tensorfold-$V-$PLATFORM.tar.gz.sha256"
tar -xzf "tensorfold-$V-$PLATFORM.tar.gz"
export PATH="$PWD/tensorfold-$V-$PLATFORM/bin:$PATH"
tensorfold() { command tensorfold-native "$@"; }
tensorfold pull mlx-community/Qwen3.5-2B-MLX-4bit@93760be4f1f69842a46bc13dbdc0f19e291392a3
tensorfold serve mlx-community/Qwen3.5-2B-MLX-4bit --name local-model --context 8192 --no-thinkingLeave the server running. In another terminal, on either platform:
curl -fsS http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"local-model","messages":[{"role":"user","content":"Say hello in one sentence."}],"max_tokens":128,"temperature":0}'The API also serves completions, Responses and Anthropic Messages. /v1/models lists the served name.
To talk to a model in the terminal in one command, tensorfold chat MODEL starts the server itself, streams each
reply and stops the server when the chat ends; /help lists the chat's commands, and /bye or Ctrl-D ends it:
tensorfold chat mlx-community/Qwen3.5-2B-MLX-4bitWith the server above still running, tensorfold chat --url http://127.0.0.1:8080 chats with it instead, and
echo "Say hello." | tensorfold chat --url http://127.0.0.1:8080 prints one reply for a script.
See the chat command for its flags and the serve flags it passes on.
- Run guide: platform setup, requests, parallel requests, exact decoding, context compaction, Cortheon mode, every serve flag, the Mac app and TensorFold Starlight.
- Supported models: which checkpoints run on which machines, models on disk and model discovery.
- Living Weights (experimental): teach a running model new facts in its own weights.
- Installation runbook: macOS and CUDA installation, paired Metal serving and troubleshooting.
- Windows, TensorFold Web and the Mac app.
- API, Cortheon mode and known issues.
- Evaluating TensorFold and community benchmarks.
- Usable memory profiles: how to record applied budgets, context, cache state and process peaks.
- Contributing: ATC and agents, builds, tests and the exactness and performance gates.
- Changes, 1.0.0 release notes and the Zig engine preview.
- TensorFold is Apache-2.0: LICENSE, NOTICE, THIRD_PARTY_NOTICES.md and SECURITY.md.
Thank you to everyone who has sent TensorFold a pull request, a measurement or a bug report:
@0mao0, @321sssrt-bit, @AbrahamCete, @aditya1503, @AdrianBinDC, @ajensenwaud, @akol1, @alenperic, @Alexbob0, @alexderz, @anvilsong, @Arminova, @b-ostrov, @BadAd84, @barelyworkingcode, @Benjamin-Wegener, @benthecarman, @benwilson, @BHCC2025, @Bizuayeu, @BlivionIaG, @BobClawblaw, @borodach23, @Boscoeuk, @boxabirds, @brandonmmusic-max, @bunnyfu, @CanadaApollo6, @CerebralCoding, @cesarswong, @chadhurley25075-png, @chaog992, @Charlie-Louis, @Chedrian07, @chris247474, @christrade215, @crescit, @cshintov, @cwschroeder, @DakotaTexas, @danyo1399, @Deesha08, @Defilan, @DevRico003, @di37, @diwakergupta, @drowzeys, @easthambuilds, @ecohash-co, @edurdias, @eleqtrizit, @ericlsimplifi, @EugeneClaw, @fche, @feni6, @florianbrede-ayet, @gbgbgbg, @GDACONSULT, @gecobattya, @gilby, @Gogo6969, @gprot42, @GraithSecurity, @grantoverton, @grearjake-star, @greatyingzi, @GustavBlack, @harrisonfriia, @haxudev, @heitke, @hichaiuse, @ivanfioravanti, @jasontitus, @javierbits, @jayleaton, @jeffpeng3, @jeidbugs404, @Jenpo, @jerryfane, @jetnet, @jishnuvenugopal, @jkuepker, @joeynyc, @johnymoo, @JordiPosthumus, @joshuaswarren, @JRaxworthy, @jrb-ai, @jschmied, @juliankang4, @justincasey, @kelchm, @kieranscotney, @kingjamez, @kky42, @laudney, @lcgutierrez, @LECYWZA, @lijian1999, @liumorrisclaw, @LXD-8, @m-naoki-m, @mapamalu, @mcclanahanaman, @MESevenJourney, @mgoldwasser, @MiaAI-Lab, @mikolaj92, @millaguie, @Mirrdhyn, @Moutonc, @MovieMaker93, @mrpmorris, @mrv777, @MV10, @NeoAiLabs, @Nipale-ai, @nmcbride, @nood-co1, @nullburn, @olexale, @omar16100, @optimisme, @otarkhan, @outcastofmusic, @paragontasx, @peacockesq, @philip-pentatonic, @plotarmordev, @pmeenan, @ppardus, @pulseandthread, @quigles1977, @rafafortes, @raymondkpwong, @robertpitt, @rootsbymenda, @RoscoeTT, @RS91Dev, @russellmm, @salmanarshad321, @samwang0041-star, @sanjaibalajee, @satindergrewal, @scottleimroth, @sethforprivacy, @sfxnz, @shantanugoel, @simon-lin88, @simonmd, @spenchey, @squarrier, @ss-cong, @Stephane-Eden, @styles01, @SvangenStudios, @swbiggart, @SxMShaDoW, @sxuff, @taussoe, @tfolkman, @ThinkOffApp, @ThomasHalwax, @Thotheris, @tinyapps, @tolewis, @tomByrer, @tonydehnke, @tournierjc, @tovereast, @tpischke, @urtho, @valacroix, @vcruz305, @vinicius-symetrix, @wojo, @xjqx2z, @yguanrong, @Yuepixel, @YvesLaRose, @ZackSample, @zweck.
docs/credits.md names whose Python engine work each part of the native engine carries.
