Documentation: 1bit.gg · Measured results: wiki · Community: Discord
The 1bit engine runs inside Lemonade. Lemonade stays
the server you talk to (its catalog, downloads, router and UI), and for the models it hands to 1bit,
Lemonade runs the engine as one of its backends, the same way it runs llama-server. The engine
serves each model behind an OpenAI-compatible API (1bit serve), whatever device runs it:
- the XDNA 2 NPU engine
- HRX on the Radeon iGPU, on the llama.cpp + hrx-system pair AMD tests
- Vulkan on the Radeon iGPU, from upstream llama.cpp's latest release, so new architectures land the day upstream ships them; architectures upstream lacks, such as Zyphra's ZAYA1, run from our llama.cpp (docs/vulkan.md)
- MoE experts streamed from the drive on Vulkan (
1bit serve --moe-slots N), for MoE models larger than memory (docs/moe-streaming.md) - a lean option, ROCmFPX's ROCmFP4 and ROCmI4 formats: faster, less accurate (docs/lean.md)
- ZINC, which also reaches NVIDIA GPUs (CUDA) and Apple GPUs (Metal)
- DwarfStar, for DeepSeek V4 Flash, GLM 5.x and Qwen3.8-Flash-Next in its own GGUFs, with SSD expert streaming
- MLX on Apple Silicon, through lemon-mlx-engine
- ONNX Runtime GenAI models (Lemonade's ONNX format) on the CPU, and on the Radeon through ONNX Runtime's WebGPU provider (docs/onnx.md)
- Laya, which decides where each request runs
- ComfyUI.cpp: ComfyUI workflows (Stable Diffusion 1.5 text-to-image and image-to-image) in C++, matching ComfyUI's output to 50 dB (docs/comfyui.md)
- every Hugging Face model architecture, kept current by a daily census
Packages ship every Sunday, rebuilt at that week's upstream pins: Linux, Lemonade with the engine, Windows and 1bit OS (docs/releases.md).
Status: the engine runs inside Lemonade through
1bit serve(docs/lemonade.md, docs/serve.md); the NPU, Vulkan, HRX and ZINC each pass its end-to-end test on Strix Halo, and the Lemonade recipe that runs it (onebit, in our fork 1bit-MONSTER/lemonade) passes Lemonade's own LLM test suite on Vulkan and HRX. Following geramyL's review the engine no longer vendors Lemonade: Lemonade is the host, 1bit is the engine inside it. Ported so far: HRX on AMD's live ggml-hrx (docs/hrx.md), Vulkan from upstream llama.cpp's latest release (docs/vulkan.md), the NPU engine on full ELFs with the upstream XDNA stack pinned (docs/npu.md; its layer kernel is not yet built from source), ZINC (docs/zinc.md) and MLX (docs/apple.md). ZAYA1-8B (Zyphra) runs from our llama.cpp on Vulkan, HRX and ROCm, matching transformers, at 93 tok/s decode in Q4_K_M on Vulkan, and ZAYA1-74B-preview at 35 tok/s (docs/vulkan.md); the rest of Zyphra's family (Zamba, Zamba2, BlackMamba) runs on Vulkan too, and so do its vision models, ZAYA1-VL-8B and Zamba2-VL, through1bit serve --mmproj. Experimental, and closed source: Qwen3.6-35B-A3B on the NPU through a private add-on, parity against fp64 passes, 16.3-16.5 tok/s decode (docs/npu.md). Step 4, the Laya router, has landed as an opt-in:1bit serve --device auto --laya-model <dir>picks the device per request with the C++ scorer, which matches the Python reference; a decision takes 0.38 s on Strix Halo (docs/laya.md). Step 5, the model registry, has landed: of 332,565 HF text-generation models with an architecture, 93.28% are mapped to a backend and 64.18% have an architecture checked end to end on Strix Halo; a daily census keeps the counts current (docs/registry.md). The working engine is being ported from 1bit-MONSTER, our private development repository; see docs/PORTING.md. Measured results are on the wiki. This repository holds the verified code without the development history.
Apache-2.0. See LICENSE.
Osmantic / ODS comes first. ODS introduced me to vibecoding, and that is where all of this started. Without it, this engine would not exist.
The Lemonade team and AMD's developers. Lemonade is where this engine runs. AMD's developers left breadcrumbs all over the place: the XDNA driver and XRT, IRON and Peano, HRX, their tested llama.cpp integration, their issues, their examples. This engine is what following those breadcrumbs built.
The repositories this engine is built on, in order of importance:
| # | Repository | What it gives this engine | License |
|---|---|---|---|
| 1 | amd/xdna-driver | The XDNA 2 NPU driver and its XRT shim | Apache-2.0 (shim) |
| 2 | Xilinx/XRT | The runtime every NPU kernel runs through: full ELFs, hardware contexts, buffers | Apache-2.0 (userspace) |
| 3 | Xilinx/mlir-aie | IRON and aiecc: how our own NPU kernels are written and compiled | Apache-2.0 WITH LLVM-exception |
| 4 | Xilinx/llvm-aie | Peano, the C++ compiler for the NPU's AI Engine cores | Apache-2.0 WITH LLVM-exception |
| 5 | ggml-org/llama.cpp | GGUF inference on the Radeon iGPU: Vulkan (upstream release) and HRX (AMD's tested pair) | MIT |
| 6 | ROCm/hrx-system | HRX, AMD's HIP Runtime Extended, behind the HRX0 device | Apache-2.0 |
| 7 | torvalds/linux | The kernel, with amdxdna and amdgpu in-tree |
GPL-2.0 WITH Linux-syscall-note |
| 8 | zolotukhin/zinc | Its own GPU kernels, and the engine's route to NVIDIA through CUDA | MIT |
| 9 | huggingface/tokenizers | Every model's tokenizer.json, byte-exact, behind our C ABI |
Apache-2.0 |
| 10 | NandhaKishorM/laya | The router that decides where each request runs | Apache-2.0 |
| 11 | ROCm/FastFlowLM | The Q4NX NPU model format and its models on Hugging Face (FastFlowLM/*-NPU2), which the engine's NPU route runs on its own kernels |
MIT |
| 12 | antirez/ds4 (DwarfStar) | DeepSeek V4 Flash, GLM 5.x and Qwen3.8-Flash-Next on its own kernels (ROCm on Strix Halo, CUDA, Metal) | MIT |
Also built on nlohmann/json (MIT) and yhirose/cpp-httplib (MIT).
Every third-party copyright and license is listed in NOTICE. Each project keeps its own license; nothing here relicenses anyone's work.