Work in progress. This project is under development; expect missing features and breaking changes.
One complete contract for hosted and local LLMs in Common Lisp. cl-inference/local runs models in-process on cl-tensor; its architectures are declared in a Lisp DSL where a similar architecture inherits from an existing one and overrides what differs.
| Area | Implemented |
|---|---|
| Remote protocols | OpenAI-compatible chat completions; Ollama native chat |
| Remote providers | Built-in :ollama; custom providers through define-provider |
| Local architectures | Llama, Qwen3 and Gemma 3; verified variants in Support |
| Local formats | GGUF v3; Safetensors with HF configuration and indexed shards |
| Inference weights | F32, F16, BF16, Q8_0, Q4_0, Q4_K and Q6_K |
Real-model CPU inference is verified on SBCL; the HTTP client and CPU fixture suites pass on SBCL, CCL and ECL. Safetensors CPU models are fixture-tested, with Gemma 3 270M IT real-model acceptance; the optional MLX backend has Llama 3.2 1B Instruct and Qwen3 0.6B float and 4-bit acceptance. See Support for configuration, verified variants and limitations. Gemma 3 270M IT has CPU/MLX floating Safetensors and Q8_0 GGUF acceptance. ONNX has SmolLM2 135M FP32 CPU acceptance.
- Start here: Getting started, Runnable examples
- Calling models: Client, Local backend, MLX, MLX GGUF, ONNX
- Extending models: Architectures, Tokenizers, Chat templates, Blocks
- Generation: Sessions, Sampling
- Storage: Weights, GGUF, Safetensors, Quantized weights
- Development: Design, Testing, Model acceptance, Limitations
- Planning: Cross-repository roadmap
Work is tracked at https://github.com/communal-software/cl-inference/issues.