LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

A from-scratch Prefill/Decode disaggregation inference engine for LLMs

Prefills factories with faker method suggestions to increase productivity

Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.

Generate pre-filled PO files from POT file, using source PO files in gulp.

Multi-GPU prefill acceleration for llama.cpp

NanoPD, a minimalistic implementation of the disaggregated PD serve

Generate pre-filled PO files from POT file, using source PO files in NodeJS.

Save VLM prefill for VQA tasks by letting a policy select which tokens are important at test-time!

React component to create a button/link to prefill Codepen.io with supplied code.

Measure and visualize why LLM inference is slow: bottleneck analysis, model dissection, KV-cache, GEMM/GEMV, quantization, and memory-bound decoding.

Automatically pre-fill course section CRNs or select a template for instant RPI class registration when your time ticket is activated.

Analyse approfondie de Dual Path (DeepSeek) : architecture double chemin pour charger le cache KV depuis le stockage, mutualisation de la bande passante, +100% de débit d'inférence et -56% sur le premier token sans GPU supplémentaire. Fiche de lecture, mindmap textuelle et Q/R détaillées sur l'optimisation agentique.

Event-driven benchmark of adaptive batch composition policies for LLM serving, measuring how prefill and decode interference affects TTFT, TPOT, and throughput under different memory pressure regimes.

A maintained continuation of celeste's Loom — a recursively-branching LLM interface for Obsidian (AGPL-3.0)

Simulates system prompt KV cache reuse across 540 configurations — measuring prefill savings, TTFT reduction, and eviction policy tradeoffs under variable prompt sizes, output lengths, and competing prompt workloads.

Production-ready LiteLLM Custom Callback to fix Claude 4.x prefill compatibility issue on Amazon Bedrock (resolves LiteLLM #22930)

End-to-end latency profiler for LLM inference showing that decode accounts for 94-97% of total pipeline time. Includes per-phase timing, sensitivity analysis, and predictive latency models with ~2–3% total error.

Benchmark for prefill/decode interference on a single GPU, measuring TTFT inflation when new requests arrive while decode is already running.

A minimalist, educational deep-dive into the vLLM architecture.