AI Engineer · IIT Madras '24 · Bengaluru
I build AI infrastructure: reliable agent runtimes, inference systems, and experiments that measure what actually helps. I write about the architecture, results, and trade-offs behind them.
Website · Writing · LinkedIn · X · Email
Mycelium — reliability for agent tool calls
Creator and maintainer of an open-source Python runtime that validates tool inputs and authority, controls retries and concurrency, and records durable outcomes for crash recovery. Integrates with LangGraph, CrewAI, and custom agents; experimental TypeScript and Go clients connect through an HTTP sidecar.
pip install mycelium-runtime · Docs · Design write-up
tiny-reranker — inference performance experiments
Studied a pretrained MiniLM reranker on Apple M5 and NVIDIA L40S, comparing PyTorch, MLX, compilation, custom Metal/CUDA/Triton kernels, and Python/Rust serving.
- 11 implementation audits passed; 12,288 accepted HTTP responses validated.
- Compiled PyTorch reached 736.8 pairs/sec on L40S and 102.1 pairs/sec on M5 in warm direct scoring: FP32, 100 documents/query, batch 16, maximum 512 tokens.
- Custom CUDA/Triton kernels did not beat compiled PyTorch on this workload; Rust coordination showed no practical HTTP throughput advantage.
Results and protocol · Engineering write-up
LLM serving on Apple Silicon — studying the serving stack
An experimental MLX/FastAPI inference server with continuous batching, chunked prefill, prefix caching, paged KV-cache experiments, SSE streaming, and an OpenAI-compatible API. Includes a load-testing harness for latency, time to first token, throughput, and memory.
nanoLLM — modern LLM architecture, one component at a time
Extended nanoGPT with 7 architectural changes: RMSNorm, SwiGLU, RoPE, GQA, MLA, MoE, and multi-token prediction. On Shakespeare character-level experiments (4 layers, 4 heads, 128 dimensions, 2,000 CPU training steps, same seed), the combined stack reduced validation loss from 1.6645 to 1.5259 and reached the baseline's final loss by step 500.
Identiti — personal AI memory
A React/Flask app that extracts confidence-scored entities from conversations into a Neo4j knowledge graph, with Redis-backed history, Supabase authentication, review, and portable export.
TicTic — voice typing for Indian languages
A native SwiftUI macOS app with 23 language choices, automatic language detection, global dictation shortcuts, transliteration, English translation, and direct text insertion into the active app. Powered by Sarvam AI.
- 13 Performance Experiments on One Reranker: The Limits of Faster GPU Code — kernels, compilation, serving, and negative results.
- Mycelium: Turning AI Agents into Reliable Systems — the runtime behind reliable tool execution.
- Checklist to Prevent All the Agent Failure Modes — failure modes and execution controls.
- Building a vLLM-Inspired LLM Serving Engine on Apple Silicon — scheduling, batching, and KV-cache design with MLX.
- Swapped Every Part of nanoGPT With Modern Alternatives — component-by-component architecture experiments.
