A learning implementation of an automatic differentiation engine, inspired by Andrej Karpathy's micrograd.
This is a personal learning repository built to study and understand the fundamentals of neural networks and backpropagation. It implements a minimal automatic differentiation (autograd) engine in Python that enables building and training simple neural networks from scratch.
The project is based on Andrej Karpathy's original micrograd implementation and serves as a hands-on exploration of how deep learning frameworks compute gradients and optimize neural network parameters.
- Automatic Differentiation: A lightweight autograd engine supporting scalar values and computational graphs
- Backpropagation: Full gradient computation through computational graphs using the chain rule
- Neural Network Layers: Basic building blocks for constructing neural networks
- Visualization: Integration with graphviz for visualizing computational graphs
- Scientific Computing: Leverages numpy and matplotlib for numerical operations and visualization
- PyTorch Compatibility: Familiar API inspired by PyTorch for easy learning transition
micrograd/
├── README.md # Project documentation
├── micrograd/ # Core autograd engine
├── examples/ # Usage examples and demonstrations
├── tests/ # Test suite
└── [other files] # Configuration and auxiliary files
- Python 3: Core implementation language
- NumPy: Numerical computing
- Matplotlib: Data visualization
- Graphviz: Computational graph visualization
- PyTorch: Reference implementation and testing
MIT License - See LICENSE file for details
This implementation closely follows Andrej Karpathy's educational approach to teaching automatic differentiation and neural networks from first principles.