Diff Zoo

Diff-zoo is a learning-focused handbook designed to demystify algorithmic differentiation (AD), the core technique powering modern machine learning frameworks. The project introduces AD from a foundational calculus perspective and gradually builds towards toy implementations that resemble systems like PyTorch and TensorFlow. It clarifies the differences and connections between forward mode, reverse mode, symbolic, numeric, tracing, and source transformation approaches to differentiation. Unlike production-grade AD systems that are often obscured by complex implementation details, these examples are deliberately simple and coherent to highlight the fundamental ideas. The repository is organized as a set of Julia notebooks, allowing learners to explore concepts interactively and compare different methods side by side. By stripping away unnecessary complexity, diff-zoo serves as both an educational resource and a practical guide for anyone.