LLM Training Puzzles

LLM Training Puzzles is a notebook-based collection of eight exercises about training large neural networks across many GPUs. It introduces the core primitives behind distributed language model training without requiring access to a thousand-machine cluster. The puzzles focus on memory efficiency, communication, partitioning, and compute pipelining. Learners work through simplified scenarios that expose the tradeoffs hidden inside large training systems. The hands-on format emphasizes deriving behavior rather than memorizing framework commands. Google Colab is recommended so readers can copy the notebook and begin experimenting quickly. The project belongs to a broader puzzle series covering tensors, automatic differentiation, Transformers, GPUs, and related topics.