build nanoGPT
Video+code lecture on building nanoGPT from scratch
...Its Git history is intentionally organized so learners can follow each architectural and training improvement as it is introduced. The accompanying video lecture explains how the code develops into a reproduction of the 124-million-parameter GPT-2 model. The project covers tokenization, transformer architecture, optimization, distributed training, data loading, and performance improvements. It includes FineWeb data preparation and HellaSwag evaluation utilities. With sufficient computing resources, the same general code can scale toward larger GPT-3-style configurations. ...