Download
makemore is a compact educational framework for training character-level language models on line-separated text datasets. Given examples such as personal names, company names, or dictionary words, it learns their statistical structure and generates new examples with similar patterns. The project intentionally keeps most functionality in one hackable Python file and requires only PyTorch. It includes implementations ranging from simple bigram models to MLPs, recurrent networks, LSTMs, GRUs, and transformers. Training progress, checkpoints, logs, and generated samples are written to a selected working directory. Models can also be loaded in sample-only mode for generating additional outputs without retraining. Its main purpose is to teach how increasingly sophisticated autoregressive language models work.
Features
- Character-level autoregressive generation
- Bigram and multilayer perceptron models
- RNN, LSTM, and GRU architectures
- Transformer model implementation
- Training checkpoints and logging
- Standalone sample generation mode
Project Samples
