Download
OLMo-core is a Python library from AI2 that provides reusable PyTorch building blocks for training large language models at scale. It underpins the broader OLMo ecosystem and includes components for models, data loading, distributed execution, optimization, checkpointing, evaluation, and generation. Official scripts reproduce training configurations for released OLMo models. The library supports advanced attention backends, mixture-of-experts architectures, float8 training, and fused low-memory operations through optional dependencies. Jobs can run with standard torchrun or AI2’s Beaker infrastructure. Direct autoregressive generation is also available for experimentation and chat-style inference. The project is Apache-2.0 licensed and designed for transparent, reproducible LLM research.
Features
- Large-scale distributed PyTorch training
- Official OLMo model training scripts
- Mixture-of-experts model support
- Float8 and optimized attention backends
- Data loading, checkpointing, and evaluation tools
- Built-in autoregressive generation
Project Samples
