DeepSpeed
Deep learning optimization library: makes distributed training easy
DeepSpeed is an easy-to-use deep learning optimization software suite that enables unprecedented scale and speed for Deep Learning Training and Inference. With DeepSpeed you can:
1. Train/Inference dense or sparse models with billions or trillions of parameters
2. Achieve excellent system throughput and efficiently scale to thousands of GPUs
3. Train/Inference on resource constrained GPU systems
4.