NeMo Automodel
Pytorch Distributed native training library for LLMs/VLMs
...Its DTensor-native SPMD approach lets the same training code scale from one GPU to large multi-node clusters by changing configuration. Hugging Face integration provides broad model compatibility without requiring format conversion. YAML recipes and CLI overrides keep experiments concise while preserving reproducibility. The library supports composable parallelism, optimized kernels, MoE acceleration, mixed precision, sequence packing, and asynchronous checkpointing. Jobs can run through interactive environments, Slurm, SkyPilot, or Kubernetes-based workflows. ...