Search Results for "tensor"

...The repository provides both a reference training implementation and Megatron Core, a composable library of high-performance building blocks for custom large-model pipelines. It supports advanced parallelism strategies including tensor, pipeline, data, expert, and context parallelism, enabling training across massive multi-GPU and multi-node clusters. The framework includes mixed-precision training options such as FP16, BF16, FP8, and FP4 to maximize performance and memory efficiency on modern hardware. Megatron-LM is widely used in research and industry for pretraining GPT-, BERT-, T5-, and multimodal-style models, with tooling for checkpoint conversion and interoperability with Hugging Face. ...

...The project also includes demo scripts that visualize learning curves and allow students to observe policy improvement over training iterations. By using TensorFlow as the backbone, it highlights practical considerations such as tensor shapes, loss computation, optimization steps, and batching in an RL context.

pytorch-tutorial is a highly popular educational repository that teaches deep learning with PyTorch through step-by-step examples and well-structured lessons. It is designed primarily for beginners and intermediate practitioners who want to understand PyTorch fundamentals and quickly move toward building real neural network models. The repository walks users through core concepts such as tensors, autograd, neural network modules, convolutional networks, recurrent networks, and transfer...