Search Results for "tensor"

...The repository provides both a reference training implementation and Megatron Core, a composable library of high-performance building blocks for custom large-model pipelines. It supports advanced parallelism strategies including tensor, pipeline, data, expert, and context parallelism, enabling training across massive multi-GPU and multi-node clusters. The framework includes mixed-precision training options such as FP16, BF16, FP8, and FP4 to maximize performance and memory efficiency on modern hardware. Megatron-LM is widely used in research and industry for pretraining GPT-, BERT-, T5-, and multimodal-style models, with tooling for checkpoint conversion and interoperability with Hugging Face. ...

...The project also includes demo scripts that visualize learning curves and allow students to observe policy improvement over training iterations. By using TensorFlow as the backbone, it highlights practical considerations such as tensor shapes, loss computation, optimization steps, and batching in an RL context.

pytorch-tutorial is a highly popular educational repository that teaches deep learning with PyTorch through step-by-step examples and well-structured lessons. It is designed primarily for beginners and intermediate practitioners who want to understand PyTorch fundamentals and quickly move toward building real neural network models. The repository walks users through core concepts such as tensors, autograd, neural network modules, convolutional networks, recurrent networks, and transfer...

This is the corresponding code for the book "The Deep Learning Framework PyTorch: Getting Started and Practical", but it can also be used as a standalone PyTorch Getting Started Guide and Tutorial. The current version of the code is based on pytorch 1.0.1, if you want to use an older version please git checkout v0.4or git checkout v0.3. Legacy code has better python2/python3 compatibility, CPU/GPU compatibility test. The new version of the code has not been fully tested, it has been tested...