Search Results for "tensor"

DualPipe is a bidirectional pipeline parallelism algorithm open-sourced by DeepSeek, introduced in their DeepSeek-V3 technical framework. The main goal of DualPipe is to maximize overlap between computation and communication phases during distributed training, thus reducing idle GPU time (i.e. “pipeline bubbles”) and improving cluster efficiency. Traditional pipeline parallelism methods (e.g. 1F1B or staggered pipelining) leave gaps because forward and backward phases can’t fully overlap...

...Rubix ML utilizes a versatile modular architecture that is defined by a few key abstractions and their types and interfaces. Train models in a fraction of the time by installing the optional Tensor extension powered by C. Learners such as neural networks will automatically get a performance boost.

AlphaTensor, developed by Google DeepMind, is the research codebase accompanying the 2022 Nature publication “Discovering faster matrix multiplication algorithms with reinforcement learning.” The project demonstrates how reinforcement learning can be used to automatically discover efficient algorithms for matrix multiplication — a fundamental operation in computer science and numerical computation. The repository is organized into four main components: algorithms, benchmarking,...