Search Results for "gpu hardware"

Megatron-LM is a GPU-optimized deep learning framework from NVIDIA designed to train extremely large transformer-based language models efficiently at scale. The repository provides both a reference training implementation and Megatron Core, a composable library of high-performance building blocks for custom large-model pipelines. It supports advanced parallelism strategies including tensor, pipeline, data, expert, and context parallelism, enabling training across massive multi-GPU and multi-node clusters. ...

Documentation, simulator, compiler, and Verilog implementation of a completely open-architecture graphics processing unit. This design is intended for academic and commercial purposes. The first step is to develop a detailed GPU simulator and compiler. The second step is to implement the GPU in synthesizable Verilog. The third step is to develop a feedback loop between the simulator and implementation, allowing power, performance, and reliability aspects of the hardware to feed back into ever more detailed and accurate simulations of a complete GPU. ...