Search Results for "framework"

...We developed efficient, model-parallel (tensor, sequence, and pipeline), and multi-node pre-training of transformer based models such as GPT, BERT, and T5 using mixed precision. Megatron is also used in NeMo Megatron, a framework to help enterprises overcome the challenges of building and training sophisticated natural language processing models with billions and trillions of parameters. Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.

This repository records EleutherAI's library for training large-scale language models on GPUs. Our current framework is based on NVIDIA's Megatron Language Model and has been augmented with techniques from DeepSpeed as well as some novel optimizations. We aim to make this repo a centralized and accessible place to gather techniques for training large-scale autoregressive language models, and accelerate research into large-scale training. For those looking for a TPU-centric codebase, we recommend Mesh Transformer JAX. ...

A composable GAN built for developers, researchers, and artists. HyperGAN builds generative adversarial networks in PyTorch and makes them easy to train and share. HyperGAN is currently in pre-release and open beta. Everyone will have different goals when using hypergan. HyperGAN is currently beta. We are still searching for a default cross-data-set configuration. Each of the examples supports search. Automated search can help find good configurations. If you are unsure, you can start with...