Search Results for "library python"

This repository records EleutherAI's library for training large-scale language models on GPUs. Our current framework is based on NVIDIA's Megatron Language Model and has been augmented with techniques from DeepSpeed as well as some novel optimizations. We aim to make this repo a centralized and accessible place to gather techniques for training large-scale autoregressive language models, and accelerate research into large-scale training. For those looking for a TPU-centric codebase, we...

This app allows you to chat with ChatGPT using a reverse-engineered API library called revChatGPT. Replies from the Chatbot are streamed back to the user in real-time, which gives the user an experience similar to how ChatGPT streams back its answers. To get started with the app, you'll need to create an account on OpenAI's ChatGPT and save your credentials. You can choose from three authentication methods: Email/Password, Session token, or Access token. Once you have your credentials, you can...

An implementation of model & data parallel GPT3-like models using the mesh-tensorflow library. If you're just here to play with our pre-trained models, we strongly recommend you try out the HuggingFace Transformer integration. Training and inference is officially supported on TPU and should work on GPU as well. This repository will be (mostly) archived as we move focus to our GPU-specific repo, GPT-NeoX. NB, while neo can technically run a training step at 200B+ parameters, it is very...