Search Results for "transformer"
Sort By:
Showing 27 open source projects for "transformer"
View related business solutions-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Transformer Engine
A library for accelerating Transformer models on NVIDIA GPUs
Transformer Engine (TE) is a library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper GPUs, to provide better performance with lower memory utilization in both training and inference. TE provides a collection of highly optimized building blocks for popular Transformer architectures and an automatic mixed precision-like API that can be used seamlessly with your framework-specific code. -
2
RF-DETR
RF-DETR is a real-time object detection and segmentation
RF-DETR is an open-source computer vision framework that implements a real-time object detection and instance segmentation model based on transformer architectures. Developed by Roboflow, the project builds upon modern vision transformer backbones such as DINOv2 to achieve strong accuracy while maintaining efficient inference speeds suitable for real-time applications. The model is designed to detect objects and segment them within images or video streams using a unified detection pipeline. ... -
3
FlashAttention
Fast and memory-efficient exact attention
FlashAttention is a high-performance deep learning optimization library that reimplements the attention mechanism used in transformer models to be significantly faster and more memory-efficient than standard implementations. It achieves this by using IO-aware algorithms that minimize memory reads and writes, reducing the quadratic memory overhead typically associated with attention operations. The project provides implementations of FlashAttention, FlashAttention-2, and newer iterations optimized for modern GPU architectures such as NVIDIA Hopper and AMD accelerators. ... -
4
x-transformers
A simple but complete full-attention transformer
A simple but complete full-attention transformer with a set of promising experimental features from various papers. Proposes adding learned memory key/values prior to attending. They were able to remove feedforwards altogether and attain a similar performance to the original transformers. I have found that keeping the feedforwards and adding the memory key/values leads to even better performance. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
TabPFN
Foundation Model for Tabular Data
TabPFN is an open-source machine learning system that introduces a foundation model designed specifically for tabular data analysis. The model is based on transformer architectures and implements a prior-data fitted network that can perform supervised learning tasks such as classification and regression with minimal configuration. Unlike many traditional machine learning workflows that require extensive hyperparameter tuning and training cycles, TabPFN is pre-trained to perform inference directly on tabular datasets. ... -
6
Advanced AI explainability for PyTorch
Advanced AI Explainability for computer vision
...The project implements Grad-CAM and several related visualization methods that highlight the regions of an image that most strongly influence a neural network’s decision. These visualization techniques allow developers and researchers to better understand how convolutional neural networks and transformer-based vision models make predictions. The library supports a wide variety of tasks including image classification, object detection, semantic segmentation, and similarity analysis. It also provides metrics and evaluation tools that help measure the reliability and quality of the generated explanations. By integrating easily with PyTorch models, the library allows developers to diagnose model errors, detect biases in datasets, and improve model transparency. -
7
Colossal-AI
Making large AI models cheaper, faster and more accessible
The Transformer architecture has improved the performance of deep learning models in domains such as Computer Vision and Natural Language Processing. Together with better performance come larger model sizes. This imposes challenges to the memory wall of the current accelerator hardware such as GPU. It is never ideal to train large models such as Vision Transformer, BERT, and GPT on a single GPU or a single machine. -
8
Nixtla TimeGPT
TimeGPT-1: production ready pre-trained Time Series Foundation Model
TimeGPT is a production ready, generative pretrained transformer for time series. It's capable of accurately predicting various domains such as retail, electricity, finance, and IoT with just a few lines of code. Whether you're a bank forecasting market trends or a startup predicting product demand, TimeGPT democratizes access to cutting-edge predictive insights, eliminating the need for a dedicated team of machine learning engineers. -
9
AiLearning-Theory-Applying
Quickly get started with AI theory and practical applications
AiLearning-Theory-Applying is a comprehensive educational repository designed to help learners quickly understand artificial intelligence theory and apply it in practical machine learning and deep learning projects. The repository provides extensive tutorials covering mathematical foundations, machine learning algorithms, deep learning concepts, and modern large language model architectures. It includes well-commented notebooks, datasets, and implementation examples that allow learners to... -
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
10
TorchCode
Practice implementing softmax, attention, GPT-2 and more
...The platform provides a collection of curated problems that cover fundamental topics such as activation functions, normalization layers, attention mechanisms, and full transformer architectures. It runs in a Jupyter-based environment, allowing users to write, test, and debug their code interactively while receiving immediate feedback. An automated judging system evaluates correctness, gradient flow, and numerical stability, helping users understand both functional and theoretical aspects of their implementations. -
11
TimeMixer
Decomposable Multiscale Mixing for Time Series Forecasting
...The model focuses on decomposing time series data into multiple temporal scales in order to capture both short-term seasonal patterns and long-term trends. Instead of relying on traditional recurrent or transformer-based architectures, TimeMixer is implemented as a fully multilayer perceptron–based model that performs temporal mixing across different resolutions of the data. The architecture introduces specialized components such as Past-Decomposable-Mixing blocks, which extract information from historical sequences at different scales, and Future-Multipredictor-Mixing modules that combine predictions from multiple forecasting paths. ... -
12
BitNet
BitNet: Scaling 1-bit Transformers for Large Language Models
BitNet is a machine learning research implementation that explores extremely low-precision neural network architectures designed to dramatically reduce the computational cost of large language models. The project implements the BitNet architecture described in research on scaling transformer models using extremely low-bit quantization techniques. In this approach, neural network weights are quantized to approximately one bit per parameter, allowing models to operate with far lower memory usage than traditional 16-bit or 32-bit neural networks. The architecture introduces specialized layers such as BitLinear, which replace standard linear projections in transformer networks with quantized operations. ... -
13
Transformer Reinforcement Learning X
A repo for distributed training of language models with Reinforcement
trlX is a distributed training framework designed from the ground up to focus on fine-tuning large language models with reinforcement learning using either a provided reward function or a reward-labeled dataset. Training support for Hugging Face models is provided by Accelerate-backed trainers, allowing users to fine-tune causal and T5-based language models of up to 20B parameters, such as facebook/opt-6.7b, EleutherAI/gpt-neox-20b, and google/flan-t5-xxl. For models beyond 20B parameters,... -
14
MetaTransformer
Meta-Transformer for Unified Multimodal Learning
We're thrilled to present OneLLM, an ensembling Meta-Transformer framework with Multimodal Large Language Models, which performs multimodal joint training, supports more modalities including fMRI, Depth, and Normal Maps, and demonstrates very impressive performances on 25 benchmarks. -
15
picoGPT
An unnecessarily tiny implementation of GPT-2 in NumPy
picoGPT is a minimal implementation of the GPT-2 language model designed to demonstrate how transformer-based language models work at a conceptual level. The repository focuses on educational clarity rather than production performance, implementing the core components of the GPT architecture in a concise and readable way. It allows users to understand how tokenization, transformer layers, attention mechanisms, and autoregressive text generation operate in modern large language models. ... -
16
DialoGPT
Large-scale pretraining for dialogue
DialoGPT is an open-source conversational language model developed by Microsoft Research for generating natural dialogue responses using large-scale transformer architectures. The system is built on the GPT-2 architecture and is designed specifically for multi-turn conversation tasks, enabling machines to produce coherent responses during interactive dialogue. The model was trained on a massive dataset of approximately 147 million conversational exchanges extracted from Reddit discussion threads, allowing it to learn patterns of natural human conversation. ... -
17
Alphafold2
Unofficial Pytorch implementation / replication of Alphafold2
...The underlying assumption is that the trunk works on the residue level, and then constitutes to atomic level for the structure module, whether it be SE3 Transformers, E(n)-Transformer, or EGNN doing the refinement. -
18
Deep learning time series forecasting
Deep learning PyTorch library for time series forecasting
...It provides all the latest state-of-the-art models (transformers, attention models, GRUs) and cutting-edge concepts with easy-to-understand interpretability metrics, cloud provider integration, and model serving capabilities. Flow Forecast was the first time series framework to feature support for transformer-based models and remains the only true end-to-end deep learning for time series forecasting framework. Currently, Task-TS from CoronaWhy primarily maintains this repository. Pull requests are welcome. Historically, this repository provided open-source benchmarks and codes for flash flood and river flow forecasting. Full transformer (SimpleTransformer in model_dict): The full original transformer with all 8 encoder and decoder blocks. ... -
19
StudioGAN
StudioGAN is a Pytorch library providing implementations of networks
...Moreover, StudioGAN provides an unprecedented-scale benchmark for generative models. The benchmark includes results from GANs (BigGAN-Deep, StyleGAN-XL), auto-regressive models (MaskGIT, RQ-Transformer), and Diffusion models (LSGM++, CLD-SGM, ADM-G-U). StudioGAN is a self-contained library that provides 7 GAN architectures, 9 conditioning methods, 4 adversarial losses, 13 regularization modules, 6 augmentation modules, 8 evaluation metrics, and 5 evaluation backbones. Among these configurations, we formulate 30 GANs as representatives. ... -
20
Hugging Face Transformer
CPU/GPU inference server for Hugging Face transformer models
Optimize and deploy in production Hugging Face Transformer models in a single command line. At Lefebvre Dalloz we run in-production semantic search engines in the legal domain, in the non-marketing language it's a re-ranker, and we based ours on Transformer. In that setup, latency is key to providing a good user experience, and relevancy inference is done online for hundreds of snippets per user query. -
21
Reformer PyTorch
Reformer, the efficient Transformer, in Pytorch
This is a Pytorch implementation of Reformer. It includes LSH attention, reversible network, and chunking. It has been validated with an auto-regressive task (enwik8). -
22
Trax
Deep learning with clear code and speed
Trax is an end-to-end library for deep learning that focuses on clear code and speed. It is actively used and maintained in the Google Brain team. Run a pre-trained Transformer, create a translator in a few lines of code. Features and resources, API docs, where to talk to us, how to open an issue and more. Walkthrough, how Trax works, how to make new models and train on your own data. Trax includes basic models (like ResNet, LSTM, Transformer) and RL algorithms (like REINFORCE, A2C, PPO). It is also actively used for research and includes new models like the Reformer and new RL algorithms like AWR. ... -
23
SRU
Training RNNs as Fast as CNNs
...SRU achieves 5--9x speed-up over cuDNN-optimized LSTM on classification and question answering datasets, and delivers stronger results than LSTM and convolutional models. We also obtain an average of 0.7 BLEU improvement over the Transformer model on the translation by incorporating SRU into the architecture. The experimental code and SRU++ implementation are available on the dev branch which will be merged into master later. -
24
PORORO
Platform of neural models for natural language processing
pororo performs Natural Language Processing and Speech-related tasks. It is easy to solve various subtasks in the natural language and speech processing field by simply passing the task name. Recognized speech sentences using the trained model. Currently English, Korean and Chinese support. Get vector or find similar words and entities from pretrained model using Wikipedia. -
25
TensorFlow 2.0 Tutorials
TensorFlow 2.x version's Tutorials and Examples
...These examples cover a wide range of topics including convolutional neural networks, recurrent neural networks, generative adversarial networks, autoencoders, and transformer-based models such as GPT and BERT. Each section of the repository includes runnable code and structured experiments designed to illustrate how different architectures and algorithms function in real applications. The tutorials use well-known benchmark datasets such as MNIST, CIFAR, and Fashion-MNIST to demonstrate practical model training and evaluation workflows.
