Search Results for "transformer"
Sort By:
Showing 356 open source projects for "transformer"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Transformer Debugger
Tool for exploring and debugging transformer model behaviors
Transformer Debugger (TDB) is a research tool developed by OpenAI’s Superalignment team to investigate and interpret the behaviors of small language models. It combines automated interpretability methods with sparse autoencoders, enabling researchers to analyze how specific neurons, attention heads, and latent features contribute to a model’s outputs. -
2
Transformer Engine
A library for accelerating Transformer models on NVIDIA GPUs
Transformer Engine (TE) is a library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper GPUs, to provide better performance with lower memory utilization in both training and inference. TE provides a collection of highly optimized building blocks for popular Transformer architectures and an automatic mixed precision-like API that can be used seamlessly with your framework-specific code. -
3
Vision Transformer Pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA
...It’s widely used as an educational reference for people learning transformers in vision and as a lightweight baseline for research prototypes. The project encourages experimentation—swap optimizers, change augmentations, or plug the transformer backbone into downstream tasks. -
4
Megatron
Ongoing research training transformer models at scale
Megatron is a large, powerful transformer developed by the Applied Deep Learning Research team at NVIDIA. This repository is for ongoing research on training large transformer language models at scale. We developed efficient, model-parallel (tensor, sequence, and pipeline), and multi-node pre-training of transformer based models such as GPT, BERT, and T5 using mixed precision. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Buzz
Buzz transcribes and translates audio offline
...Speech separation can improve results on noisy recordings, while speaker identification distinguishes voices within transcribed media. Multiple Whisper backends, Transformer models, and GPU acceleration options provide flexibility across different computers. Transcripts can be searched, reviewed alongside playback, exported in common subtitle formats, automated through a CLI, or extended with plugins. -
6
spacy-transformers
Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy
...These techniques can be used to import knowledge from raw text into your pipeline, so that your models are able to generalize better from your annotated examples. You can convert word vectors from popular tools like FastText and Gensim, or you can load in any pre trained transformer model if you install spacy-transformers. You can also do your own language model pretraining via the spacy pre train command. You can even share your transformer or another contextual embedding model across multiple components, which can make long pipelines several times more efficient. To use transfer learning, you’ll need at least a few annotated examples for what you’re trying to predict. -
7
CTranslate2
Fast inference engine for Transformer models
CTranslate2 is a C++ and Python library for efficient inference with Transformer models. The project implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering, etc., to accelerate and reduce the memory usage of Transformer models on CPU and GPU. The execution is significantly faster and requires less resources than general-purpose deep learning frameworks on supported models and tasks thanks to many advanced optimizations: layer fusion, padding removal, batch reordering, in-place operations, caching mechanism, etc. ... -
8
MiniMax H3
MiniMax H3 is a general-purpose, omni-modal generative system
...Ref2VA accepts multiple reference images, videos, and audio clips for more controlled results. Its architecture uses a unified multimodal sequence and a 33-billion-parameter H3-Omni-Transformer that jointly predicts video and audio latents. The repository also provides model components, inference code, prompt guidance, and specialized generation skills. -
9
OpenMythos
A theoretical reconstruction of the Claude Mythos architecture
OpenMythos is an experimental, open-source implementation that attempts to reconstruct a hypothesized architecture behind advanced language models using a design called a Recurrent-Depth Transformer. The project explores the idea that instead of stacking hundreds of unique transformer layers, a smaller set of layers can be reused iteratively during inference to achieve deeper reasoning without increasing parameter count. It divides computation into three main stages, including a pre-processing phase, a looped recurrent reasoning block, and a final output refinement stage, creating a structured pipeline for inference. ... -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
10
Detoxify
Trained models & code to predict toxic comments
Detoxify is a deep learning-based tool for detecting and filtering toxic language in online conversations, leveraging Transformer models for high accuracy. -
11
Heretic
Fully automatic censorship removal for language models
Heretic is an open-source Python tool that automatically removes the built-in censorship or “safety alignment” from transformer-based language models so they respond to a broader range of prompts with fewer refusals. It works by applying directional ablation techniques and a parameter optimization strategy to adjust internal model behaviors without expensive post-training or altering the core capabilities. Designed for researchers and advanced users, Heretic makes it possible to study and experiment with uncensored model responses in a reproducible, automated way. ... -
12
Transformers & LLMs cheatsheet
VIP cheatsheet for Stanford's CME 295 Transformers and Large Language
Transformers & LLMs cheatsheet is an educational repository focused on transformers, large language models, and modern generative AI architectures based on Stanford CME 295 coursework. The project compiles structured notes, mathematical explanations, diagrams, and implementation references covering transformer internals, attention mechanisms, tokenization, training pipelines, and inference strategies. It is designed to help students and practitioners understand the technical foundations behind contemporary AI systems such as GPT-style models and multimodal architectures. The repository emphasizes conceptual clarity while still addressing practical engineering considerations involved in training and scaling transformer models. ... -
13
RF-DETR
RF-DETR is a real-time object detection and segmentation
RF-DETR is an open-source computer vision framework that implements a real-time object detection and instance segmentation model based on transformer architectures. Developed by Roboflow, the project builds upon modern vision transformer backbones such as DINOv2 to achieve strong accuracy while maintaining efficient inference speeds suitable for real-time applications. The model is designed to detect objects and segment them within images or video streams using a unified detection pipeline. ... -
14
MoBA
MoBA: Mixture of Block Attention for Long-Context LLMs
...The approach allows language models to scale to significantly longer input contexts without the quadratic computational cost normally associated with transformer attention mechanisms. -
15
Pyreft
ReFT: Representation Finetuning for Language Models
PyreFT is a tool by Stanford NLP for fine-tuning transformer models with an emphasis on efficient, resource-conserving training and customizability for NLP tasks. -
16
Megatron-LM
Ongoing research training transformer models at scale
Megatron-LM is a GPU-optimized deep learning framework from NVIDIA designed to train extremely large transformer-based language models efficiently at scale. The repository provides both a reference training implementation and Megatron Core, a composable library of high-performance building blocks for custom large-model pipelines. It supports advanced parallelism strategies including tensor, pipeline, data, expert, and context parallelism, enabling training across massive multi-GPU and multi-node clusters. ... -
17
Minecraft Development for IntelliJ
Plugin for IntelliJ IDEA that gives special support for Minecraft mods
Experience first class support for all of the major Java Minecraft development platforms, including Bukkit and derivatives such as Spigot and Paper, Sponge, Forge, Fabric, MCP, Mixins, LiteLoader, BungeeCord, and Waterfall. It also provides in-depth support for Access Transformer and NBT files, and more. Because of this, you can install the plugin through IntelliJ's internal plugin browser. Navigate to File -> Settings -> Plugins and click the Browse Repositories... button at the bottom of the window. In the search box, simply search for Minecraft. You can install it from there and restart IntelliJ to activate the plugin. -
18
ACE-Step 1.5
The most powerful local music generation model
ACE-Step 1.5 is an advanced open-source foundation model for AI-driven music generation that pushes beyond traditional limitations in speed, musical coherence, and controllability by innovating in architecture and training design. It integrates cutting-edge generative techniques—such as diffusion-based synthesis combined with compressed autoencoders and lightweight transformer elements—to produce high-quality full-length music tracks with rapid inference times, capable of generating a complete song in seconds on modern GPUs while remaining efficient enough to run on consumer-grade hardware with minimal memory requirements. Beyond straightforward text-to-music synthesis, ACE-Step 1.5 enables flexible creative workflows, including tasks like cover generation, editing existing tracks, transforming vocals to background accompaniment, and stylistic personalization using low-rank adaptation from just a few example songs. -
19
Z-Image
Image generation model with single-stream diffusion transformer
Z-Image is an efficient, open-source image generation foundation model built to make high-quality image synthesis more accessible. With just 6 billion parameters — far fewer than many large-scale models — it uses a novel “single-stream diffusion Transformer” architecture to deliver photorealistic image generation, demonstrating that excellence does not always require extremely large model sizes. The project includes several variants: Z-Image-Turbo, a distilled version optimized for speed and low resource consumption; Z-Image-Base, the full-capacity foundation model; and Z-Image-Edit, fine-tuned for image editing tasks. ... -
20
parcel/css
A CSS parser, transformer, and minifier written in Rust
A CSS parser, transformer, and minifier written in Rust. Parsing and minifying large files are completed in milliseconds, often with significantly smaller output than other tools. Many other CSS parsers treat property values as an untyped series of tokens. This means that each transformer that wants to do something with these values must interpret them itself, leading to duplicate work and inconsistencies. -
21
SeedVR
Repo for SeedVR2 & SeedVR
...SeedVR’s transformer-based design allows it to handle variable frame resolutions and lengths, and its architecture is optimized to overcome traditional limitations of windowed attention in high-resolution contexts. -
22
UniMate
UniMate: One Unified Model to Animate Diverse Skeletons
...It is designed to handle bipeds, quadrupeds, birds, marine creatures, insects, serpentine rigs, and articulated rigid objects with one unified model. Its topology-aware diffusion transformer conditions generation on skeleton structure rather than requiring a fixed character template. The accompanying UniML3D dataset contains 13,006 text-paired motion sequences with unified skeletal canonicalization. Inference can generate motion for unseen skeleton topologies without per-skeleton retraining or test-time optimization. ... -
23
FlashAttention
Fast and memory-efficient exact attention
FlashAttention is a high-performance deep learning optimization library that reimplements the attention mechanism used in transformer models to be significantly faster and more memory-efficient than standard implementations. It achieves this by using IO-aware algorithms that minimize memory reads and writes, reducing the quadratic memory overhead typically associated with attention operations. The project provides implementations of FlashAttention, FlashAttention-2, and newer iterations optimized for modern GPU architectures such as NVIDIA Hopper and AMD accelerators. ... -
24
Janus
Unified Multimodal Understanding and Generation Models
...By splitting those pathways but keeping one unified core transformer, Janus maintains flexibility and achieves strong performance across tasks previously requiring distinct architectures. The repository includes pretrained checkpoints (for example 1.3B and 7B parameter versions), a Gradio demo, and guidance for local deployment. -
25
model2Vec
Fast State-of-the-Art Static Embeddings
...The resulting models can be used for a wide range of tasks, including semantic search, clustering, classification, and retrieval-augmented generation systems. One of its key advantages is its simplicity, as it requires minimal dependencies and can generate embeddings extremely quickly compared to traditional transformer-based approaches.
