Search Results for "transformers"
Sort By:
Showing 214 open source projects for "transformers"
View related business solutions-
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
One Monitoring Tool for IT, OT and Cloud | Free TrialModern infrastructure spans data centers, cloud platforms and factory floors, and every blind spot between them is a risk. PRTG supports SNMP, WMI, SSH and other standard protocols to monitor IT, OT and hybrid environments through one customizable dashboard. Build the views your team needs, from network health to application performance, without switching tools. Try PRTG free for 30 days now.
-
1
Transformers
State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX
Hugging Face Transformers provides APIs and tools to easily download and train state-of-the-art pre-trained models. Using pre-trained models can reduce your compute costs, carbon footprint, and save you the time and resources required to train a model from scratch. These models support common tasks in different modalities. Text, for tasks like text classification, information extraction, question answering, summarization, translation, text generation, in over 100 languages. -
2
spacy-transformers
Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy
...These techniques can be used to import knowledge from raw text into your pipeline, so that your models are able to generalize better from your annotated examples. You can convert word vectors from popular tools like FastText and Gensim, or you can load in any pre trained transformer model if you install spacy-transformers. You can also do your own language model pretraining via the spacy pre train command. You can even share your transformer or another contextual embedding model across multiple components, which can make long pipelines several times more efficient. To use transfer learning, you’ll need at least a few annotated examples for what you’re trying to predict. -
3
x-transformers
A simple but complete full-attention transformer
...Proposes adding learned memory key/values prior to attending. They were able to remove feedforwards altogether and attain a similar performance to the original transformers. I have found that keeping the feedforwards and adding the memory key/values leads to even better performance. Proposes adding learned tokens, akin to CLS tokens, named memory tokens, that is passed through the attention layers alongside the input tokens. You can also use the l2 normalized embeddings proposed as part of fixnorm. ... -
4
Transformers & LLMs cheatsheet
VIP cheatsheet for Stanford's CME 295 Transformers and Large Language
Transformers & LLMs cheatsheet is an educational repository focused on transformers, large language models, and modern generative AI architectures based on Stanford CME 295 coursework. The project compiles structured notes, mathematical explanations, diagrams, and implementation references covering transformer internals, attention mechanisms, tokenization, training pipelines, and inference strategies. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
SetFit
Efficient few-shot learning with Sentence Transformers
SetFit is an efficient and prompt-free framework for few-shot fine-tuning of Sentence Transformers. It achieves high accuracy with little labeled data - for instance, with only 8 labeled examples per class on the Customer Reviews sentiment dataset, SetFit is competitive with fine-tuning RoBERTa Large on the full training set of 3k examples. -
6
GeoAI
GeoAI: Artificial Intelligence for Geospatial Data
GeoAI is a comprehensive open-source Python package designed to integrate artificial intelligence techniques with geospatial data analysis, enabling users to perform advanced geographic modeling and visualization tasks with ease. It provides a unified framework that combines machine learning libraries such as PyTorch and Transformers with geospatial tools, allowing users to process satellite imagery, aerial photos, and vector datasets in a streamlined workflow. The platform supports a wide range of tasks including image classification, object detection, segmentation, and change detection, making it suitable for applications in environmental monitoring, urban planning, and disaster response. ... -
7
SageMaker Hugging Face Inference Toolkit
Library for serving Transformers models on Amazon SageMaker
SageMaker Hugging Face Inference Toolkit is an open-source library for serving Transformers models on Amazon SageMaker. This library provides default pre-processing, predict and postprocessing for certain Transformers models and tasks. It utilizes the SageMaker Inference Toolkit for starting up the model server, which is responsible for handling inference requests. For the Dockerfiles used for building SageMaker Hugging Face Containers, see AWS Deep Learning Containers. ... -
8
Adapters
A Unified Library for Parameter-Efficient Learning
Adapters is an add-on library to HuggingFace's Transformers, integrating 10+ adapter methods into 20+ state-of-the-art Transformer models with minimal coding overhead for training and inference. Adapters provide a unified interface for efficient fine-tuning and modular transfer learning, supporting a myriad of features like full-precision or quantized training (e.g. Q-LoRA, Q-Bottleneck Adapters, or Q-PrefixTuning), adapter merging via task arithmetics or the composition of multiple adapters via composition blocks, allowing advanced research in parameter-efficient transfer learning for NLP tasks. -
9
Feature-engine
Feature engineering package with sklearn like functionality
Feature-engine is a Python library with multiple transformers to engineer and select features for use in machine learning models. Feature-engine's transformers follow Scikit-learn's functionality with fit() and transform() methods to learn the transforming parameters from the data and then transform it. -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
bert4torch
An elegent pytorch implement of transformers
An elegant PyTorch implement of transformers. -
11
BERTopic
Leveraging BERT and c-TF-IDF to create easily interpretable topics
...Instead, we can visualize the topics that were generated in a way very similar to LDAvis. By default, the main steps for topic modeling with BERTopic are sentence-transformers, UMAP, HDBSCAN, and c-TF-IDF run in sequence. -
12
LocalAI
The free, Open Source alternative to OpenAI, Claude and others
...LocalAI can run on consumer-grade hardware and does not necessarily require a GPU, making it accessible for local development and private deployments. It integrates with multiple backends like llama.cpp, transformers, and diffusers to support different AI workloads. With its self-hosted architecture and OpenAI-compatible API, LocalAI enables developers to build secure, local-first AI applications. -
13
DINOv3
Reference PyTorch implementation and models for DINOv3
...DINOv3 removes the need for complex augmentations or momentum encoders, streamlining the pipeline while maintaining or improving feature quality. The model supports multiple backbone architectures, including Vision Transformers (ViT), and can handle larger image resolutions with improved stability during training. The learned embeddings generalize robustly across tasks like classification, retrieval, and segmentation without fine-tuning, showing state-of-the-art transfer performance among self-supervised models. -
14
Unlimited OCR Works
Welcome the Era of One-shot Long-horizon Parsing
...The project supports single-image parsing as well as multi-page and PDF-style parsing by converting pages into images. It provides inference paths for Hugging Face Transformers, vLLM, and SGLang, which gives users several deployment options. The repository also includes example code for batch inference over image folders or PDF inputs. Overall, it is useful for researchers and developers who need advanced OCR, long-document parsing, and model-based extraction from complex visual documents. -
15
Torch Pruning
DepGraph: Towards Any Structural Pruning
...It introduces a graph-based algorithm called DepGraph that automatically identifies dependencies between layers, allowing parameters to be pruned safely across complex architectures. This dependency analysis makes it possible to prune large networks such as transformers, convolutional networks, and diffusion models without breaking the computational graph. Torch-Pruning physically removes parameters rather than masking them, which results in smaller and faster models during both training and inference. The toolkit supports a wide variety of architectures used in computer vision and large language models, making it a flexible solution for model compression tasks. -
16
Attention Residuals (AttnRes)
Drop-in replacement for standard residual connections in Transformers
Attention Residuals is a research-driven architectural innovation for transformer-based models that replaces traditional residual connections with an attention-based mechanism to improve information flow across layers. In standard transformers, residual connections simply sum outputs from previous layers, which can lead to uncontrolled growth of hidden states and dilution of early-layer information in deep networks. Attention Residuals introduces a learnable softmax attention mechanism that allows each layer to selectively retrieve and weight useful representations from earlier layers, making depth dynamically adaptive rather than uniformly aggregated. ... -
17
llm-abliteration
Make abliterated models with transformers, easy and fast
LLM Abliteration is an open-source Python toolkit for modifying refusal behavior in large language models without conventional fine-tuning. Built with Hugging Face Transformers, it identifies refusal-related activation directions by comparing responses to harmful and harmless prompts. Researchers can analyze these directions across model layers and selectively remove their influence from model weights. The software supports dense and selected mixture-of-experts architectures, including Llama, Qwen, Mistral, and Gemma models. ... -
18
AiLearning-Theory-Applying
Quickly get started with AI theory and practical applications
...The project also introduces important concepts such as probability theory, linear algebra, regression models, clustering methods, and neural network architectures. Advanced sections explore modern AI topics including transformers, BERT-based natural language processing systems, and practical competition-style machine learning workflows. -
19
The SpeechBrain Toolkit
A PyTorch-based Speech Toolkit
...Competitive or state-of-the-art performance is obtained in various domains. SpeechBrain supports state-of-the-art methods for end-to-end speech recognition, including models based on CTC, CTC+attention, transducers, transformers, and neural language models relying on recurrent neural networks and transformers. Speaker recognition is already deployed in a wide variety of realistic applications. SpeechBrain provides different models for speaker recognition, including X-vector, ECAPA-TDNN, PLDA, and contrastive learning. Spectral masking, spectral mapping, and time-domain enhancement are different methods already available within SpeechBrain. ... -
20
Detoxify
Trained models & code to predict toxic comments
Detoxify is a deep learning-based tool for detecting and filtering toxic language in online conversations, leveraging Transformer models for high accuracy. -
21
LLaMA-Factory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
LLaMA-Factory is a fine-tuning and training framework for Meta's LLaMA language models. It enables researchers and developers to train and customize LLaMA models efficiently using advanced optimization techniques. -
22
Kimi Linear
An expressive, efficient attention architecture
...Released Base and Instruct checkpoints contain 48 billion total parameters, activate 3 billion parameters per token, and support contexts up to one million tokens. The models were trained on 5.7 trillion tokens and can reduce KV cache requirements by up to 75 percent. Kimi Linear supports inference through Hugging Face Transformers and deployment as an OpenAI-compatible API with vLLM. -
23
AI Engineering Transition Path
Research papers and blogs to transition to AI Engineering
AI Engineering Resources is an open educational repository that compiles research papers, tutorials, and learning materials for software engineers transitioning into artificial intelligence engineering roles. The project organizes resources that cover fundamental topics required to understand modern AI systems, including transformers, vector embeddings, tokenization, infrastructure design, and mixture-of-experts architectures. Instead of presenting isolated tutorials, the repository provides a structured pathway that guides engineers through the technical knowledge needed to build and deploy large language model systems. The materials include curated research papers, blog posts, and code examples that explain both theoretical foundations and practical implementation strategies. ... -
24
SwanLab
An open-source, modern-design AI training tracking and visualization
...SwanLab supports both cloud and self-hosted deployments, allowing organizations to run the system privately or integrate it into shared development environments. The platform integrates with a wide range of machine learning frameworks including PyTorch, Transformers, Keras, and other widely used training ecosystems. -
25
DFlash
Block Diffusion for Ultra-Fast Speculative Decoding
...The project includes support for multiple draft models, example integration code, and scripts to benchmark performance, and it is structured to work with popular model serving stacks like SGLang and the Hugging Face Transformers ecosystem.
