Search Results for "transformer"
Sort By:
Showing 10 open source projects for "transformer"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
1
CTranslate2
Fast inference engine for Transformer models
CTranslate2 is a C++ and Python library for efficient inference with Transformer models. The project implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering, etc., to accelerate and reduce the memory usage of Transformer models on CPU and GPU. The execution is significantly faster and requires less resources than general-purpose deep learning frameworks on supported models and tasks thanks to many advanced optimizations: layer fusion, padding removal, batch reordering, in-place operations, caching mechanism, etc. ... -
2
ONNX Runtime
ONNX Runtime: cross-platform, high performance ML inferencing
...ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms. ONNX Runtime training can accelerate the model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition for existing PyTorch training scripts. Support for a variety of frameworks, operating systems and hardware platforms. Built-in optimizations that deliver up to 17X faster inferencing and up to 1.4X faster training. -
3
ggml
Tensor library for machine learning
...The project emphasizes portability and performance, enabling machine learning inference across a wide range of hardware environments including CPUs and specialized accelerators. It is widely used as a foundational component in projects that run large language models locally, including tools that perform inference for transformer-based models. The library also implements optimization algorithms and computation graph functionality so developers can build training and inference workflows directly on top of its tensor operations. -
4
rwkv.cpp
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
...This project is focused on CPU, but cuBLAS is also supported. RWKV is a novel large language model architecture, with the largest model in the family having 14B parameters. In contrast to Transformer with O(n^2) attention, RWKV requires only state from the previous step to calculate logits. This makes RWKV very CPU-friendly on large context lengths. -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
5
RTP-LLM
Alibaba's high-performance LLM inference engine for diverse apps
...The system focuses on improving throughput, latency, and resource utilization when running large models in production environments. It achieves this by implementing optimized GPU kernels, batching strategies, and memory management techniques tailored for transformer inference workloads. The framework is designed for large-scale AI services and is already used internally across several Alibaba platforms such as Taobao, Amap, and other business systems that rely on conversational or search-related AI services. RTP-LLM supports a wide variety of modern model architectures, including Qwen, DeepSeek, and Llama-based models, making it a flexible engine for deploying many different open-source LLMs. -
6
PowerInfer
High-speed Large Language Model Serving for Local Deployment
PowerInfer is a high-performance inference engine designed to run large language models efficiently on personal computers equipped with consumer-grade GPUs. The project focuses on improving the performance of local AI inference by optimizing how neural network computations are distributed between CPU and GPU resources. Its architecture exploits the observation that only a subset of neurons in large models are frequently activated, allowing the system to preload frequently used neurons into... -
7
OnnxStream
Lightweight inference library for ONNX files, written in C++
The challenge is to run Stable Diffusion 1.5, which includes a large transformer model with almost 1 billion parameters, on a Raspberry Pi Zero 2, which is a microcomputer with 512MB of RAM, without adding more swap space and without offloading intermediate results on disk. The recommended minimum RAM/VRAM for Stable Diffusion 1.5 is typically 8GB. Generally, major machine learning frameworks and libraries are focused on minimizing inference latency and/or maximizing throughput, all of which at the cost of RAM usage. ... -
8
FasterTransformer
Transformer related optimization, including BERT, GPT
FasterTransformer is a high-performance inference library designed to accelerate transformer-based models such as BERT, GPT, and T5 on NVIDIA GPUs. It provides optimized implementations of transformer encoder and decoder layers using CUDA, cuBLAS, and custom kernels to maximize throughput and minimize latency. The library supports multiple deep learning frameworks, including TensorFlow, PyTorch, and Triton, allowing developers to integrate it into existing pipelines without major changes. ... -
9
LightSeq
A High Performance Library for Sequence Processing and Generation
...Because of this, it’s particularly useful for researchers and developers who want to fine-tune or run transformer-based models without requiring top-tier GPUs or massive computational resources. -
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
10
TurboTransformers
Fast and user-friendly runtime for transformer inference
TurboTransformers is a high-performance inference framework optimized for running Transformer models efficiently on CPUs and GPUs. It improves latency and throughput for NLP applications.
- Previous
- You're on page 1
- Next
