AutoGPTQ

AutoGPTQ is an implementation of GPTQ (Quantized GPT) that optimizes large language models (LLMs) for faster inference by reducing their computational footprint while maintaining accuracy.

Features

Efficient quantization for large language models
Reduces memory usage without major performance loss
Supports various precision levels (e.g., 4-bit, 8-bit)
Compatible with Hugging Face Transformers
Accelerates inference on GPUs and CPUs
Helps deploy LLMs on resource-constrained hardware

Project Samples

Project Activity

See All Activity >

License

MIT License

Follow AutoGPTQ

AutoGPTQ Web Site

Other Useful Business Software

Build Agents and Models on One Platform

Everything you need to build production-ready agents and models. Access 200+ Google and third-party AI models and tools.

Gemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.

Try It Free

Rate This Project

User Reviews

Be the first to post a review of AutoGPTQ!

Additional Project Details

Operating Systems

Linux, Mac, Windows

Programming Language

Python

Related Categories

Python Natural Language Processing (NLP) Tool, Python LLM Inference Tool

Registered

2025-01-21

Similar Business Software

LM-Kit.NET

LM-Kit.NET is a complete local AI runtime for .NET that lets engineering teams ship AI-powered features without cloud dependencies, per-token costs, or data leaving the network. Most .NET AI integrations stop at inference. LM-Kit.NET covers the full range of capabilities production...

See Software
Gemini Enterprise Agent Platform

Gemini Enterprise Agent Platform is a comprehensive solution from Google Cloud designed to help organizations build, scale, govern, and optimize AI agents. It represents the evolution of Vertex AI, combining advanced model development with new capabilities for agent orchestration and...

See Software
Google AI Studio

Google AI Studio is a unified development platform that helps teams explore, build, and deploy applications using Google’s most advanced AI models, including Gemini 3.5. It brings text, image, audio, and video models together in one interactive playground. With vibe coding, developers can use...

See Software
Runpod

Runpod offers a cloud-based platform designed for running AI workloads, focusing on providing scalable, on-demand GPU resources to accelerate machine learning (ML) model training and inference. With its diverse selection of powerful GPUs like the NVIDIA A100, RTX 3090, and H100, Runpod supports...

See Software
kama.ai

A Responsible AI Agent platform providing accurate, accountable, and safe AI for your organization. As a Composite (hybrid) platform, it combines Knowledge Graph AI, governed Generative AI, and Intelligent Automation technologies. This combination gives you trusted answers that are accurate...

See Software
OpenVINO

The Intel® Distribution of OpenVINO™ toolkit is an open-source AI development toolkit that accelerates inference across Intel hardware platforms. Designed to streamline AI workflows, it allows developers to deploy optimized deep learning models for computer vision, generative AI, and large...

See Software