Download
AutoGPTQ is an implementation of GPTQ (Quantized GPT) that optimizes large language models (LLMs) for faster inference by reducing their computational footprint while maintaining accuracy.
Features
- Efficient quantization for large language models
- Reduces memory usage without major performance loss
- Supports various precision levels (e.g., 4-bit, 8-bit)
- Compatible with Hugging Face Transformers
- Accelerates inference on GPUs and CPUs
- Helps deploy LLMs on resource-constrained hardware
Project Samples
License
MIT LicenseFollow AutoGPTQ
Other Useful Business Software
Earn up to 16% annual interest with Nexo.
Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform.
Geographic restrictions, eligibility, and terms apply.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of AutoGPTQ!
