DownloadSelf-hosted, community-driven, local OpenAI compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU is required. Runs ggml, GPTQ, onnx, TF compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many others. LocalAI is a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer-grade hardware, supporting multiple model families that are compatible with the ggml format. Does not require GPU.
Features
- Local, OpenAI drop-in alternative REST API
- NO GPU required
- Supports multiple models
- Once loaded the first time, it keep models loaded in memory for faster inference
- Doesn’t shell-out, but uses C++ bindings for a faster inference and better performance
- You own your data
Project Samples
Categories
Large Language Models (LLM)License
MIT LicenseFollow LocalAI
You Might Also Like
HubSpot is an AI-powered customer platform with all the software, integrations, and resources you need to connect your marketing, sales, and customer service. HubSpot's connected platform enables you to grow your business faster by focusing on what matters most: your customers.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of LocalAI!