Search Results for "llama-cpp-python.whl" - Page 2
Sort By:
Showing 235 open source projects for "llama-cpp-python.whl"
View related business solutions-
Cloud-based help desk software with ServoDeskWhat if You Could Automate 90% of Your Repetitive Tasks in Under 30 Days? At ServoDesk, we help businesses like yours automate operations with AI, allowing you to cut service times in half and increase productivity by 25% - without hiring more staff.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
restc-cpp C++ library
Modern C++ REST Client library
The magic that takes the pain out of accessing JSON API's from C++. It formulates a HTTP request to a REST API server. Then, it transforms the JSON formatted payload in the reply into a native C++ object (GET). It Serializes a native C++ object or a container of C++ objects into a JSON payload and sends it to the REST API server (POST, PUT). It formulates an HTTP request to the REST API without serializing any data in either direction (typically DELETE). It uploads a stream of data, like a... -
2
CodeLlama
Inference code for CodeLlama models
Code Llama is a family of Llama-based code models optimized for programming tasks such as code generation, completion, and repair, with variants specialized for base coding, Python, and instruction following. The repo documents the sizes and capabilities (e.g., 7B, 13B, 34B) and highlights features like infilling and large input context to support real IDE workflows. -
3
Unsloth
Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster
Unsloth is a framework designed to significantly improve the performance of Llama 3.3, DeepSeek-R1, and other reasoning large language models (LLMs). It optimizes these models to run up to 2x faster while using 70% less memory. Unsloth aims to make finetuning large models more efficient, offering users a simple, resource-efficient solution for customizing LLMs with their datasets. It provides a user-friendly experience through free notebooks and the ability to export finetuned models to various formats. -
4
Text Generation Web UI
A gradio web UI for running Large Language Models like LLaMA
...Very efficient text streaming. Parameter presets, 8-bit mode. Layers splitting across GPU(s), CPU, and disk. CPU mode, FlexGen, DeepSpeed ZeRO-3, API with streaming and without streaming. LLaMA model, including 4-bit GPTQ. RWKV model, LoRA (loading and training), Softprompts, and extensions. -
Keep company data safe with Chrome EnterpriseMake AI work your way with Chrome Enterprise. Block unapproved sites and set custom data controls that align with your company's policies.
-
5
LLamaSharp
C#/.NET binding of llama.cpp, including LLaMa/GPT model inference
The C#/.NET binding of llama.cpp. It provides APIs to infer the LLaMa Models and deploy it on the local environment. It works on both Windows, Linux and MAC without the requirement for compiling llama.cpp yourself. Its performance is close to llama.cpp. Furthermore, it provides integrations with other projects such as BotSharp to provide higher-level applications and UI. -
6
LLM Foundry
LLM training code for MosaicML foundation models
Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available for commercial use, and matches the quality of LLaMA-7B. MPT-7B was trained on the MosaicML platform in 9.5 days with zero human intervention at a cost of ~$200k. Large language models (LLMs) are changing the world, but for those outside well-resourced industry labs, it can be extremely difficult to train and deploy these models. This has led to a flurry of activity centered on open-source LLMs, such as the LLaMA series from Meta, the Pythia series from EleutherAI, the StableLM series from StabilityAI, and the OpenLLaMA model from Berkeley AI Research. -
7
bert4torch
An elegent pytorch implement of transformers
An elegant PyTorch implement of transformers. -
8
Jan.ai
Open source alternative to ChatGPT that runs 100% offline
...It allows you to download and run LLMs (local language models) offline while also offering optional integration with cloud-based model providers—giving you full control over your data and AI interactions. Download and run LLMs (Llama, Gemma, Qwen, GPT-oss etc.) from HuggingFace. Connect to GPT models via OpenAI, Claude models via Anthropic, Mistral, Groq, and others. Create specialized AI assistants for your tasks. MCP integration for agentic capabilities. -
9
DeepSeek R1
Open-source, high-performance AI model with advanced reasoning
...This approach has resulted in performance comparable to leading models like OpenAI's o1, while maintaining cost-efficiency. To further support the research community, DeepSeek has released distilled versions of the model based on architectures such as LLaMA and Qwen. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
fullmoon
Chat with private and local large language models
...Users can personalize the app by adjusting themes, fonts, and system prompts, and it integrates with Apple's Shortcuts for enhanced functionality. Fullmoon supports models like Llama-3.2-1B-Instruct-4bit and Llama-3.2-3B-Instruct-4bit, facilitating efficient on-device AI interactions without the need for an internet connection. -
11
Orpheus TTS
Towards Human-Sounding Speech
Orpheus TTS is a state-of-the-art open-source text-to-speech system built on a Llama-3B backbone, treating speech synthesis as a large language model problem instead of a traditional TTS pipeline. It is designed to produce human-like speech with natural intonation, emotion, and rhythm, targeting quality comparable to or better than many closed-source systems. The project ships both pretrained and finetuned English models, as well as a family of multilingual models released as a research preview, and includes data-processing scripts so users can train or finetune their own variants. ... -
12
GPT4All
Run Local LLMs on Any Device. Open-source
GPT4All is an open-source project that allows users to run large language models (LLMs) locally on their desktops or laptops, eliminating the need for API calls or GPUs. The software provides a simple, user-friendly application that can be downloaded and run on various platforms, including Windows, macOS, and Ubuntu, without requiring specialized hardware. It integrates with the llama.cpp implementation and supports multiple LLMs, allowing users to interact with AI models privately. This... -
13
tinygrad
Deep learning framework
This may not be the best deep learning framework, but it is a deep learning framework. Due to its extreme simplicity, it aims to be the easiest framework to add new accelerators to, with support for both inference and training. If XLA is CISC, tinygrad is RISC. -
14
-
15
llamafile
Distribute and run LLMs with a single file
...We're doing that by combining llama.cpp with Cosmopolitan Libc into one framework that collapses all the complexity of LLMs down to a single-file executable (called a "llamafile") that runs locally on most computers, with no installation. The easiest way to try it for yourself is to download our example llamafile for the LLaVA model (license: LLaMA 2, OpenAI). LLaVA is a new LLM that can do more than just chat; you can also upload images and ask it questions about them. With llamafile, this all happens locally; no data ever leaves your computer. -
16
LocalAI
Self-hosted, community-driven, local OpenAI compatible API
...Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU is required. Runs ggml, GPTQ, onnx, TF compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many others. LocalAI is a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer-grade hardware, supporting multiple model families that are compatible with the ggml format. ... -
17
SillyTavern
LLM Frontend for Power Users
Mobile-friendly, Multi-API (KoboldAI/CPP, Horde, NovelAI, Ooba, OpenAI, OpenRouter, Claude, Scale), VN-like Waifu Mode, Horde SD, System TTS, WorldInfo (lorebooks), customizable UI, auto-translate, and more prompt options than you'd ever want or need. Optional Extras server for more SD/TTS options + ChromaDB/Summarize. SillyTavern is a user interface you can install on your computer (and Android phones) that allows you to interact with text generation AIs and chat/roleplay with characters you or the community create. ... -
18
Axolotl
Go ahead and axolotl questions
Axolotl is a powerful and flexible framework for fine-tuning large language models on custom datasets. Built for researchers and developers, Axolotl simplifies the process of adapting LLMs for specific tasks, including chat, code generation, and instruction following. It supports a wide variety of model architectures and offers out-of-the-box optimization strategies for efficient training. -
19
far2l
Linux port of FAR v2
Linux fork of FAR Manager v2. Works also on OSX/MacOS and BSD (but the latter is not tested on a regular manner). Plug-ins that are currently working: NetRocks (SFTP/SCP/FTP/FTPS/SMB/NFS/WebDAV), colorer, multiarc, tmppanel, align, autowrap, drawing, edit case, SimpleIndent, Calculator, Python (optional scripting support). -
20
QuivrHQ
Opiniated RAG for integrating GenAI in your apps
Quivr is an open-source platform that leverages Retrieval-Augmented Generation (RAG) to integrate Generative AI into applications. It serves as a "second brain," enabling users to build powerful AI-driven assistants that can process and retrieve information efficiently. Quivr supports various large language models and vector stores, providing flexibility and customization for developers. -
21
Lepton AI
A Pythonic framework to simplify AI service building
A Pythonic framework to simplify AI service building. Cutting-edge AI inference and training, unmatched cloud-native experience, and top-tier GPU infrastructure. Ensure 99.9% uptime with comprehensive health checks and automatic repairs. -
22
OpenLLM
Operating LLMs in production
...With OpenLLM, you can run inference with any open-source large-language models, deploy to the cloud or on-premises, and build powerful AI apps. Built-in supports a wide range of open-source LLMs and model runtime, including Llama 2， StableLM, Falcon, Dolly, Flan-T5, ChatGLM, StarCoder, and more. Serve LLMs over RESTful API or gRPC with one command, query via WebUI, CLI, our Python/Javascript client, or any HTTP client. -
23
Bee Agent Framework
The framework for building scalable agentic applications
...The Bee Agent Framework makes it easy to build scalable agent-based workflows with your model of choice. The framework is been designed to perform robustly with IBM Granite and Llama 3.x models, and we’re actively working on optimizing its performance with other popular LLMs. Our goal is to empower developers to adopt the latest open-source and proprietary models with minimal changes to their current agent implementation. -
24
LazyLLM
Easiest and laziest way for building multi-agent LLMs applications
LazyLLM is an optimized, lightweight LLM server designed for easy and fast deployment of large language models. It is fully compatible with the OpenAI API specification, enabling developers to integrate their own models into applications that normally rely on OpenAI’s endpoints. LazyLLM emphasizes low resource usage and fast inference while supporting multiple models. -
25
llama2-webui
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere
Running Llama 2 with gradio web UI on GPU or CPU from anywhere (Linux/Windows/Mac).
