Search Results for "llm"
Sort By:
Showing 46 open source projects for "llm"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
llmfit
157 models, 30 providers, one command to find what runs on hardware
...It provides both an interactive terminal user interface and a traditional CLI mode, enabling flexible workflows for different user preferences. llmfit also supports advanced configurations including multi-GPU setups, mixture-of-experts architectures, and dynamic quantization recommendations. By presenting clear performance estimates and compatibility guidance, the project reduces the trial-and-error typically involved in local LLM experimentation. Overall, llmfit serves as a practical decision assistant for developers who want to run language models efficiently on their own machines. -
2
LangChain Rust
LangChain for Rust, the easiest way to write LLM-based programs
...It also enables the construction of multi-step pipelines where LLM outputs feed into subsequent actions or tool calls. -
3
Arch Gateway
The AI-native (edge and LLM) proxy for agents
...It provides unified access and observability of Large Language Models (LLMs), enabling developers to build applications more efficiently. Arch supports both edge and LLM deployments, offering flexibility in various environments. -
4
LangDB AI Gateway
Govern, secure, and optimize your AI traffic
AI Gateway is a high-performance, open-source API gateway optimized for managing and monitoring LLM traffic at scale. Developed by the LangDB team, AI Gateway acts as an intermediary between clients and backend LLMs, providing advanced features like caching, rate limiting, prompt management, and observability. It helps teams secure and optimize their LLM deployments, whether using local models or external APIs like OpenAI or Anthropic. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
rtk
CLI proxy that reduces LLM token consumption
rtk is an open-source command-line proxy designed to optimize interactions between AI coding agents and the terminal by reducing unnecessary token consumption. When AI assistants execute shell commands during software development tasks, the resulting terminal output often contains large amounts of repetitive or irrelevant information that can overwhelm the model’s context window. RTK intercepts these command outputs and compresses them into concise summaries before sending them to the... -
6
Paddler
Open-source LLM load balancer and serving platform for hosting LLMs
Paddler is an open-source LLM infrastructure platform designed to deploy, manage, and scale large language models on private infrastructure. The system acts as a specialized load balancer and serving layer for language models, enabling organizations to run inference workloads without relying on external API providers. It supports running models locally through engines such as llama.cpp while distributing requests across multiple compute nodes to improve performance and reliability. ... -
7
mistral.rs
Fast, flexible LLM inference
mistral.rs is a fast and flexible LLM inference engine implemented in Rust, designed to run and serve modern language models with an emphasis on performance and practical deployment. It provides multiple entry points for developers, including a CLI for running models locally and an HTTP server that exposes an OpenAI-compatible API surface for easy integration with existing clients. -
8
Switchyard
Switchyard lets LLM applications route traffic across models
Switchyard is a Rust proxy and library for routing traffic between LLM clients and model backends. It translates among OpenAI Chat, OpenAI Responses, and Anthropic Messages formats so agents can keep using their native APIs. Requests can be distributed across vLLM, NVIDIA NIM, Ollama, OpenRouter, and other compatible endpoints. Routing strategies include random splits, LLM classification, signal-driven stage routing, escalation, and custom algorithms. -
9
Goose
AI coding agent that's more than suggestions - install, execute, edit+
Goose is an open-source, extensible AI agent that enhances the software development process by going beyond traditional code suggestions. It allows developers to install, execute, edit, and test code with any large language model (LLM), facilitating a more efficient and streamlined workflow. Designed to operate locally within a developer's environment, Goose integrates seamlessly with various tools and platforms, providing a customizable and powerful assistant for coding tasks. Its architecture supports extensibility, enabling users to tailor the agent to their specific needs and preferences. ... -
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
10
BAML
The AI framework that adds the engineering to prompt engineering
BAML is an open-source framework and domain-specific language designed to bring structured engineering practices to prompt development for large language model applications. Instead of treating prompts as unstructured text, BAML introduces a schema-driven approach where prompts are defined as typed functions with explicit inputs and outputs. This design allows developers to treat language model interactions as predictable software components rather than ad-hoc prompt strings. The framework... -
11
Plano
Delivery infrastructure for agentic apps
...Built on modern proxy technology and compatible with any language or AI framework, Plano enables developers to focus on core agent logic instead of infrastructure complexity. The system provides intelligent LLM routing APIs that support model agility, along with filter chains for safety, moderation, and memory hooks. It also exposes rich traces, metrics, and logs to support continuous improvement of agent behavior in production. Overall, Plano functions as delivery infrastructure for scalable, maintainable AI agent systems. -
12
VectorChord
Scalable, fast, and disk-friendly vector search in Postgres
VectorChord is an open-source vector database built for local and edge deployment. It supports efficient vector indexing and retrieval using ANN (approximate nearest neighbor) algorithms and is optimized for integration with LLM and AI applications. VectorChord is lightweight and can be embedded in a variety of environments for fast semantic search. -
13
MusicGPT
Generate music based on natural language prompts using LLMs
MusicGPT is an open-source application designed to generate music from natural language prompts using locally executed artificial intelligence models. The software allows users to run advanced music generation systems directly on their own devices without requiring heavy dependencies such as Python or full machine learning frameworks. Instead, it provides a lightweight environment capable of executing music generation models locally on CPUs or GPUs while maintaining strong performance across... -
14
Floneum
Instant, controllable, local pre-trained AI models in Rust
Floneum is an open-source platform for building AI-powered workflows using large language models through a visual and extensible interface. The system allows users to design complex AI pipelines using a drag-and-drop workflow builder rather than writing extensive code. It focuses on enabling developers and researchers to create language model applications that combine different tools, data sources, and AI capabilities into automated workflows. Floneum supports a plugin architecture that... -
15
MagicAPI AI Gateway
Built for demanding AI workflows
The world's fastest AI Gateway proxy, written in Rust and optimized for maximum performance. This high-performance API gateway routes requests to various AI providers (OpenAI, GROQ) with streaming support, making it perfect for developers who need reliable and blazing-fast AI API access. -
16
pgvecto.rs
Vector database plugin for Postgres, written in Rust
pgvecto.rs is a Postgres extension that provides vector similarity search functions. It is written in Rust and based on pgrx. It is currently under heavy development, please take care when using it in production. pgvecto.rs is a Postgres extension, which means that you can use it directly within your existing database. This makes it easy to integrate into your existing workflows and applications. pgvecto.rs supports filtering. You can set conditions when searching or retrieving points. This... -
17
webclaw
Fast, local-first web content extraction for LLMs
webclaw is a high-performance web content extraction tool designed specifically for AI agents and large language models, focusing on delivering clean, structured data instead of raw HTML. It is built in Rust and operates without a headless browser, using advanced techniques such as TLS fingerprinting to bypass common scraping barriers and mimic real browser behavior. The tool addresses a major inefficiency in AI workflows by removing irrelevant elements like navigation menus, ads, and... -
18
Cake
Distributed LLM and StableDiffusion inference
Cake is a compact, powerful toolkit that combines a flexible TCP/UDP proxy, port forwarding system, and connection manager designed for both development and penetration testing scenarios. It enables users to create complex networking flows where traffic can be proxied, relayed, and manipulated between endpoints — useful for debugging networked applications, inspecting protocols, or tunneling traffic through different hops. The tool is designed to work with multiple protocols and supports... -
19
HASH
The best way to use and work with blocks
This is HASH's public monorepo which contains our public code, docs, and other key resources. HASH is a platform for decision-making, which helps you integrate, understand and use data in a variety of different ways. HASH does this by combining various different powerful tools together into one simple interface. These range from data pipelines and a graph database, through to an all-in-one workspace, no-code tool builder, and agent-based simulation engine. These exist at varying stages of... -
20
Arch
Arch is an intelligent prompt gateway. Engineered with (fast) LLMs
Arch is an intelligent Layer 7 gateway designed to protect, observe, and personalize LLM applications (agents, assistants, co-pilots) with your APIs. Engineered with purpose-built LLMs, Arch handles the critical but undifferentiated tasks related to the handling and processing of prompts, including detecting and rejecting jailbreak attempts, intelligently calling "backend" APIs to fulfill the user's request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability of prompts and LLM interactions in a centralized way. -
21
Kheish
Kheish: A multi-role LLM agent for tasks like code auditing
Kheish is a framework designed for cybersecurity professionals to automate penetration testing tasks, providing tools to streamline security assessments. -
22
uzu
A high-performance inference engine for AI models
uzu is a high-performance inference engine designed to run artificial intelligence models efficiently on Apple Silicon hardware. Written primarily in Rust and leveraging Apple’s Metal framework, the project focuses on maximizing performance when executing large language models and other AI workloads on devices such as Mac computers with M-series chips. The engine implements a hybrid architecture in which model layers can be executed either as custom GPU kernels or through Apple’s MPSGraph... -
23
Korvus
Korvus is a search SDK that unifies the entire RAG pipeline
Korvus is an open-source retrieval-augmented generation (RAG) pipeline designed to run entirely inside PostgreSQL, allowing developers to build AI search and knowledge systems directly within a database environment. The project consolidates the typical steps of a RAG pipeline—including embedding generation, document retrieval, reranking, and text generation—into a single query executed within the Postgres ecosystem. By leveraging PostgresML and vector extensions such as pgvector, Korvus... -
24
Extractous
Fast and efficient unstructured data extraction
Extractous is a Rust-based unstructured data extraction library focused on fast local parsing of documents and other content-heavy files. Its purpose is to extract text and metadata efficiently from formats such as PDF, Word, HTML, email archives, images, and more, without depending on external APIs or separate parsing servers. The project emphasizes performance and low memory usage, and its maintainers describe it as a local-first alternative to heavier extraction stacks. For broader format... -
25
yek
Serialize repositories into LLM-ready context w/ smart prioritization
Yek is a Rust-based CLI tool designed to serialize text-based files from a repository or directory into a single structured output for large language model use. It scans projects using .gitignore rules to exclude irrelevant files and automatically filters out binary or oversized content. Yek prioritizes files based on Git history, placing more important content later in the output to align with how language models process context. Yek supports multiple directories, individual files, and glob...