Search Results for "deploy"
Sort By:
Showing 45 open source projects for "deploy"
View related business solutions-
Stop Storing Third-Party Tokens in Your DatabaseRolling your own OAuth token storage can be a security liability. Token Vault securely stores access and refresh tokens from federated providers and handles exchange and renewal automatically. Connected accounts, refresh exchange, and privileged worker flows included.
-
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
1
OpenLLM
Operating LLMs in production
An open platform for operating large language models (LLMs) in production. Fine-tune, serve, deploy, and monitor any LLMs with ease. With OpenLLM, you can run inference with any open-source large-language models, deploy to the cloud or on-premises, and build powerful AI apps. Built-in supports a wide range of open-source LLMs and model runtime, including Llama 2， StableLM, Falcon, Dolly, Flan-T5, ChatGLM, StarCoder, and more. Serve LLMs over RESTful API or gRPC with one command, query via WebUI, CLI, our Python/Javascript client, or any HTTP client. -
2
LlamaDeploy
Deploy your agentic worfklows to production
...Developers can define workflows that involve multiple steps such as data retrieval, reasoning, tool invocation, and response generation, then deploy them using the framework’s infrastructure tools. The design emphasizes scalability, modularity, and fault-tolerant execution so that agent systems can run reliably in production environments. -
3
Xorbits Inference
Replace OpenAI GPT with another LLM in your app
...Xorbits Inference(Xinference) is a powerful and versatile library designed to serve language, speech recognition, and multimodal models. With Xorbits Inference, you can effortlessly deploy and serve your or state-of-the-art built-in models using just a single command. Whether you are a researcher, developer, or data scientist, Xorbits Inference empowers you to unleash the full potential of cutting-edge AI models. -
4
Beelzebub
A secure low code honeypot framework
...Honeypots are systems intentionally exposed to attackers in order to capture malicious behavior, and Beelzebub enhances this concept by incorporating artificial intelligence and virtualization techniques. The platform allows organizations and researchers to deploy decoy services that mimic real infrastructure while recording attacker interactions. By using AI models to simulate realistic system behavior, the honeypot becomes harder for attackers to identify, increasing the likelihood that malicious activity can be observed and analyzed. The framework is designed with a low-code configuration approach so security teams can easily deploy honeypots for multiple services and ports. -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
LangServe
Helps developers deploy LangChain runnables and chains as a REST API
...The framework is built on top of FastAPI and uses Pydantic for request validation and structured data handling. It also includes client libraries that allow developers to interact with deployed chains from Python or JavaScript applications. LangServe is commonly used to deploy AI applications such as chatbots, document analysis pipelines, and agent-based systems that require scalable access through APIs. -
6
Deta Surf
Personal AI Notebooks. Organize files & webpages and generate notes
Surf is an open-source AI-driven development tool designed to simplify the process of building and experimenting with artificial intelligence applications. The platform provides a streamlined development environment where developers can test models, run experiments, and deploy small AI services with minimal infrastructure overhead. It focuses on simplicity and speed, allowing developers to prototype ideas quickly without managing complex cloud configurations. Surf integrates modern AI workflows such as prompt-based applications, lightweight APIs, and automated deployment pipelines. The platform is particularly useful for developers who want to experiment with AI models locally while maintaining the option to deploy them in production environments later. ... -
7
Infinity
Low-latency REST API for serving text-embeddings
Infinity is a high-throughput, low-latency REST API for serving vector embeddings, supporting all sentence-transformer models and frameworks. Infinity is developed under MIT License. Infinity powers inference behind Gradient.ai and other Embedding API providers. -
8
LLamaSharp
C#/.NET binding of llama.cpp, including LLaMa/GPT model inference
The C#/.NET binding of llama.cpp. It provides APIs to infer the LLaMa Models and deploy it on the local environment. It works on both Windows, Linux and MAC without the requirement for compiling llama.cpp yourself. Its performance is close to llama.cpp. Furthermore, it provides integrations with other projects such as BotSharp to provide higher-level applications and UI. -
9
Guidance
A guidance language for controlling large language models
Guidance is an efficient programming paradigm for steering language models. With Guidance, you can control how output is structured and get high-quality output for your use case—while reducing latency and cost vs. conventional prompting or fine-tuning. It allows users to constrain generation (e.g. with regex and CFGs) as well as to interleave control (conditionals, loops, tool use) and generation seamlessly. -
Compliant and Reliable File Transfers Backed by Top Security CertificationsStop relying on non-certified, legacy file transfer tools that creak under the weight of modern security demands. Get full audit trails, advanced access controls and more supported by an award-winning team of experts. Start your free 25-day trial today.
-
10
Cybersecurity AI
Cybersecurity AI (CAI), the framework for AI Security
CAI (Cybersecurity AI) is a lightweight open-source framework intended to help security practitioners build and deploy AI-assisted automation for defensive and offensive security workflows. The project frames itself as a practical foundation for “AI security,” focusing on turning security tasks into agentic workflows that can be composed, executed, and iterated on by practitioners. Rather than being a single-purpose tool, CAI is positioned as a framework that supports building multiple security automations and integrating them into existing processes. ... -
11
ChatGLM2-6B
ChatGLM2-6B: An Open Bilingual Chat LLM
ChatGLM2-6B is the second-gen Chinese-English conversational LLM from ZhipuAI/Tsinghua. It upgrades the base model with GLM’s hybrid pretraining objective, 1.4 TB bilingual data, and preference alignment—delivering big gains on MMLU, CEval, GSM8K, and BBH. The context window extends up to 32K (FlashAttention), and Multi-Query Attention improves speed and memory use. The repo includes Python APIs, CLI & web demos, OpenAI-style/FASTAPI servers, and quantized checkpoints for lightweight local... -
12
Chitu
High-performance inference framework for large language models
Chitu is a high-performance inference engine designed to deploy and run large language models efficiently in production environments. The framework focuses on improving efficiency, flexibility, and scalability for organizations that need to run LLM inference workloads across different hardware platforms. It supports heterogeneous computing environments, including CPUs, GPUs, and various specialized AI accelerators, allowing models to run across a wide range of infrastructure configurations. ... -
13
GPUStack
Performance-optimized AI inference on your GPUs
...The platform supports GPUs from a wide range of vendors and can run on laptops, workstations, and servers across operating systems such as macOS, Windows, and Linux. It also enables developers to deploy models from common repositories like Hugging Face and access them through APIs similar to cloud-based AI services. -
14
LangBot
Production-grade platform for building agentic IM bots
LangBot is an open source platform designed to build and deploy AI-powered chatbots across multiple instant messaging ecosystems. The system allows developers to integrate large language models into messaging platforms so that bots can perform tasks, answer questions, and automate workflows directly within everyday communication tools. It supports numerous messaging services including Discord, Slack, Telegram, WeChat, and other enterprise communication systems, making it a flexible solution for both personal projects and organizational deployments. ... -
15
LLM-Pruner
On the Structural Pruning of Large Language Models
LLM-Pruner is an open-source framework designed to compress large language models through structured pruning techniques while maintaining their general capabilities. Large language models often require enormous computational resources, making them expensive to deploy and inefficient for many practical applications. LLM-Pruner addresses this issue by identifying and removing non-essential components within transformer architectures, such as redundant attention heads or feed-forward structures. The framework relies on gradient-based analysis to determine which parameters contribute least to model performance, enabling targeted structural pruning rather than simple weight removal. ... -
16
handy-ollama
Implement CPU from scratch and play with large model deployments
handy-ollama is an open-source educational project designed to help developers and AI enthusiasts learn how to deploy and run large language models locally using the Ollama platform. The repository serves as a structured tutorial that explains how to install, configure, and use Ollama to run modern language models on personal hardware without requiring advanced infrastructure. A key focus of the project is enabling users to run large models even without GPUs by leveraging optimized CPU-based inference pipelines. ... -
17
browserable
Open source and self-hostable browser automation library for AI agents
...Built primarily in JavaScript, the framework offers both a developer-friendly SDK and a REST API that allow integration with AI applications and automation pipelines. It is designed to be self-hostable, which means developers can deploy and run it on their own infrastructure without relying on third-party services. The platform enables the creation of browser-based agents capable of performing complex online workflows such as data collection, research tasks, and automated interactions with web platforms. -
18
AI Engineering Transition Path
Research papers and blogs to transition to AI Engineering
...The project organizes resources that cover fundamental topics required to understand modern AI systems, including transformers, vector embeddings, tokenization, infrastructure design, and mixture-of-experts architectures. Instead of presenting isolated tutorials, the repository provides a structured pathway that guides engineers through the technical knowledge needed to build and deploy large language model systems. The materials include curated research papers, blog posts, and code examples that explain both theoretical foundations and practical implementation strategies. By consolidating these resources into a single repository, the project helps developers navigate the rapidly expanding AI ecosystem without needing to search through scattered materials. -
19
LLMChat
Unified interface for AI chat, Agentic workflows and more
LLMChat is an open-source AI chat platform designed to provide a unified interface for interacting with multiple large language model providers while emphasizing privacy and advanced research capabilities. The system is built as a modern monorepo using technologies such as Next.js and TypeScript, enabling developers to deploy a full-featured web-based chatbot environment. One of its primary goals is to support sophisticated research workflows that combine conversational AI with information retrieval and reasoning tools. The platform includes specialized interaction modes such as deep research analysis and enhanced search capabilities that help users explore complex topics more effectively. ... -
20
TaxHacker
Self-hosted AI accounting app. LLM analyzer for receipts
...It integrates large language models to analyze these documents, extract relevant financial information, and categorize expenses or income based on configurable rules. Users can deploy the application on their own infrastructure, ensuring that financial data remains private and under their control rather than being processed by external services. The software provides tools for tracking income streams, monitoring expenses, and organizing financial records in a structured format. Because the system supports customizable prompts and categories, users can adapt the AI analysis to match their accounting workflows or tax requirements. -
21
SmythOS
Cloud-native runtime for agentic AI
...It provides a foundational infrastructure layer that functions similarly to an operating system for agentic AI systems, managing resources such as language models, storage, vector databases, and caching through a unified interface. Developers can use the runtime to create, deploy, and orchestrate intelligent agents across local machines, cloud environments, or hybrid infrastructures without rewriting their application logic. The platform includes a software development kit and command-line interface that allow developers to define agent workflows, manage execution environments, and automate deployment processes. ... -
22
Generative AI Use Cases (GenU)
Application implementation with business use cases
...These examples cover tasks such as document analysis, conversational assistants, content generation, and knowledge retrieval systems. The repository is intended to serve as both a learning resource and a starting point for developers who want to deploy generative AI solutions using AWS infrastructure. -
23
Agent Development Kit (ADK) for Java
An open-source, code-first Java toolkit
Google’s Agent Development Kit for Java is an open-source toolkit that helps developers design, evaluate, and deploy advanced AI agents using the Java programming language. The framework follows a code-first approach that treats agent development as a structured software engineering task rather than a collection of prompt scripts. It provides abstractions and tools that allow developers to create agents capable of executing complex workflows, calling tools, and interacting with external services. ... -
24
Paddler
Open-source LLM load balancer and serving platform for hosting LLMs
Paddler is an open-source LLM infrastructure platform designed to deploy, manage, and scale large language models on private infrastructure. The system acts as a specialized load balancer and serving layer for language models, enabling organizations to run inference workloads without relying on external API providers. It supports running models locally through engines such as llama.cpp while distributing requests across multiple compute nodes to improve performance and reliability. ... -
25
MaxText
A simple, performant and scalable Jax LLM
...The framework focuses on simplicity while still supporting advanced techniques such as model sharding, distributed computation, and high-throughput training pipelines. MaxText includes ready-to-use configurations and reproducible training examples that help developers understand how to deploy large-scale AI workloads with modern machine learning infrastructure.
