Search Results for "requests"
Sort By:
Showing 30 open source projects for "requests"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
OmniRoute
OmniRoute is an AI gateway for multi-provider LLM
...It is particularly useful in distributed systems where requests need to be intelligently routed between APIs, microservices, or processing pipelines. OmniRoute aims to reduce boilerplate by centralizing routing logic and providing reusable patterns for managing complex flows. Its architecture supports scalability and maintainability, making it suitable for both small applications and larger systems with multiple integrations. -
2
AxonHub
Use any SDK to call 100+ LLMs
AxonHub is an open-source AI gateway platform designed to simplify the process of integrating and switching between different large language model providers. The system acts as a compatibility layer that allows developers to use the same SDK interface while routing requests to various AI services behind the scenes. Instead of rewriting code when switching providers such as OpenAI or Anthropic, developers can simply change configuration settings within the gateway. AxonHub translates requests from one provider’s API format into another, enabling seamless interoperability across different AI platforms. ... -
3
AI-Codereview-Gitlab
GitLab automatic code review tool based on large models
...When code changes occur, the system can automatically generate review comments and feedback that are posted directly into GitLab merge requests, allowing developers to see suggestions alongside human reviewer comments. In addition to code analysis, the tool can produce daily development summaries and notifications that help teams track progress and review activity across projects. -
4
Infinity
Low-latency REST API for serving text-embeddings
Infinity is a high-throughput, low-latency REST API for serving vector embeddings, supporting all sentence-transformer models and frameworks. Infinity is developed under MIT License. Infinity powers inference behind Gradient.ai and other Embedding API providers. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
vLLM
A high-throughput and memory-efficient inference and serving engine
vLLM is a fast and easy-to-use library for LLM inference and serving. High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more. -
6
OpenOutreach
Linkedin Automation Tool
...According to the repository, it combines large language model classification with a Bayesian machine learning layer based on profile embeddings, which helps it shift from broad exploration to more confident qualification as it gathers more decisions. It is designed to automate personalized outreach as well, including connection requests and follow-up messaging, while keeping deployment under the user’s control through a local or self-hosted setup. -
7
FreeLLMAPI
OpenAI-compatible proxy that aggregates free-tier keys from ~14 AI
FreeLLMAPI is an OpenAI-compatible proxy that aggregates free-tier API keys from multiple AI providers into one unified endpoint. It is designed for personal experimentation, testing, and lightweight development workflows where users want to route requests through several providers without rewriting client code for each one. The project can automatically fail over between configured providers when one is unavailable or exhausted. Its OpenAI-compatible design makes it easier to use with existing tools, SDKs, and applications that already expect that API shape. It is not positioned as an enterprise-grade service or a way to bypass provider terms, but as a local coordination layer for personally owned free-tier credentials. freellmapi is useful for developers who want a practical testing proxy for comparing models, managing limits, and improving request continuity. -
8
Envoy AI Gateway
Manages Unified Access to Generative AI Services
Envoy AI Gateway is an open-source gateway system designed to manage network traffic between applications and generative AI services using the Envoy proxy ecosystem. The project extends Envoy Gateway to support AI-specific workloads, enabling organizations to route, secure, and scale requests to large language models and other generative AI services. In a typical deployment, the architecture uses a two-tier gateway model where an outer gateway handles authentication, routing, and global rate limiting while a second gateway manages traffic to self-hosted model serving clusters. This design allows organizations to centralize control over AI service access while maintaining flexibility in how backend model infrastructure is deployed. ... -
9
PasteGuard
Masks sensitive data and secrets before they reach AI
...It sits between an application and the LLM provider, automatically replacing names, emails, tokens, and other personally identifiable information (PII) with placeholders so that external services never see raw sensitive values, and then optionally unmasking them in the returned output. PasteGuard supports two primary modes: mask mode, which anonymizes data and still uses external APIs; and route mode, which forwards sensitive requests to a local LLM inference engine while sending the rest to the cloud. It can be self-hosted via Docker, works with a wide range of SDKs and tools, and includes a browser extension for automatic protection in everyday AI chats. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
Parallax
Parallax is a distributed model serving framework
...Parallax divides model layers across different nodes and dynamically coordinates them to form a complete inference pipeline. A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. This scheduling system optimizes latency, throughput, and hardware utilization even when nodes have different computational capabilities. The platform also supports model sharding and pipeline parallelism, allowing very large models to run across distributed resources. -
11
UFO³
Weaving the Digital Agent Galaxy
...This enables the agent to navigate complex software environments and perform tasks that normally require manual interaction. UFO integrates mechanisms for task decomposition, planning, and execution so that high-level user requests can be broken down into smaller steps performed by specialized agents. The framework can operate across multiple applications simultaneously, allowing workflows that span several programs to be automated seamlessly. -
12
Chitu
High-performance inference framework for large language models
...It supports heterogeneous computing environments, including CPUs, GPUs, and various specialized AI accelerators, allowing models to run across a wide range of infrastructure configurations. Chitu is designed to scale from small single-machine deployments to large distributed clusters that handle high volumes of concurrent inference requests. The system also includes performance optimizations for large models, including support for quantized formats and efficient computation operators that reduce memory usage and latency. Its architecture aims to support enterprise adoption by ensuring stable long-term operation under production workloads. -
13
Bedrock Chat
AWS-native chatbot using Bedrock
Bedrock Chat is a mirrored version of an open-source project that provides a conversational interface for interacting with large language models and AI services through a chat-style application. The project typically focuses on delivering a user interface that allows individuals or teams to communicate with AI models, manage conversations, and experiment with prompts and responses. Implementations like Bedrock Chat often integrate with model hosting platforms or APIs that provide access to... -
14
TokenCost
Easy token price estimates for 400+ LLMs. TokenOps
...The tool focuses on helping developers understand how much their prompts and generated completions cost when interacting with commercial AI models. It works by counting tokens in prompts and responses before or after sending requests and then applying pricing information associated with different models. This allows engineers building AI applications, chatbots, or autonomous agents to monitor and predict API expenses during development and production. The library includes pricing information for hundreds of language models and is frequently updated to reflect pricing changes from major AI providers. -
15
LangServe
Helps developers deploy LangChain runnables and chains as a REST API
LangServe is an open-source deployment framework designed to expose LangChain applications as production-ready REST APIs. The tool simplifies the process of turning language-model pipelines, chains, and agents into web services that can be accessed by external applications. Instead of manually writing API endpoints, developers can use LangServe to automatically generate a server that exposes LangChain workflows through HTTP interfaces. The framework is built on top of FastAPI and uses... -
16
LightLLM
LightLLM is a Python-based LLM (Large Language Model) inference
...LightLLM is designed to handle large-scale model workloads in production environments, supporting efficient batching and GPU utilization for fast inference across multiple requests. Its architecture allows models to be deployed with minimal overhead while maintaining compatibility with popular transformer-based model families such as LLaMA and GPT-style architectures. -
17
TokenSpeed
TokenSpeed is a speed-of-light LLM inference engine
...It aims to combine TensorRT-LLM-level speed with vLLM-level usability, making it relevant for teams that need fast generation without sacrificing developer ergonomics. The project is focused on the specific needs of agentic systems, where latency, throughput, and efficient scheduling matter across many short or tool-heavy requests. It builds on ideas and components from the broader open-source inference ecosystem while presenting its own execution stack. TokenSpeed is useful for developers building local or server-side LLM infrastructure for agents, coding systems, and high-volume AI applications. Its main value is providing an inference layer optimized for fast token generation under practical agent workloads. -
18
Integuru v0
The first AI agent that builds permissionless integrations
...Instead of relying on official developer documentation or publicly available APIs, the system analyzes network traffic generated by user interactions within a web application. Developers capture browser requests and authentication data, which the agent then uses to infer the structure of the platform’s internal API endpoints. Based on this information, the system generates executable code that can replicate the original action programmatically. This approach allows developers to automate workflows and build integrations with services that do not provide official APIs or developer tools. ... -
19
Mosec
A high-performance ML model serving framework, offers dynamic batching
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API. -
20
Shell-AI
LangChain powered shell command generator and runner CLI
...Instead of requiring users to remember complex command syntax, the tool lets them describe their intent in plain English and automatically suggests commands that accomplish the task. The system is powered by large language models and integrates with frameworks such as LangChain to interpret user requests and translate them into executable shell instructions. Users interact with the program through an interactive terminal interface where multiple command suggestions are presented for review before execution. This approach improves productivity for developers and system administrators who frequently use terminal environments but may not recall every command variation. ... -
21
Paddler
Open-source LLM load balancer and serving platform for hosting LLMs
...The system acts as a specialized load balancer and serving layer for language models, enabling organizations to run inference workloads without relying on external API providers. It supports running models locally through engines such as llama.cpp while distributing requests across multiple compute nodes to improve performance and reliability. The architecture is designed with privacy and cost control in mind, making it suitable for organizations that handle sensitive data or require predictable operational costs. Paddler also includes tools for monitoring, request buffering, and autoscaling integration so that deployments can adapt dynamically to changing workloads. ... -
22
MCP Router
A Unified MCP Server Management App (MCP Manager)
...This enables AI applications to access multiple tools and knowledge sources through a unified interface rather than connecting to each service individually. The project provides infrastructure for routing requests between clients and MCP servers, enabling scalable multi-tool agent systems. Developers building AI agents can use the platform to manage tool endpoints, control service availability, and simplify agent integration workflows. -
23
vLLM Semantic Router
System Level Intelligent Router for Mixture-of-Models at Cloud
Semantic Router is an open-source system designed to intelligently route requests across multiple large language models based on the semantic meaning and complexity of user queries. Instead of sending every prompt to the same model, the system analyzes the intent and reasoning requirements of the request and dynamically selects the most appropriate model to process it. This approach allows developers to combine multiple models with different strengths, such as lightweight models for simple queries and more advanced reasoning models for complex tasks. ... -
24
Granite 3.0 Language Models
New set of lightweight state-of-the-art, open foundation models
...Documentation highlights the capability mix (reasoning, tool use, code) and points to model artifacts and guidance for evaluation. Activity on the project shows an evolving codebase with open pull requests and standard GitHub project structure for issues and security visibility. In practice, this is a hub for acquiring Granite 3.0 variants and understanding how to integrate them into applications. -
25
Kaleidoscope-SDK
User toolkit for analyzing and interfacing with Large Language Models
...Users must authenticate using their Vector Institute cluster credentials. This can be done interactively instantiating a client object. This will generate an authentication token that will be used for all subsequent requests. The token will expire after 30 days, at which point the user will be prompted to re-authenticate.
