Search Results for "requests"
Sort By:
Showing 20 open source projects for "requests"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Infinity
Low-latency REST API for serving text-embeddings
Infinity is a high-throughput, low-latency REST API for serving vector embeddings, supporting all sentence-transformer models and frameworks. Infinity is developed under MIT License. Infinity powers inference behind Gradient.ai and other Embedding API providers. -
2
AI-Codereview-Gitlab
GitLab automatic code review tool based on large models
...When code changes occur, the system can automatically generate review comments and feedback that are posted directly into GitLab merge requests, allowing developers to see suggestions alongside human reviewer comments. In addition to code analysis, the tool can produce daily development summaries and notifications that help teams track progress and review activity across projects. -
3
vLLM
A high-throughput and memory-efficient inference and serving engine
vLLM is a fast and easy-to-use library for LLM inference and serving. High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more. -
4
OpenOutreach
Linkedin Automation Tool
...According to the repository, it combines large language model classification with a Bayesian machine learning layer based on profile embeddings, which helps it shift from broad exploration to more confident qualification as it gathers more decisions. It is designed to automate personalized outreach as well, including connection requests and follow-up messaging, while keeping deployment under the user’s control through a local or self-hosted setup. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Headroom
Compress tool outputs, logs, files, and RAG chunks
Headroom is a context optimization layer for LLM applications that compresses information before it reaches the model. It sits between an application and an LLM provider, intercepting requests and forwarding a shorter optimized prompt. The project is designed to reduce token usage while preserving the answer quality needed for agent workflows. It can compress tool outputs, logs, RAG chunks, files, and conversation history. Headroom can be used as a transparent proxy, a Python function, a TypeScript SDK, or through integrations with frameworks such as LangChain and LiteLLM. ... -
6
Chitu
High-performance inference framework for large language models
...It supports heterogeneous computing environments, including CPUs, GPUs, and various specialized AI accelerators, allowing models to run across a wide range of infrastructure configurations. Chitu is designed to scale from small single-machine deployments to large distributed clusters that handle high volumes of concurrent inference requests. The system also includes performance optimizations for large models, including support for quantized formats and efficient computation operators that reduce memory usage and latency. Its architecture aims to support enterprise adoption by ensuring stable long-term operation under production workloads. -
7
oMLX
LLM inference server with continuous batching & SSD caching
...It serves text models, vision-language models, OCR models, embeddings, and rerankers through OpenAI- and Anthropic-compatible APIs. Continuous batching allows concurrent requests, while a tiered KV cache keeps active data in RAM and moves colder blocks to SSD for reuse. Models can be pinned, unloaded manually, or evicted automatically when memory runs low. A web dashboard provides monitoring, chat, downloads, benchmarks, integrations, and per-model settings. It also supports tool calling, structured output, MCP integration, and experimental multi-Mac inference. -
8
LightLLM
LightLLM is a Python-based LLM (Large Language Model) inference
...LightLLM is designed to handle large-scale model workloads in production environments, supporting efficient batching and GPU utilization for fast inference across multiple requests. Its architecture allows models to be deployed with minimal overhead while maintaining compatibility with popular transformer-based model families such as LLaMA and GPT-style architectures. -
9
TokenSpeed
TokenSpeed is a speed-of-light LLM inference engine
...It aims to combine TensorRT-LLM-level speed with vLLM-level usability, making it relevant for teams that need fast generation without sacrificing developer ergonomics. The project is focused on the specific needs of agentic systems, where latency, throughput, and efficient scheduling matter across many short or tool-heavy requests. It builds on ideas and components from the broader open-source inference ecosystem while presenting its own execution stack. TokenSpeed is useful for developers building local or server-side LLM infrastructure for agents, coding systems, and high-volume AI applications. Its main value is providing an inference layer optimized for fast token generation under practical agent workloads. -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
Parallax
Parallax is a distributed model serving framework
...Parallax divides model layers across different nodes and dynamically coordinates them to form a complete inference pipeline. A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. This scheduling system optimizes latency, throughput, and hardware utilization even when nodes have different computational capabilities. The platform also supports model sharding and pipeline parallelism, allowing very large models to run across distributed resources. -
11
TokenCost
Easy token price estimates for 400+ LLMs. TokenOps
...The tool focuses on helping developers understand how much their prompts and generated completions cost when interacting with commercial AI models. It works by counting tokens in prompts and responses before or after sending requests and then applying pricing information associated with different models. This allows engineers building AI applications, chatbots, or autonomous agents to monitor and predict API expenses during development and production. The library includes pricing information for hundreds of language models and is frequently updated to reflect pricing changes from major AI providers. -
12
Integuru v0
The first AI agent that builds permissionless integrations
...Instead of relying on official developer documentation or publicly available APIs, the system analyzes network traffic generated by user interactions within a web application. Developers capture browser requests and authentication data, which the agent then uses to infer the structure of the platform’s internal API endpoints. Based on this information, the system generates executable code that can replicate the original action programmatically. This approach allows developers to automate workflows and build integrations with services that do not provide official APIs or developer tools. ... -
13
UFO³
Weaving the Digital Agent Galaxy
...This enables the agent to navigate complex software environments and perform tasks that normally require manual interaction. UFO integrates mechanisms for task decomposition, planning, and execution so that high-level user requests can be broken down into smaller steps performed by specialized agents. The framework can operate across multiple applications simultaneously, allowing workflows that span several programs to be automated seamlessly. -
14
Mosec
A high-performance ML model serving framework, offers dynamic batching
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API. -
15
Shell-AI
LangChain powered shell command generator and runner CLI
...Instead of requiring users to remember complex command syntax, the tool lets them describe their intent in plain English and automatically suggests commands that accomplish the task. The system is powered by large language models and integrates with frameworks such as LangChain to interpret user requests and translate them into executable shell instructions. Users interact with the program through an interactive terminal interface where multiple command suggestions are presented for review before execution. This approach improves productivity for developers and system administrators who frequently use terminal environments but may not recall every command variation. ... -
16
Kaleidoscope-SDK
User toolkit for analyzing and interfacing with Large Language Models
...Users must authenticate using their Vector Institute cluster credentials. This can be done interactively instantiating a client object. This will generate an authentication token that will be used for all subsequent requests. The token will expire after 30 days, at which point the user will be prompted to re-authenticate. -
17
Ailice
AIlice is a fully autonomous, general-purpose AI agent
AIlice is an open-source autonomous AI agent framework built to function as a general-purpose assistant that can plan, decompose, and execute complex tasks through a structured multi-agent architecture. The project presents itself as a standalone assistant powered by open-source language models, with an internal design that treats user requests almost like executable programs rather than simple chat prompts. Its core IACT architecture allows the system to break large goals into smaller sub-tasks, assign them to dynamically created agents, and combine the results with a focus on resilience and fault tolerance. AIlice is designed for a wide range of workloads, including coding, thematic research, literature analysis, system management, and mixed workflows that require several reasoning modes at once. -
18
Punica
Serving multiple LoRA finetuned LLM as one
...This design allows a single GPU cluster to host many task-specific models while maintaining high throughput and minimal latency. The architecture also includes scheduling mechanisms that coordinate requests from multiple tenants and distribute workloads efficiently across available resources. -
19
Langcorn
Serving LangChain LLM apps automagically with FastApi
LangCorn is an API server that enables you to serve LangChain models and pipelines with ease, leveraging the power of FastAPI for a robust and efficient experience. -
20
mindflow
AI-powered CLI git wrapper, boilerplate code generator, chat history
I-powered CLI git wrapper, boilerplate code generator, chat history manager, and code search engine to streamline your dev workflow. The ChatGPT-powered swiss army knife for the modern developer! We provide an AI-powered CLI git wrapper, boilerplate code generator, code search engine, a conversation history manager, and much more! Configure the model used for generating responses by running mf config and selecting either GPT 3.5 Turbo (default) or GPT 4. In order to use GPT 4, you'll need...
- Previous
- You're on page 1
- Next
