Search Results for "token queue management"
Sort By:
Showing 17 open source projects for "token queue management"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
Application Monitoring That Won't Slow Your App DownFull APM with errors, performance, logs, and uptime monitoring. 99.999% uptime SLA on the platform itself.
-
1
Claude Cognitive
Persistent context and multi-instance coordination
Claude Cognitive is an advanced memory and context-management extension designed to address the stateless limitations of Claude Code by giving the model a form of persistent “working memory” and multi-instance coordination. It introduces an attention-based context router that prioritizes files and content relevant to the current development discussion — tagging them as HOT, WARM, or COLD based on recency and keyword activation — so Claude Code doesn’t waste token budget rereading irrelevant code. ... -
2
Claude Code Usage Monitor
Real-time Claude Code usage monitor with predictions and warnings
Claude Code Usage Monitor is a developer-focused terminal tool that provides real-time visibility into Claude Code token consumption and session behavior. The project is designed to help users avoid unexpectedly hitting usage caps by continuously tracking token burn rate, message counts, and estimated costs during active sessions. It presents analytics through a visually rich terminal interface built with modern Python tooling, making it easy to interpret usage trends at a glance. ... -
3
OpenAI Forward
An efficient forwarding service designed for LLMs
OpenAI Forward is an open-source forwarding and reverse proxy service for large language model APIs, designed to sit between client applications and model providers. Its main purpose is to make model access more manageable and efficient by adding operational controls such as request rate limiting, token rate limiting, caching, logging, routing, and key management around existing LLM endpoints. The project can proxy both local and cloud-hosted language model services, which makes it useful for teams that want a single control layer regardless of whether they are using something like LocalAI or a hosted provider compatible with OpenAI-style APIs. ... -
4
AGiXT
AGiXT is a dynamic AI Automation Platform
AGiXT is a dynamic Artificial Intelligence Automation Platform engineered to orchestrate efficient AI instruction management and task execution across a multitude of providers. Our solution infuses adaptive memory handling with a broad spectrum of commands to enhance AI's understanding and responsiveness, leading to improved task completion. The platform's smart features, like Smart Instruct and Smart Chat, seamlessly integrate web search, planning strategies, and conversation continuity,... -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
5
Continuous Claude v3
Context management for Claude Code. Hooks maintain state via ledgers
Continuous Claude v3 is a persistent, multi-agent development environment built around the Claude Code CLI that aims to overcome the limitations of standard LLM context windows. Rather than relying on a single session’s context, Continuous Claude uses mechanisms like ledgers, YAML handoffs, and a memory system to preserve and recall state across multiple sessions, ensuring that learned insights and plans are not lost when context compaction occurs. The project orchestrates many specialized... -
6
InfiAgent
Build your own Cowork, AI Scientist and other SoTA Agents
infiAgent is an open-source AI agent framework for building powerful, long-running autonomous agents capable of tackling complex tasks without collapsing under growing context or tool invocation histories. Designed as a “Multi-Level Agent” (MLA) system, it externalizes persistent state to the file system so that agents can operate over unlimited runtime without the need for token-intensive context compression, enabling workflows such as research paper drafting, experiments, coding, and... -
7
API-for-Open-LLM
Openai style api for open large language models
API-for-Open-LLM is a lightweight API server designed for deploying and serving open large language models (LLMs), offering a simple way to integrate LLMs into applications. -
8
SkyPilot
SkyPilot: Run AI and batch jobs on any infra
SkyPilot is a framework for running AI and batch workloads on any infra, offering unified execution, high cost savings, and high GPU availability. Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface. -
9
LLM TLDR
95% token savings. 155x faster queries. 16 languages
LLM TLDR is a tool that leverages large language models (LLMs) to generate concise, coherent summaries (TL;DRs) of long documents, articles, or text files, helping users quickly understand large amounts of content without reading every word. It integrates with LLM APIs to handle input texts of varying lengths and complexity, applying techniques like chunking, context management, and multi-pass summarization to preserve accuracy even when the source is very large. The system supports both... -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
Cleanlab
The standard data-centric AI package for data quality and ML
cleanlab helps you clean data and labels by automatically detecting issues in a ML dataset. To facilitate machine learning with messy, real-world data, this data-centric AI package uses your existing models to estimate dataset problems that can be fixed to train even better models. cleanlab cleans your data's labels via state-of-the-art confident learning algorithms, published in this paper and blog. See some of the datasets cleaned with cleanlab at labelerrors.com. This package helps you... -
11
AutoClip
AI-powered video clipping and highlight generation
...It uses a modern web application stack with a front end (React + TypeScript) for user interaction and a back end that handles downloading, processing, clipping, and queue management, allowing real-time progress feedback and easy deployment, e.g. via Docker. -
12
StableSwarmUI
Multi-user UI for managing and running Stable Diffusion workflows tool
StableSwarmUI is a web-based interface designed to manage and coordinate Stable Diffusion image generation workflows in a multi-user environment. It focuses on enabling multiple users to interact with shared resources, making it suitable for collaborative or server-based deployments. It provides a centralized system where users can submit, monitor, and manage generation tasks through a browser interface. It abstracts much of the complexity involved in running diffusion models by offering a... -
13
FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs
...The platform enables developers to deploy trained models quickly using optimized inference pipelines that support GPUs, specialized AI accelerators, and other hardware architectures. FastDeploy includes advanced acceleration technologies such as speculative decoding, multi-token prediction, and efficient KV cache management to improve throughput and latency during inference. It also offers compatibility with OpenAI-style APIs and vLLM-like interfaces, allowing developers to integrate deployed models easily into existing applications and services. -
14
Stanza
Stanford NLP Python library for many human languages
Stanza is a collection of accurate and efficient tools for the linguistic analysis of many human languages. Starting from raw text to syntactic analysis and entity recognition, Stanza brings state-of-the-art NLP models to languages of your choosing. Stanza is a Python natural language analysis package. It contains tools, which can be used in a pipeline, to convert a string containing human language text into lists of sentences and words, to generate base forms of those words, their parts of... -
15
GPUStack
Performance-optimized AI inference on your GPUs
GPUStack is an open-source GPU cluster management platform designed to simplify the deployment and operation of artificial intelligence models across heterogeneous hardware environments. The system aggregates GPU resources from multiple machines into a unified cluster so developers and administrators can run large language models and other AI workloads efficiently across distributed infrastructure. Instead of requiring complex orchestration systems such as Kubernetes, GPUStack provides a... -
16
Extended Dreambooth How-To Guides
Implementation of Dreambooth
Extended Dreambooth How-To Guides is an implementation and extended toolkit for fine-tuning Stable Diffusion models using the DreamBooth technique, enabling users to train AI image generators to reproduce specific subjects, styles, or identities from a small set of reference images. The project adapts and expands upon earlier DreamBooth research by providing practical scripts, notebooks, and workflows that allow users to train personalized models on local machines, cloud environments, or... -
17
Zylthra
Zylthra: A PyQt6 app to generate synthetic datasets with DataLLM.
Welcome to Zylthra, a powerful Python-based desktop application built with PyQt6, designed to generate synthetic datasets using the DataLLM API from data.mostly.ai. This tool allows users to create custom datasets by defining columns, configuring generation parameters, and saving setups for reuse, all within a sleek, dark-themed interface.
- Previous
- You're on page 1
- Next
