Search Results for "key"
Sort By:
Showing 18 open source projects for "key"
View related business solutions-
Host LLMs in Production With On-Demand GPUsDeploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
Free LLM API resources
A list of free LLM inference resources accessible via API
...This list helps developers, hobbyists, and researchers quickly find models they can use for prototyping, experimentation, or production proofs-of-concept without needing paid subscriptions, reducing friction for innovation. The repository typically categorizes offerings by provider, type of service (text, embeddings, vision), availability conditions (open without key, free tier with key), and usage examples to make discovery and adoption easier. -
2
vLLM
A high-throughput and memory-efficient inference and serving engine
vLLM is a fast and easy-to-use library for LLM inference and serving. High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more. -
3
Paper2Slides
From Paper to Presentation in One Click
Paper2Slides is an automation tool that converts research papers, reports, and other documents into polished slide decks and posters with minimal manual effort. It is designed to replace the repetitive work of turning dense technical documents into presentation-friendly structure by extracting key points, figures, and data into a coherent visual narrative. The system supports multiple input formats, so you can process PDFs and common office documents rather than being locked to a single file type. It uses an extraction approach intended to capture critical insights comprehensively, including important visuals and data points that often get missed in naive summarization. ... -
4
KVCache-Factory
Unified KV Cache Compression Methods for Auto-Regressive Models
KVCache-Factory is an open-source research framework designed to explore and implement unified key-value cache compression techniques for autoregressive transformer models. In large language models, the key-value cache stores intermediate attention states that enable efficient token generation during inference, but these caches can consume large amounts of GPU memory when handling long contexts. KVCache-Factory provides a platform for implementing and evaluating multiple compression strategies that reduce memory usage while preserving model performance. ... -
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
5
R-KV
Redundancy-aware KV Cache Compression for Reasoning Models
R-KV is an open-source research project that focuses on improving the efficiency of large language model inference through key-value cache compression techniques. Modern transformer models rely heavily on KV caches during autoregressive decoding, which store intermediate attention states to accelerate generation. However, these caches can consume large amounts of memory, especially in reasoning-oriented models with long context windows. R-KV introduces a method for compressing the KV cache during decoding, allowing models to maintain reasoning performance while reducing memory consumption and computational overhead. ... -
6
Scikit-LLM
Seamlessly integrate LLMs into scikit-learn
Seamlessly integrate powerful language models like ChatGPT into sci-kit-learn for enhanced text analysis tasks. At the moment the majority of the Scikit-LLM estimators are only compatible with some of the OpenAI models. Hence, a user-provided OpenAI API key is required. Additionally, Scikit-LLM will ensure that the obtained response contains a valid label. If this is not the case, a label will be selected randomly (label probabilities are proportional to label occurrences in the training set). Note: unlike in a typical supervised setting, the performance of a zero-shot classifier greatly depends on how the label itself is structured. ... -
7
GPT-API-free
Free ChatGPT&DeepSeek API Key
GPT-API-free is a project that provides access to GPT-style APIs without requiring direct integration with paid official endpoints, focusing on accessibility and ease of experimentation. It offers a proxy-based approach that allows developers to interact with language models through a simplified interface, often requiring minimal configuration. The system is designed to lower barriers for developers who want to test or build applications using conversational AI without managing billing or... -
8
Mirascope
LLM abstractions that aren't obstructions
Mirascope is a powerful, flexible, and user-friendly library that simplifies the process of working with LLMs through a unified interface that works across various supported providers, including OpenAI, Anthropic, Mistral, Gemini, Groq, Cohere, LiteLLM, Azure AI, Vertex AI, and Bedrock. Whether you're generating text, extracting structured information, or developing complex AI-driven agent systems, Mirascope provides the tools you need to streamline your development process and create... -
9
CAG
Cache-Augmented Generation: A Simple, Efficient Alternative to RAG
...Traditional retrieval-augmented generation systems rely on real-time retrieval of documents from databases or vector stores during inference. CAG proposes a different approach by preloading relevant knowledge into the model’s context window and precomputing the model’s key-value cache before queries are processed. This strategy allows the model to generate responses using the cached context directly, eliminating the need for repeated retrieval operations during runtime. As a result, the approach can significantly reduce latency and simplify system architecture compared with traditional RAG pipelines. ... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
BertViz
BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)
...It is based on the excellent Tensor2Tensor visualization tool. The model view shows a bird's-eye view of attention across all layers and heads. The neuron view visualizes individual neurons in the query and key vectors and shows how they are used to compute attention. -
11
SageAttention
NeurIPS2025 Spotlight] Quantized Attention
...Since attention operations are often the most computationally expensive component of modern AI models, SageAttention introduces quantization techniques that significantly reduce computational overhead while preserving model accuracy. The system achieves this by using low-precision numerical formats such as INT4, FP8, or INT8 to represent key matrices within the attention computation. These optimizations allow models to perform matrix operations faster and consume less memory during inference. SageAttention is designed to function as a plug-and-play replacement for standard attention implementations, enabling developers to accelerate existing models without modifying their architecture. -
12
OpenAI Forward
An efficient forwarding service designed for LLMs
...Its main purpose is to make model access more manageable and efficient by adding operational controls such as request rate limiting, token rate limiting, caching, logging, routing, and key management around existing LLM endpoints. The project can proxy both local and cloud-hosted language model services, which makes it useful for teams that want a single control layer regardless of whether they are using something like LocalAI or a hosted provider compatible with OpenAI-style APIs. A major emphasis of the repository is asynchronous performance, using tools such as uvicorn, aiohttp, and asyncio to support high-throughput forwarding workloads. -
13
AgentEvolver
Towards Efficient Self-Evolving Agent System
...The system focuses on improving the efficiency and scalability of training autonomous agents by allowing them to generate tasks, explore environments, and refine strategies without heavy reliance on manually curated datasets. Its architecture combines reinforcement learning with LLM-driven reasoning mechanisms to guide exploration and learning. The framework introduces several key mechanisms, including self-questioning to create new learning tasks, self-navigating to improve exploration through experience reuse, and self-attributing to assign rewards based on the usefulness of actions. These mechanisms enable agents to continuously improve their capabilities while interacting with complex environments and tools. ... -
14
LLM TLDR
95% token savings. 155x faster queries. 16 languages
...The system supports both extractive and abstractive summarization styles so that users can choose whether they want condensed highlights or a more narrative paraphrase of key ideas. To enhance usability, LLM-TLDR includes command-line tools and integration examples for common workflows like batch summarization, webhook ingestion, and automation in documentation pipelines. -
15
JamAI Base
The collaborative spreadsheet for AI
...It includes built-in orchestration for large language models, vector search, and reranking pipelines so that AI applications can retrieve relevant information before generating responses. JamAI Base exposes its functionality through a simple REST API and a spreadsheet-style interface that allows users to manage AI workflows visually. One of the key ideas behind the platform is the concept of generative tables, which allow database columns to automatically populate with AI-generated content. The system also supports action tables and chat tables that simplify the creation of interactive AI features such as conversational interfaces and dynamic workflows. -
16
handy-ollama
Implement CPU from scratch and play with large model deployments
...The repository serves as a structured tutorial that explains how to install, configure, and use Ollama to run modern language models on personal hardware without requiring advanced infrastructure. A key focus of the project is enabling users to run large models even without GPUs by leveraging optimized CPU-based inference pipelines. The project includes step-by-step guides that walk learners through tasks such as installing Ollama, managing local models, calling model APIs, and building simple AI applications on top of locally hosted models. ... -
17
Controllable-RAG-Agent
This repository provides an advanced RAG
...At query time, it anonymizes entities, creates a high-level plan, de-anonymizes and expands that plan into concrete retrieval or reasoning tasks, and executes them in sequence while continuously revising the plan. A key focus is hallucination control: each answer is verified against retrieved context, and responses are reworked when they are not sufficiently grounded in the source documents. -
18
Repo of Tree of Thoughts (ToT)
Implementation of "Tree of Thoughts
Language models are increasingly being deployed for general problem-solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role. To surmount these challenges, we introduce a new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of...
- Previous
- You're on page 1
- Next
