Showing 22 open source projects for "cache"

View related business solutions
  • Build Securely on AWS with Proven Frameworks Icon
    Build Securely on AWS with Proven Frameworks

    Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

    Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
    Download Now
  • $300 Free Credits to Build on Google Cloud Icon
    $300 Free Credits to Build on Google Cloud

    New customers can spin up VMs, build with AI, and query data at no cost.

    Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
    Start Free
  • 1
    R-KV

    R-KV

    Redundancy-aware KV Cache Compression for Reasoning Models

    R-KV is an open-source research project that focuses on improving the efficiency of large language model inference through key-value cache compression techniques. Modern transformer models rely heavily on KV caches during autoregressive decoding, which store intermediate attention states to accelerate generation. However, these caches can consume large amounts of memory, especially in reasoning-oriented models with long context windows. R-KV introduces a method for compressing the KV cache during decoding, allowing models to maintain reasoning performance while reducing memory consumption and computational overhead. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 2
    KVCache-Factory

    KVCache-Factory

    Unified KV Cache Compression Methods for Auto-Regressive Models

    KVCache-Factory is an open-source research framework designed to explore and implement unified key-value cache compression techniques for autoregressive transformer models. In large language models, the key-value cache stores intermediate attention states that enable efficient token generation during inference, but these caches can consume large amounts of GPU memory when handling long contexts. KVCache-Factory provides a platform for implementing and evaluating multiple compression strategies that reduce memory usage while preserving model performance. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 3
    CAG

    CAG

    Cache-Augmented Generation: A Simple, Efficient Alternative to RAG

    CAG, or Cache-Augmented Generation, is an experimental framework that explores an alternative architecture for integrating external knowledge into large language model responses. Traditional retrieval-augmented generation systems rely on real-time retrieval of documents from databases or vector stores during inference. CAG proposes a different approach by preloading relevant knowledge into the model’s context window and precomputing the model’s key-value cache before queries are processed. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 4
    oMLX

    oMLX

    LLM inference server with continuous batching & SSD caching

    ...It serves text models, vision-language models, OCR models, embeddings, and rerankers through OpenAI- and Anthropic-compatible APIs. Continuous batching allows concurrent requests, while a tiered KV cache keeps active data in RAM and moves colder blocks to SSD for reuse. Models can be pinned, unloaded manually, or evicted automatically when memory runs low. A web dashboard provides monitoring, chat, downloads, benchmarks, integrations, and per-model settings. It also supports tool calling, structured output, MCP integration, and experimental multi-Mac inference.
    Downloads: 172 This Week
    Last Update:
    See Project
  • Patching and Endpoint Management for Under $1/Endpoint Icon
    Patching and Endpoint Management for Under $1/Endpoint

    OS and third-party patching, vulnerability insights, and remote control included

    Splashtop AEM gives IT teams automated patching, monitoring, and unattended remote access for Windows and macOS. No long setup, no paying extra for key features.
    Start Free Trial
  • 5
    LMCache

    LMCache

    Supercharge Your LLM with the Fastest KV Cache Layer

    ...These capabilities aim to lower latency, cut GPU cycles, and stabilize performance for production workloads with overlapping prompts or retrieval-augmented contexts. The end result is a cache fabric for LLMs that complements engines rather than replacing them.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 6
    MSA: Memory Sparse Attention

    MSA: Memory Sparse Attention

    Trainable latent-memory framework for 100M-token contexts

    ...It replaces full attention over all tokens with sparse selection of compressed latent memory states. Document-wise rotary position encoding and top-k routing keep training and inference close to linear complexity. A tiered KV-cache design stores routing keys on GPU while larger content states can remain on CPU. Its Memory Parallel engine distributes scoring and transfers only selected memory back to the accelerator. Memory Interleave alternates retrieval, context expansion, and generation to improve multi-hop reasoning across distant segments. The project reports experiments extending from 16K to 100M tokens, including inference on two A800 GPUs.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 7
    claude-obsidian

    claude-obsidian

    Claude + Obsidian knowledge companion

    claude-obsidian is an AI-powered knowledge engine that transforms an Obsidian vault into a self-organizing, continuously evolving wiki. Instead of acting as a simple chat assistant, it autonomously creates, links, and maintains structured knowledge based on user inputs and external sources. The system follows the LLM Wiki pattern, where information is stored as persistent markdown files that grow richer over time through cross-referencing and synthesis. It includes features such as...
    Downloads: 18 This Week
    Last Update:
    See Project
  • 8
    FastDeploy

    FastDeploy

    High-performance Inference and Deployment Toolkit for LLMs and VLMs

    ...The platform enables developers to deploy trained models quickly using optimized inference pipelines that support GPUs, specialized AI accelerators, and other hardware architectures. FastDeploy includes advanced acceleration technologies such as speculative decoding, multi-token prediction, and efficient KV cache management to improve throughput and latency during inference. It also offers compatibility with OpenAI-style APIs and vLLM-like interfaces, allowing developers to integrate deployed models easily into existing applications and services.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 9
    mac code

    mac code

    Claude Code, but it runs on your Mac for free

    mac code is a local AI coding agent designed to run large language models directly on Apple Silicon machines without relying on cloud services, effectively transforming a Mac into a self-contained AI development environment. The project focuses on enabling models that traditionally exceed available RAM to run efficiently by streaming model weights from SSD storage, thereby overcoming hardware limitations through innovative memory management techniques. It operates as a CLI-based assistant...
    Downloads: 2 This Week
    Last Update:
    See Project
  • Paessler: Easy to Use With Enterprise Power. Free Trial Icon
    Paessler: Easy to Use With Enterprise Power. Free Trial

    A low-code dashboard makes monitoring intuitive for any admin, while scripting and custom sensors give experts full control.

    You shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
    Get Free Download
  • 10
    TensorRT LLM

    TensorRT LLM

    TensorRT LLM provides users with an easy-to-use Python API

    TensorRT-LLM is an open-source high-performance inference library specifically designed to optimize and accelerate large language model deployment on NVIDIA GPUs. It provides a Python-based API built on top of PyTorch that allows developers to define, customize, and deploy LLMs efficiently across a variety of hardware configurations, from single GPUs to large multi-node clusters. The library focuses on maximizing throughput and minimizing latency through advanced techniques such as...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 11
    How to Train Your GPT

    How to Train Your GPT

    Build a modern LLM from scratch. Every line commented

    ...The project covers the same broad family of architecture behind systems such as GPT-style models, LLaMA-style models, Claude-style systems, and Mistral-style models. It includes chapters and topic explainers on tokenizers, embeddings, attention, RoPE, RMSNorm, SwiGLU, KV cache, AdamW, mixed precision, training loops, and inference. The guide emphasizes writing every important component manually rather than only calling high-level APIs. Its purpose is to make the internals of language models understandable through runnable code and step-by-step explanations.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 12
    ModelScope

    ModelScope

    Bring the notion of Model-as-a-Service to life

    ModelScope is built upon the notion of “Model-as-a-Service” (MaaS). It seeks to bring together most advanced machine learning models from the AI community, and streamlines the process of leveraging AI models in real-world applications. The core ModelScope library open-sourced in this repository provides the interfaces and implementations that allow developers to perform model inference, training and evaluation. In particular, with rich layers of API abstraction, the ModelScope library offers...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 13
    HunyuanWorld-Voyager

    HunyuanWorld-Voyager

    RGBD video generation model conditioned on camera input

    HunyuanWorld-Voyager is a next-generation video diffusion framework developed by Tencent-Hunyuan for generating world-consistent 3D scene videos from a single input image. By leveraging user-defined camera paths, it enables immersive scene exploration and supports controllable video synthesis with high realism. The system jointly produces aligned RGB and depth video sequences, making it directly applicable to 3D reconstruction tasks. At its core, Voyager integrates a world-consistent video...
    Downloads: 2 This Week
    Last Update:
    See Project
  • 14
    ContextForge MCP Gateway

    ContextForge MCP Gateway

    A Model Context Protocol (MCP) Gateway & Registry

    MCP Context Forge is a feature-rich gateway and registry that federates Model Context Protocol (MCP) servers and traditional REST services behind a single, governed endpoint. It exposes an MCP-compliant interface to clients while handling discovery, authentication, rate limiting, retries, and observability on the server side. The gateway scales horizontally, supports multi-cluster deployments on Kubernetes, and uses Redis for federation and caching across instances. Operators can define...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 15
    tiny-llm

    tiny-llm

    A course of learning LLM inference serving on Apple Silicon

    tiny-llm is an educational open-source project designed to teach system engineers how large language model inference and serving systems work by building them from scratch. The project is structured as a guided course that walks developers through the process of implementing the core components required to run a modern language model, including attention mechanisms, token generation, and optimization techniques. Rather than relying on high-level machine learning frameworks, the codebase uses...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    Ytranslate

    Ytranslate

    AI Desktop Translator & Universal Voice Typing for Windows (5 Engines)

    ...Features 4 modes (Popup Translator, Minimal Popup, Translation Studio Window with Back-Translation, and natural Text-to-Speech Voice mode), Mini Strip (Yt) mode with background selection memory, Double Right-Click quick menu, smart clipboard protection, 0ms local SHA-256 cache, 6 modern themes, and full UI localization in 93 languages.
    Downloads: 6 This Week
    Last Update:
    See Project
  • 17
    GPTCache

    GPTCache

    Semantic cache for LLMs. Fully integrated with LangChain

    ...Additionally, LLM services might exhibit slow response times, especially when dealing with a significant number of requests. To tackle this challenge, we have created GPTCache, a project dedicated to building a semantic cache for storing LLM responses. This project is undergoing swift development, and as such, the API may be subject to change at any time.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 18
    Bert-VITS2

    Bert-VITS2

    VITS2 backbone with multilingual-bert

    Bert-VITS2 is a neural text-to-speech project that combines a VITS2 backbone with a multilingual BERT front-end to produce high-quality speech in multiple languages. The core idea is to use BERT-style contextual embeddings for text encoding while relying on a refined VITS2 architecture for acoustic generation and vocoding. The repository includes everything needed to train, fine-tune, and run the model, from configuration files to preprocessing scripts, spectrogram utilities, and training...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 19
    RWKV

    RWKV

    RNN with great LLM performance

    ...The project is built around the idea that a model can be trained in a parallelizable way like a GPT-style transformer while running inference with recurrent efficiency. This gives RWKV important advantages for long-context use, including lower memory pressure and no traditional key-value cache requirement. The repository includes training code, model notes, research material, and references to current RWKV weights. Its main value is providing the foundation for experimenting with efficient large language models that combine transformer-like scalability with RNN-like runtime behavior.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 20
    Audio Webui

    Audio Webui

    A webui for different audio related Neural Networks

    ...Installation is streamlined through automatic installers and platform-specific scripts that create a virtual environment, install dependencies, and launch the web app with minimal manual setup. For more advanced users, it exposes a rich set of command-line flags to control behavior such as skipping installation, disabling venv, changing model cache directories, sharing Gradio links, setting passwords, and specifying themes or ports.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    Photonix Photo Manager

    Photonix Photo Manager

    A modern, web-based photo management server

    ...Once you have finished trying out the system you can edit the volume in the docker-compose.yml file where it says ./data/photos to mount wherever you usually keep photos. System database, thumbnails and other cache data is stored separately from the photos so shouldn't pollute the area. You are responsible for keeping your own backups in case of error.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    fastNLP

    fastNLP

    fastNLP: A Modularized and Extensible NLP Framework

    ...Built-in Loader and Pipe for multiple datasets, eliminating the need for preprocessing code. Various convenient NLP tools, such as Embedding loading (including ELMo and BERT), intermediate data cache, etc.. Provide a variety of neural network components and recurrence models (covering tasks such as Chinese word segmentation, named entity recognition, syntactic analysis, text classification, text matching, metaphor resolution, summarization, etc.). Trainer provides a variety of built-in Callback functions to facilitate experiment recording, exception capture, etc. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • Next