Search Results for "cpu memory usage"
Sort By:
56 projects for "cpu memory usage" with 2 filters applied:
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
FlexLLMGen
Running large language models on a single GPU
...The system focuses on high-throughput generation workloads where large batches of text must be processed quickly, such as large-scale data extraction or document analysis tasks. Instead of requiring expensive multi-GPU systems, the framework uses techniques such as memory offloading, compression, and optimized batching to run large models on commodity hardware. The architecture distributes computation and memory usage across the GPU, CPU, and disk in order to maximize the number of tokens processed during inference. This design allows organizations to deploy powerful language models for high-volume tasks without the infrastructure costs typically associated with large-scale AI systems. ... -
2
GPU Hot
Real-time NVIDIA GPU dashboard
...The project offers a self-hosted web interface that streams hardware metrics directly from GPU servers, enabling developers, ML engineers, and system administrators to observe GPU utilization and system behavior in real time through a browser. The dashboard collects and displays a wide range of performance metrics including temperature, memory usage, power consumption, clock speeds, fan speed, and active processes. It can scale from monitoring a single GPU workstation to large distributed environments with dozens or even hundreds of GPUs by running lightweight containers on each node and aggregating the data centrally. -
3
shimmy
Python-free Rust inference server
...This compatibility enables developers to replace remote AI services with locally hosted models while keeping their existing software architecture intact. Shimmy focuses on performance and simplicity, using efficient runtime components to minimize memory usage and startup time compared to heavier inference frameworks. It supports modern model formats such as GGUF and SafeTensors and can automatically discover models stored locally or in common directories used by other AI tools. Advanced capabilities include CPU offloading for Mixture-of-Experts models and GPU acceleration, enabling large models to run on consumer hardware with limited VRAM. -
4
LuxTTS
A high-quality rapid TTS voice cloning model
...Its design emphasizes efficiency and practicality, fitting within modest GPU memory footprints. -
Gemini 3 and 200+ AI Models on One PlatformBuild, govern, and optimize agents and models with Gemini Enterprise Agent Platform.
-
5
AirLLM
AirLLM 70B inference with single 4GB GPU
AirLLM is an open source Python library that enables extremely large language models to run on consumer hardware with very limited GPU memory. The project addresses one of the main barriers to local LLM experimentation by introducing a memory-efficient inference technique that loads model layers sequentially rather than storing the entire model in GPU memory. This layer-wise inference approach allows models with tens of billions of parameters to run on devices with only a few gigabytes of... -
6
Hello-Agents
Building an Intelligent Agent from Scratch
Hello Agents is an open educational project designed to teach developers how to understand, design, and build AI-native agents from the ground up through structured tutorials and practical examples. The project focuses on guiding learners beyond superficial framework usage toward deeper comprehension of agent architecture, reasoning loops, and real-world implementation patterns. It walks users through core concepts such as ReAct-style reasoning, tool usage, memory handling, and multi-step task execution, enabling hands-on experimentation with modern LLM-powered agent systems. The repository is structured as a progressive learning path, combining theory, exercises, and runnable code so users can incrementally build more capable agents. ... -
7
Pocket TTS
A TTS that fits in your CPU (and pocket)
Pocket TTS is a lightweight text-to-speech project designed to run efficiently on CPUs, targeting developers who want local speech generation without depending on GPUs or hosted web APIs. It is built to feel practical in everyday applications, where installation and usage should be as simple as adding a dependency and calling a function. The project focuses on keeping the runtime footprint manageable while still producing natural-sounding speech, which makes it attractive for offline tools, prototypes, and privacy-sensitive workflows. Because it is CPU-oriented, it fits well in server environments where GPU access is limited, in desktop apps, or in edge deployments where simplicity matters more than maximum throughput. ... -
8
MemU
MemU is an open-source memory framework for AI companions
MemU is an agentic memory layer for LLM applications, specifically designed for AI companions. Transform your memory into an intelligent file system that automatically organizes, connects, and evolves with your memories. Simple, fast, and reliable memory infrastructure for AI applications. Powerful tools and dedicated support to scale your AI applications with confidence. -
9
bitnet.cpp
Official inference framework for 1-bit LLMs
bitnet.cpp is the official open-source inference framework and ecosystem designed to enable ultra-efficient execution of 1-bit large language models (LLMs), which quantize most model parameters to ternary values (-1, 0, +1) while maintaining competitive performance with full-precision counterparts. At its core is bitnet.cpp, a highly optimized C++ backend that supports fast, low-memory inference on both CPUs and GPUs, enabling models such as BitNet b1.58 to run without requiring enormous... -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
10
KVCache-Factory
Unified KV Cache Compression Methods for Auto-Regressive Models
...In large language models, the key-value cache stores intermediate attention states that enable efficient token generation during inference, but these caches can consume large amounts of GPU memory when handling long contexts. KVCache-Factory provides a platform for implementing and evaluating multiple compression strategies that reduce memory usage while preserving model performance. The framework integrates several state-of-the-art methods such as PyramidKV, SnapKV, H2O, and StreamingLLM, allowing researchers to compare and experiment with different approaches within the same environment. ... -
11
xLSTM
Neural Network architecture based on ideas of the original LSTM
...The architecture aims to provide competitive performance with transformer-based models while maintaining advantages such as linear computational scaling and efficient memory usage for long sequences. Researchers have demonstrated that xLSTM models can scale to billions of parameters and large training datasets while maintaining efficient inference speeds. -
12
PicoLM
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
PicoLM is an open-source inference framework designed to run large language models on extremely constrained hardware environments such as inexpensive single-board computers and embedded systems. The project focuses on enabling efficient local inference by optimizing memory usage, computation, and system dependencies so that relatively large models can operate on devices with minimal RAM. It is written primarily in C and designed with a minimalist architecture that removes unnecessary dependencies and external libraries. The runtime is capable of running language models with billions of parameters on devices with only a few hundred megabytes of memory, which is significantly lower than typical LLM infrastructure requirements. ... -
13
OpenAI CS Agents Demo
Demo of a customer service use case implemented with the OpenAI Agents
...It also demonstrates guardrails to validate or constrain responses, memory usage to maintain context, and tracing to help debugging of workflows. -
14
Tencent-Hunyuan-Large
Open-source large language model family from Tencent Hunyuan
...It is designed with long-context capabilities, quantization support, and high performance on benchmarks across general reasoning, mathematics, language understanding, and Chinese / multilingual tasks. It aims to provide competitive capability with efficient deployment and inference. FP8 quantization support to reduce memory usage (~50%) while maintaining precision. High benchmarking performance on tasks like MMLU, MATH, CMMLU, C-Eval, etc. -
15
R-KV
Redundancy-aware KV Cache Compression for Reasoning Models
R-KV is an open-source research project that focuses on improving the efficiency of large language model inference through key-value cache compression techniques. Modern transformer models rely heavily on KV caches during autoregressive decoding, which store intermediate attention states to accelerate generation. However, these caches can consume large amounts of memory, especially in reasoning-oriented models with long context windows. R-KV introduces a method for compressing the KV cache... -
16
AI Agent Deep Dive
AI Agent Source Code Deep Research Report
AI Agent Deep Dive is a comprehensive educational repository designed to provide a deep and structured understanding of how modern AI agents work, focusing on architecture, workflows, and real-world implementation patterns. It breaks down complex concepts such as planning, tool usage, memory management, and multi-step reasoning into digestible explanations and practical examples. The project is organized as a learning resource rather than a standalone framework, making it particularly useful for developers who want to move beyond surface-level prompt engineering into full agent system design. It explores how agents interact with environments, execute tasks, and maintain context over time, highlighting both strengths and limitations of current approaches. ... -
17
Mistral Finetune
Memory-efficient and performant finetuning of Mistral's models
mistral-finetune is an official lightweight codebase designed for memory-efficient and performant finetuning of Mistral’s open models (e.g. 7B, instruct variants). It builds on techniques like LoRA (Low-Rank Adaptation) to allow customizing models without full parameter updates, which reduces GPU memory footprint and training cost. The repo includes utilities for data preprocessing (e.g. reformat_data.py), validation scripts, and example YAML configs for training variants like 7B base or... -
18
llmfit
157 models, 30 providers, one command to find what runs on hardware
llmfit is a terminal-based utility that helps developers determine which large language models can realistically run on their local hardware by analyzing system resources and model requirements. The tool automatically detects CPU, RAM, GPU, and VRAM specifications, then ranks available models based on performance factors such as speed, quality, and memory fit. It provides both an interactive terminal user interface and a traditional CLI mode, enabling flexible workflows for different user preferences. llmfit also supports advanced configurations including multi-GPU setups, mixture-of-experts architectures, and dynamic quantization recommendations. ... -
19
Claude Cognitive
Persistent context and multi-instance coordination
Claude Cognitive is an advanced memory and context-management extension designed to address the stateless limitations of Claude Code by giving the model a form of persistent “working memory” and multi-instance coordination. It introduces an attention-based context router that prioritizes files and content relevant to the current development discussion — tagging them as HOT, WARM, or COLD based on recency and keyword activation — so Claude Code doesn’t waste token budget rereading irrelevant code. ... -
20
SAM 2
The repository provides code for running inference with SAM 2
SAM2 is a next-generation version of the Segment Anything Model (SAM), designed to improve performance, generalization, and efficiency in promptable image segmentation tasks. It retains the core promptable interface—accepting points, boxes, or masks—but incorporates architectural and training enhancements to produce higher-fidelity masks, better boundary adherence, and robustness to complex scenes. The updated model is optimized for faster inference and lower memory use, enabling real-time... -
21
GenericAgent
Self-evolving autonomous agent framework
The GenericAgent project is a flexible framework for building autonomous AI agents that can operate across diverse tasks and environments. It is designed around modularity, allowing developers to define agents with interchangeable components such as tools, memory systems, and reasoning strategies. The architecture emphasizes generality, enabling the same agent framework to be adapted for different domains including coding, research, and task automation. It integrates with modern language... -
22
Build Your Own OpenClaw
A step-by-step guide to build your own AI agent
Build Your Own OpenClaw is a step-by-step educational framework that teaches developers how to construct a fully functional AI agent system from scratch, gradually evolving from a simple chat loop into a multi-agent, production-ready architecture. The project is structured into 18 progressive stages, each introducing a new concept such as tool usage, memory persistence, event-driven design, and multi-agent coordination, with each step including both explanatory documentation and runnable code. It begins with foundational concepts like conversational loops and tool integration, then expands into more advanced capabilities such as dynamic skill loading, web interaction, and context management. ... -
23
Sopro TTS
A lightweight text-to-speech model with zero-shot voice cloning
Sopro TTS is an open-source text-to-speech (TTS) project that implements a lightweight model capable of producing speech from text with zero-shot voice cloning, meaning it can mimic a speaker’s voice from only a few seconds of reference audio. Built with a 169 million-parameter architecture that uses dilated convolutions and cross-attention layers instead of large Transformer stacks, it achieves relatively fast real-time performance even on CPUs (about a 0.25 real-time factor measured on an... -
24
BitNet
BitNet: Scaling 1-bit Transformers for Large Language Models
...The project implements the BitNet architecture described in research on scaling transformer models using extremely low-bit quantization techniques. In this approach, neural network weights are quantized to approximately one bit per parameter, allowing models to operate with far lower memory usage than traditional 16-bit or 32-bit neural networks. The architecture introduces specialized layers such as BitLinear, which replace standard linear projections in transformer networks with quantized operations. By limiting weight precision while maintaining efficient scaling and normalization strategies, the architecture aims to retain competitive performance while significantly reducing hardware requirements. -
25
OpenClaw Opik Observability Plugin
Official plugin for OpenClaw that exports agent traces to Opik
...The project integrates directly with OpenClaw’s plugin architecture so that developers can capture detailed runtime information about how their agents behave while executing tasks. Each time an AI agent performs an action—such as calling a large language model, invoking a tool, accessing memory, or delegating to a sub-agent—the plugin records the full interaction and sends it to Opik for analysis and visualization. This allows developers to inspect inputs, outputs, token usage, latency, and execution flow across complex multi-step agent workflows. The goal of the project is to provide transparency into the internal reasoning and operational pipeline of agent systems so developers can diagnose failures, control costs, and improve reliability.
