Search Results for "gpu hardware"
Sort By:
Showing 211 open source projects for "gpu hardware"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
GPU Hot
Real-time NVIDIA GPU dashboard
GPU Hot is an open-source, lightweight monitoring dashboard designed to provide real-time visibility into NVIDIA GPU performance across single machines or entire clusters. The project offers a self-hosted web interface that streams hardware metrics directly from GPU servers, enabling developers, ML engineers, and system administrators to observe GPU utilization and system behavior in real time through a browser. -
2
GPU Puzzles
Solve puzzles. Learn CUDA
GPU Puzzles is an educational project designed to teach GPU programming concepts through interactive coding exercises and puzzles. Instead of presenting traditional lecture-style explanations, the project immerses learners directly in hands-on programming tasks that demonstrate how GPU computation works. The exercises are implemented using Python with the Numba CUDA interface, which allows Python code to compile into GPU kernels that run on CUDA-enabled hardware. -
3
Libre Hardware Monitor
Monitor temperature sensors, fan speed, voltage, load & clock speeds
...LibreHardwareMonitor supports modern Intel and AMD CPUs, major GPU vendors, storage devices, and network adapters. Built on modern .NET versions, it continues to evolve with frequent updates and broad community contributions. Licensed under MPL 2.0, it offers a transparent and extensible alternative to proprietary hardware monitoring tools. -
4
Open Hardware Monitor
Monitor temperature sensors, fan speeds, voltages, load & clock speeds
...It can also read Intel and AMD processor temperature sensors, GPU sensors from ATI and Nvidia cards, and SMART hard drive temperatures. Users can view monitored values in the main window, system tray, or a customizable desktop gadget. Open Hardware Monitor runs on 32-bit and 64-bit Windows systems as well as x86-based Linux operating systems without requiring installation. -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
5
NVIDIA GPU Operator
NVIDIA GPU Operator creates/configures/manages GPUs atop Kubernetes
Kubernetes provides access to special hardware resources such as NVIDIA GPUs, NICs, Infiniband adapters and other devices through the device plugin framework. However, configuring and managing nodes with these hardware resources requires the configuration of multiple software components such as drivers, container runtimes or other libraries which are difficult and prone to errors. -
6
ChefKiss Inferno
Emulating Apple Silicon devices
Inferno by ChefKissInc is a low-level systems project focused on enabling hardware acceleration and advanced graphics compatibility on Apple Silicon devices, particularly within unsupported or experimental environments. It is designed to bridge gaps between macOS hardware capabilities and software ecosystems that traditionally rely on different GPU architectures, such as those found in Linux or Windows environments. -
7
jamesob's guide to running SOTA LLMs
Everything I know about running LLMs locally
jamesob's guide to running SOTA LLMs is a practical guide and configuration repository for running high-end language models on local hardware. It documents one developer’s local LLM setup, including hardware choices, GPU layout, storage, PCIe switches, kernel settings, and serving workflows. The repository compares budget levels ranging from dual RTX 3090 systems to high-end multi-GPU workstations with very large VRAM pools. It includes ready-to-run serving configurations for selected models and local speech-to-text workloads. ... -
8
Rio Terminal
A hardware-accelerated GPU terminal emulator
Rio is a hardware-accelerated GPU terminal emulator built with Rust and WebGPU. It is designed to run across desktops and browser-oriented environments while keeping performance and usability at the center of the experience. The project supports modern terminal expectations such as 24-bit true color, tabs, split panes, font ligatures, and terminal graphics. -
9
CubeCL
Multi-platform high-performance compute language extension for Rust
CubeCL is a low-level compute language and compiler framework designed to simplify and optimize GPU programming for high-performance workloads, particularly in machine learning and numerical computing. It provides an abstraction layer that allows developers to write portable, hardware-efficient compute kernels without directly dealing with complex GPU APIs such as CUDA or OpenCL. CubeCL focuses on delivering predictable performance and composability by exposing explicit control over memory layouts, parallelism, and execution patterns while still maintaining a developer-friendly syntax. ... -
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
10
SwiftShader
SwiftShader is a high-performance CPU-based implementation
SwiftShader is Google’s high-performance CPU-based implementation of the Vulkan 1.3 graphics API, designed to provide a hardware-independent rendering solution for 3D graphics. Unlike traditional GPU drivers, SwiftShader executes graphics commands entirely on the CPU, making it ideal for environments where dedicated graphics hardware is unavailable or unsuitable. It acts as a drop-in replacement for Vulkan drivers, allowing existing applications to run seamlessly by redirecting API calls through its software-based rendering engine. ... -
11
llmfit
157 models, 30 providers, one command to find what runs on hardware
llmfit is a terminal-based utility that helps developers determine which large language models can realistically run on their local hardware by analyzing system resources and model requirements. The tool automatically detects CPU, RAM, GPU, and VRAM specifications, then ranks available models based on performance factors such as speed, quality, and memory fit. It provides both an interactive terminal user interface and a traditional CLI mode, enabling flexible workflows for different user preferences. llmfit also supports advanced configurations including multi-GPU setups, mixture-of-experts architectures, and dynamic quantization recommendations. ... -
12
AirLLM
AirLLM 70B inference with single 4GB GPU
AirLLM is an open source Python library that enables extremely large language models to run on consumer hardware with very limited GPU memory. The project addresses one of the main barriers to local LLM experimentation by introducing a memory-efficient inference technique that loads model layers sequentially rather than storing the entire model in GPU memory. This layer-wise inference approach allows models with tens of billions of parameters to run on devices with only a few gigabytes of VRAM. ... -
13
GPUStack
Performance-optimized AI inference on your GPUs
GPUStack is an open-source GPU cluster management platform designed to simplify the deployment and operation of artificial intelligence models across heterogeneous hardware environments. The system aggregates GPU resources from multiple machines into a unified cluster so developers and administrators can run large language models and other AI workloads efficiently across distributed infrastructure. -
14
XenosRecomp
A tool for converting Xbox 360 shaders to HLSL
XenosRecomp is a specialized project within the Hedge-dev ecosystem that focuses on recompiling and reconstructing the Xenos GPU pipeline used in the Xbox 360, enabling accurate rendering when porting games to modern platforms. It works alongside CPU recompilation tools by translating GPU-specific instructions and behaviors into equivalents that can be executed on modern graphics APIs such as DirectX or Vulkan. This allows recompiled games to maintain visual fidelity while benefiting from modern hardware acceleration. ... -
15
FlexLLMGen
Running large language models on a single GPU
FlexLLMGen is an open-source inference engine designed to run large language models efficiently on limited hardware resources such as a single GPU. The system focuses on high-throughput generation workloads where large batches of text must be processed quickly, such as large-scale data extraction or document analysis tasks. Instead of requiring expensive multi-GPU systems, the framework uses techniques such as memory offloading, compression, and optimized batching to run large models on commodity hardware. -
16
CUDA-Q
C++ and Python support for the CUDA Quantum programming model
...It provides a full toolchain that includes compilers, runtimes, and libraries for writing quantum programs in both C++ and Python. The platform is designed to be hardware-agnostic, allowing developers to run applications on different quantum backends or simulate them efficiently using GPU acceleration when physical quantum hardware is unavailable. It enables complex workflows where classical and quantum computations are tightly integrated, supporting advanced research and real-world applications in quantum computing. ... -
17
HeavyDB
HeavyDB (formerly MapD/OmniSciDB)
...HeavyDB was originally developed as part of the OmniSci platform (formerly MapD) and is commonly used for large-scale analytics and geospatial data processing. The database compiles queries into optimized machine code that executes efficiently on GPU hardware, significantly accelerating analytical workloads. It supports hybrid deployment environments where queries can run on both CPU and GPU architectures depending on the available resources. -
18
LocalAI
The free, Open Source alternative to OpenAI, Claude and others
LocalAI is an open-source platform that allows users to run large language models and other AI systems locally on their own hardware. It acts as a drop-in replacement for APIs such as OpenAI, enabling developers to build AI-powered applications without relying on external cloud services. The platform supports a wide range of model types, including text generation, image creation, speech processing, and embeddings. LocalAI can run on consumer-grade hardware and does not necessarily require a GPU, making it accessible for local development and private deployments. ... -
19
jetson-stats
Simple package for monitoring and control your NVIDIA Jetson
...It supports Jetson platforms such as Orin, Xavier, Nano, TX, and newer Jetson families. The project includes jtop, a terminal-based dashboard that gives users a live view of CPU, GPU, memory, power, temperature, engine usage, and system status. It can also be imported into Python scripts, which makes it useful for custom monitoring, robotics dashboards, automation, and diagnostics. Developers working with embedded AI systems can use it to understand performance limits and troubleshoot hardware behavior during deployment. ... -
20
Lucebox
Fast LLM speculative inference server for consumer hardware
Lucebox is a local LLM inference server built for fast generation on consumer hardware. It focuses on custom kernels, speculative prefill, speculative decoding, and model-specific optimizations rather than a generic one-size-fits-all runtime. The project includes a native C++ HTTP server with an OpenAI-compatible API, making it usable with tools that already speak the Chat Completions format. It supports CUDA and ROCm workflows, with Docker images for NVIDIA and AMD GPU setups. ... -
21
Parallax
Parallax is a distributed model serving framework
Parallax is a decentralized inference framework designed to run large language models across distributed computing resources. Instead of relying on centralized GPU clusters in data centers, the system allows multiple heterogeneous machines to collaborate in serving AI inference workloads. Parallax divides model layers across different nodes and dynamically coordinates them to form a complete inference pipeline. A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. ... -
22
PowerInfer
High-speed Large Language Model Serving for Local Deployment
...PowerInfer incorporates specialized algorithms and sparse operators to manage neuron activation patterns and minimize data transfers between hardware components. As a result, it enables powerful language models to run on consumer hardware while achieving performance comparable to more expensive server-grade systems. -
23
whichllm
Find the local LLM that actually runs and performs best
whichllm is a command-line tool for finding local large language models that can realistically run on a user’s hardware. It detects the machine’s available resources, including GPU, CPU, memory, and storage, then recommends models based on practical fit rather than parameter count alone. The project is useful for users who are unsure which local LLM will perform well on their system. It focuses on real, recency-aware benchmarks so recommendations better reflect current model performance. whichllm is especially helpful for developers, AI hobbyists, and researchers comparing local inference options across NVIDIA, AMD, Apple Silicon, and CPU-only environments. ... -
24
Scalene
High-performance CPU, GPU, and memory profiler for Python
Scalene is a high-performance CPU, GPU and memory profiler for Python that does a number of things that other Python profilers do not and cannot do. It runs orders of magnitude faster than other profilers while delivering far more detailed information. Once Scalene has profiled your program, it will launch a web browser with an interactive user interface (all processing is done locally). Hover over bars to see breakdowns of CPU and memory consumption, and click on underlined column headers... -
25
llama.cpp
LLM inference in C/C++
llama.cpp is a high-performance C and C++ project for running large language models locally and in the cloud with minimal setup. It is built around efficient inference, broad hardware support, and the GGUF model format. The project supports many model families and has become a major foundation for local AI tools, model serving, and embedded inference workflows. It provides command-line tools, a server mode with an OpenAI-compatible API style, model conversion utilities, and extensive backend acceleration options. llama.cpp runs on CPUs and GPUs, with support for Apple silicon, x86, RISC-V, CUDA, HIP, Vulkan, SYCL, Metal, and hybrid CPU-GPU execution. ...
