Search Results for "hardware"
Sort By:
Showing 34 open source projects for "hardware"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
GPT4All
Run Local LLMs on Any Device. Open-source
...The software provides a simple, user-friendly application that can be downloaded and run on various platforms, including Windows, macOS, and Ubuntu, without requiring specialized hardware. It integrates with the llama.cpp implementation and supports multiple LLMs, allowing users to interact with AI models privately. This project also supports Python integrations for easy automation and customization. GPT4All is ideal for individuals and businesses seeking private, offline access to powerful LLMs. -
2
MLC LLM
Universal LLM Deployment Engine with ML Compilation
MLC LLM is a machine learning compiler and deployment framework designed to enable efficient execution of large language models across a wide range of hardware platforms. The project focuses on compiling models into optimized runtimes that can run natively on devices such as GPUs, mobile processors, browsers, and edge hardware. By leveraging machine learning compilation techniques, mlc-llm produces high-performance inference engines that maintain consistent APIs across platforms. The system supports deployment on environments including Linux, macOS, Windows, iOS, Android, and web browsers while utilizing different acceleration technologies such as CUDA, Vulkan, Metal, and WebGPU. ... -
3
AirLLM
AirLLM 70B inference with single 4GB GPU
AirLLM is an open source Python library that enables extremely large language models to run on consumer hardware with very limited GPU memory. The project addresses one of the main barriers to local LLM experimentation by introducing a memory-efficient inference technique that loads model layers sequentially rather than storing the entire model in GPU memory. This layer-wise inference approach allows models with tens of billions of parameters to run on devices with only a few gigabytes of VRAM. ... -
4
Parallax
Parallax is a distributed model serving framework
...Parallax divides model layers across different nodes and dynamically coordinates them to form a complete inference pipeline. A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. This scheduling system optimizes latency, throughput, and hardware utilization even when nodes have different computational capabilities. The platform also supports model sharding and pipeline parallelism, allowing very large models to run across distributed resources. -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
5
GPUStack
Performance-optimized AI inference on your GPUs
GPUStack is an open-source GPU cluster management platform designed to simplify the deployment and operation of artificial intelligence models across heterogeneous hardware environments. The system aggregates GPU resources from multiple machines into a unified cluster so developers and administrators can run large language models and other AI workloads efficiently across distributed infrastructure. Instead of requiring complex orchestration systems such as Kubernetes, GPUStack provides a lightweight environment that automatically selects appropriate inference engines, configures deployment parameters, and schedules workloads across available GPUs. ... -
6
FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs
FastDeploy is an open-source inference and deployment toolkit designed to simplify the process of running and serving deep learning models across a wide range of hardware platforms. Developed within the PaddlePaddle ecosystem, the toolkit focuses on providing high-performance deployment capabilities for modern AI models including large language models and vision-language systems. The platform enables developers to deploy trained models quickly using optimized inference pipelines that support GPUs, specialized AI accelerators, and other hardware architectures. ... -
7
bitsandbytes
Accessible large language models via k-bit quantization for PyTorch
...Built primarily for the PyTorch ecosystem, the library introduces advanced quantization techniques that allow models to operate using reduced numerical precision while maintaining high accuracy. These optimizations enable large language models and other deep learning architectures to run on hardware with limited memory resources, including consumer-grade GPUs. The project includes specialized optimizers and quantized matrix operations that significantly reduce the memory footprint of training and inference workloads. By lowering the hardware requirements needed to work with large models, bitsandbytes helps make modern AI development more accessible to researchers and engineers. ... -
8
Intel LLM Library for PyTorch
Accelerate local LLM inference and finetuning
Intel LLM Library for PyTorch is an open-source acceleration library developed to optimize large language model inference and fine-tuning on Intel hardware platforms. Built as an extension of the PyTorch ecosystem, the library enables developers to run modern transformer models efficiently on Intel CPUs, GPUs, and specialized AI accelerators. The framework provides hardware-aware optimizations and low-precision computation techniques that significantly improve the performance of large language models while reducing memory consumption. ... -
9
Phi-3-MLX
Phi-3.5 for Mac: Locally-run Vision and Language Models
Phi-3-Vision-MLX is an Apple MLX (machine learning on Apple silicon) implementation of Phi-3 Vision, a lightweight multi-modal model designed for vision and language tasks. It focuses on running vision-language AI efficiently on Apple hardware like M1 and M2 chips. -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
10
whichllm
Find the local LLM that actually runs and performs best
whichllm is a command-line tool for finding local large language models that can realistically run on a user’s hardware. It detects the machine’s available resources, including GPU, CPU, memory, and storage, then recommends models based on practical fit rather than parameter count alone. The project is useful for users who are unsure which local LLM will perform well on their system. It focuses on real, recency-aware benchmarks so recommendations better reflect current model performance. whichllm is especially helpful for developers, AI hobbyists, and researchers comparing local inference options across NVIDIA, AMD, Apple Silicon, and CPU-only environments. ... -
11
llmware
Unified framework for building enterprise RAG pipelines
...The system supports a wide range of inference backends including PyTorch, OpenVINO, ONNX Runtime, and other optimized runtimes, allowing developers to choose the most efficient execution environment for their hardware. -
12
Chitu
High-performance inference framework for large language models
Chitu is a high-performance inference engine designed to deploy and run large language models efficiently in production environments. The framework focuses on improving efficiency, flexibility, and scalability for organizations that need to run LLM inference workloads across different hardware platforms. It supports heterogeneous computing environments, including CPUs, GPUs, and various specialized AI accelerators, allowing models to run across a wide range of infrastructure configurations. Chitu is designed to scale from small single-machine deployments to large distributed clusters that handle high volumes of concurrent inference requests. ... -
13
handy-ollama
Implement CPU from scratch and play with large model deployments
handy-ollama is an open-source educational project designed to help developers and AI enthusiasts learn how to deploy and run large language models locally using the Ollama platform. The repository serves as a structured tutorial that explains how to install, configure, and use Ollama to run modern language models on personal hardware without requiring advanced infrastructure. A key focus of the project is enabling users to run large models even without GPUs by leveraging optimized CPU-based inference pipelines. The project includes step-by-step guides that walk learners through tasks such as installing Ollama, managing local models, calling model APIs, and building simple AI applications on top of locally hosted models. ... -
14
mergekit
Tools for merging pretrained large language models
...This approach allows researchers to combine specialized models into a more versatile system capable of performing multiple tasks. mergekit implements a variety of merging algorithms and strategies that control how model parameters are blended together during the merging process. The library is designed to operate efficiently even in environments with limited hardware resources by using memory-efficient processing methods that can run entirely on CPUs. It also provides configuration-driven workflows that allow users to experiment with different merging strategies without modifying source code. -
15
Train LLM From Scratch
A straightforward method for training your LLM
...The repository walks through the process from downloading data to generating text with a trained model. It supports training smaller or larger models, including million- and billion-parameter configurations depending on available hardware. A major goal is accessibility, since the author frames it as possible to train models using a single GPU. It is most useful for learners, researchers, and developers who want practical exposure to LLM internals. -
16
OpenDAN
OpenDAN is an open source Personal AI OS
OpenDAN is an open-source Personal AI OS , that consolidates various AI modules in one place for your personal use. The goal of OpenDAN (Open and Do Anything Now with AI) is to create a Personal AI OS , which provides a runtime environment for various Al modules as well as protocols for interoperability between them. With OpenDAN, users can securely collaborate with various AI modules using their private data to create powerful personal AI agents, such as butlers, lawyers, doctors, teachers,... -
17
ChatGLM-6B
ChatGLM-6B: An Open Bilingual Dialogue Language Model
...The project provides inference code, demos (command line, web, API), quantization support for lower memory deployment, and tools for finetuning (e.g., via P-Tuning v2). It is optimized for dialogue and question answering with a balance between performance and deployability in consumer hardware settings. Support for quantized inference (INT4, INT8) to reduce GPU memory requirements. Automatic mode switching between precision/memory tradeoffs (full/quantized). -
18
xiaogpt
Play ChatGPT and other LLM with Xiaomi AI Speaker
...The tool is aimed at hobbyists and technical users who want to extend smart speakers with more flexible AI behavior. It is especially useful for experimenting with voice-controlled assistants, home automation ideas, and custom LLM interactions through existing Xiaomi hardware. -
19
OpenPlanter
Language-model investigation agent with a terminal UI
...The system is structured to support experimentation and customization, making it suitable for both research and DIY agriculture projects. With its modular Python codebase, users can adapt the platform for different plant types, hardware setups, or automation strategies. Overall, OpenPlanter aims to simplify the creation of programmable, data-driven plant care systems. -
20
SimpleLLM
950 line, minimal, extensible LLM inference engine built from scratch
...Designed to run efficiently on high-end GPUs like NVIDIA H100 with support for models such as OpenAI/gpt-oss-120b, Simple-LLM implements continuous batching and event-driven inference loops to maximize hardware utilization and throughput. Its straightforward code structure allows anyone experimenting with custom kernels, new batching strategies, or inference optimizations to trace execution from input to output with minimal cognitive overhead. -
21
PEFT
State-of-the-art Parameter-Efficient Fine-Tuning
Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of pre-trained language models (PLMs) to various downstream applications without fine-tuning all the model's parameters. Fine-tuning large-scale PLMs is often prohibitively costly. In this regard, PEFT methods only fine-tune a small number of (extra) model parameters, thereby greatly decreasing the computational and storage costs. Recent State-of-the-Art PEFT techniques achieve performance comparable to that of full... -
22
how-to-optim-algorithm-in-cuda
How to optimize some algorithm in cuda
...Instead of presenting only theoretical explanations, the repository includes hand-written CUDA implementations of fundamental operations such as reductions, element-wise computations, softmax, and attention mechanisms. These examples show how different optimization techniques influence performance on modern GPU hardware and allow readers to experiment with real implementations. The repository also contains extensive learning notes that summarize CUDA programming concepts, GPU architecture details, and performance engineering strategies. -
23
llm.c
LLM training in simple, raw C/CUDA
...Its compact design makes it easy to trace execution, profile hotspots, and understand the cost of each operation. Portability is a goal: it aims to compile with common toolchains and run on modest hardware for small experiments. Rather than delivering a production-grade stack, it serves as a reference and learning scaffold for people who want to “see the metal” behind LLMs. -
24
MaxText
A simple, performant and scalable Jax LLM
...The project acts as both a reference implementation and a practical training library that demonstrates best practices for building and scaling transformer-based language models on modern accelerator hardware. It is optimized to run efficiently on Google Cloud TPUs and GPUs, enabling researchers and engineers to train models ranging from small experiments to extremely large distributed workloads. The framework focuses on simplicity while still supporting advanced techniques such as model sharding, distributed computation, and high-throughput training pipelines. ... -
25
MatMul-Free LM
Implementation for MatMul-free LM
...Since matrix multiplication is one of the most computationally expensive components of modern language models, the project explores alternative computational strategies that reduce hardware requirements while maintaining comparable performance. The architecture relies on quantization-aware training and lightweight operations to replace conventional dense matrix multiplications with more efficient alternatives. These optimizations can significantly reduce memory consumption and potentially improve computational efficiency during both training and inference. ...
