Search Results for "inference" - Page 9
Sort By:
Showing 897 open source projects for "inference"
View related business solutions-
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
DeepProve
Framework to prove inference of ML models blazingly fast
DeepProve is an advanced cryptographic framework designed to verify machine learning model inference using zero-knowledge proofs, enabling trustless validation of AI computations without exposing underlying data. The project focuses on zkML, a rapidly emerging field that combines machine learning with zero-knowledge cryptography to ensure both privacy and correctness. It supports neural network architectures such as multilayer perceptrons and convolutional neural networks, allowing developers to prove that a model’s output is correct without revealing inputs or model details. deep-prove leverages advanced proof systems such as sumcheck protocols and GKR-based constructions to achieve significantly faster proving times compared to earlier approaches. ... -
2
Bespoke Curator
Synthetic data curation for post-training and data extraction
...Curator includes tools for monitoring data generation processes and managing dataset quality while large batches of examples are being created. The framework also integrates with multiple inference systems and APIs, allowing users to generate data using different model providers or open-source inference engines. -
3
SageAttention
NeurIPS2025 Spotlight] Quantized Attention
...The system achieves this by using low-precision numerical formats such as INT4, FP8, or INT8 to represent key matrices within the attention computation. These optimizations allow models to perform matrix operations faster and consume less memory during inference. SageAttention is designed to function as a plug-and-play replacement for standard attention implementations, enabling developers to accelerate existing models without modifying their architecture. -
4
pytype
A static type analyzer for Python code
...Documentation and a user guide detail configuration, supported features, and how inference interacts with modern Python idioms. -
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
Mosec
A high-performance ML model serving framework, offers dynamic batching
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API. -
6
FLUX.2
Official inference repo for FLUX.2 models
FLUX.2 is a state-of-the-art open-weight image generation and editing model released by Black Forest Labs aimed at bridging the gap between research-grade capabilities and production-ready workflows. The model offers both text-to-image generation and powerful image editing, including editing of multiple reference images, with fidelity, consistency, and realism that push the limits of what open-source generative models have achieved. It supports high-resolution output (up to ~4 megapixels),... -
7
Watermark-Removal
Machine learning image inpainting task that removes watermarks
...Through these techniques, the model learns to identify regions of the image affected by the watermark and generate realistic replacements for the missing visual information. The repository contains code for preprocessing images, training the model, and running inference on images to automatically remove watermark artifacts. -
8
Embedding Studio
Framework which allows you transform your Vector Database
Embedding Studio is a framework that transforms vector databases into feature-rich search engines. It leverages embeddings to enhance search capabilities, enabling more accurate and context-aware retrieval of information. Embedding Studio supports various data types and integrates seamlessly with existing databases, providing tools for fine-tuning and optimizing embeddings to suit specific application needs. -
9
Secret Llama
Fully private LLM chatbot that runs entirely with a browser
Secret Llama is a privacy-first large-language-model chatbot that runs entirely inside your web browser, meaning no server is required and your conversation data never leaves your device. It focuses on open-source model support, letting you load families like Llama and Mistral directly in the client for fully local inference. Because everything happens in-browser, it can work offline once models are cached, which is helpful for air-gapped environments or travel. The interface mirrors the modern chat UX you’d expect—streaming responses, markdown, and a clean layout—so there’s no usability tradeoff to gain privacy. Under the hood it uses a web-native inference engine to accelerate model execution with GPU/WebGPU when available, keeping responses responsive even without a backend. ... -
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
10
Pytorch-toolbelt
PyTorch extensions for fast R&D prototyping and Kaggle farming
...Modules: CoordConv, SCSE, Hypercolumn, Depthwise separable convolution and more. GPU-friendly test-time augmentation TTA for segmentation and classification. GPU-friendly inference on huge (5000x5000) images. Every-day common routines (fix/restore random seed, filesystem utils, metrics). Losses: BinaryFocalLoss, Focal, ReducedFocal, Lovasz, Jaccard and Dice losses, Wing Loss and more. Extras for Catalyst library (Visualization of batch predictions, additional metrics). By design, both encoder and decoder produces a list of tensors, from fine (high-resolution, indexed 0) to coarse (low-resolution) feature maps. ... -
11
Pandas Profiling
Create HTML profiling reports from pandas DataFrame objects
pandas-profiling generates profile reports from a pandas DataFrame. The pandas df.describe() function is handy yet a little basic for exploratory data analysis. pandas-profiling extends pandas DataFrame with df.profile_report(), which automatically generates a standardized univariate and multivariate report for data understanding. High correlation warnings, based on different correlation metrics (Spearman, Pearson, Kendall, Cramér’s V, Phik). Most common categories (uppercase, lowercase,... -
12
TurboQuant PyTorch
From-scratch PyTorch implementation of Google's TurboQuant
TurboQuant PyTorch is a specialized deep learning optimization framework designed to accelerate neural network inference and training through advanced quantization techniques within the PyTorch ecosystem. The project focuses on reducing the computational and memory footprint of models by converting floating-point representations into lower-precision formats while preserving performance. It provides tools for experimenting with different quantization strategies, enabling developers to balance accuracy and efficiency depending on their application. ... -
13
Temporal Inference Engine
A real time inference engine for temporal logical specifications
A real time inference engine for temporal logical specifications, which is able to acquire, process and generate any binary or real signal through POSIX IPC, files or UNIX sockets. Specifications of signals and dynamic systems are represented as special graphs and executed in real time, with a predictable sampling time of few milliseconds. Real time signal processing, dynamic system control, state machine modeling and logical property verification are some fields of application of this software. ... -
14
Qwen
The official repo of Qwen chat & pretrained large language model
Qwen is a series of large language models developed by Alibaba Cloud, consisting of various pretrained versions like Qwen-1.8B, Qwen-7B, Qwen-14B, and Qwen-72B. These models, which range from smaller to larger configurations, are designed for a wide range of natural language processing tasks. They are openly available for research and commercial use, with Qwen's code and model weights shared on GitHub. Qwen's capabilities include text generation, comprehension, and conversation, making it a... -
15
SWIFT LLM
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs
...The platform provides a full machine learning pipeline that supports tasks ranging from model pre-training to reinforcement learning alignment techniques. It integrates with popular inference engines such as vLLM and LMDeploy to accelerate deployment and runtime performance. The framework also includes support for many modern training strategies, including preference learning methods and parameter-efficient fine-tuning techniques. ms-swift is designed to work with hundreds of language and multimodal models, providing a unified environment for experimentation and production deployment. -
16
DeepSeek VL
Towards Real-World Vision-Language Understanding
...The repository includes model weights (or pointers to them), evaluation metrics on standard vision + language benchmarks, and configuration or architecture files. It also supports inference tools for forwarding image + prompt through the model to produce text output. DeepSeek-VL is a predecessor to their newer VL2 model, and presumably shares core design philosophy but with earlier scaling, fewer enhancements, or capability tradeoffs. -
17
MIVisionX
Set of comprehensive computer vision & machine intelligence libraries
...AMD MIVisionX delivers highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions along with Convolution Neural Net Model Compiler & Optimizer supporting ONNX, and Khronos NNEF™ exchange formats. The toolkit allows for rapid prototyping and deployment of optimized computer vision and machine learning inference workloads on a wide range of computer hardware, including small embedded x86 CPUs, APUs, discrete GPUs, and heterogeneous servers. AMD OpenVX is a highly optimized open-source implementation of the Khronos OpenVX™ 1.3 computer vision specification. It allows for rapid prototyping as well as fast execution on a wide range of computer hardware, including small embedded x86 CPUs and large workstation discrete GPUs. -
18
LLM.swift
LLM.swift is a simple and readable library
LLM.swift is a Swift package that enables developers to run Large Language Models (LLMs) directly on Apple devices, including iOS, macOS, and watchOS. By leveraging Apple's hardware and software optimizations, LLM.swift facilitates on-device natural language processing tasks, ensuring user privacy and reducing latency associated with cloud-based solutions. -
19
Evo 2
Genome modeling and design across all domains of life
...According to the repository, it uses the StripedHyena 2 architecture, was pretrained with Savanna, and was trained autoregressively on the OpenGenome2 dataset containing 8.8 trillion tokens. The codebase is focused on local inference and generation through the Vortex inference stack rather than serving as a full training framework alone, although it also points users to training and fine-tuning resources. It supports multiple ways of working with the model, including forward passes, embeddings, generation workflows, notebooks, hosted APIs, and self-hosted deployment through NVIDIA NIM. -
20
Paddler
Open-source LLM load balancer and serving platform for hosting LLMs
...Paddler also includes tools for monitoring, request buffering, and autoscaling integration so that deployments can adapt dynamically to changing workloads. A built-in administrative interface allows developers and operations teams to manage models, observe system performance, and test inference endpoints. -
21
nanochat
The best ChatGPT that $100 can buy
...The repository stitches together every stage of the lifecycle: tokenizer training, pretraining a Transformer on a large web corpus, mid-training on dialogue and multiple-choice tasks, supervised fine-tuning, optional reinforcement learning for alignment, and finally efficient inference with caching. Its north star is approachability and speed: you can boot a fresh GPU box and drive the whole pipeline via a single script, producing a usable chat model in hours and a clear markdown report of what happened. The code is written to be read—concise training loops, transparent configs, and minimal wrappers—so you can audit each step, tweak it, and rerun without getting lost in framework indirection. -
22
Code World Model (CWM)
Research code artifacts for Code World Model (CWM)
...It is explicitly trained on execution traces, action-observation trajectories, and agentic interactions in controlled environments. It has been developed to better capture how code, actions, and state interact over time. The repository provides inference code, reproducibility scripts, prompt guides, and more. It has model cards, utilities, demos, and evaluation artifacts. Inference scripts and utilities for code generation tasks. Evaluation benchmarks on code, mathematics, and reasoning tasks. Demos, serving code, and evaluation pipelines. -
23
Tencent-Hunyuan-Large
Open-source large language model family from Tencent Hunyuan
...It is designed with long-context capabilities, quantization support, and high performance on benchmarks across general reasoning, mathematics, language understanding, and Chinese / multilingual tasks. It aims to provide competitive capability with efficient deployment and inference. FP8 quantization support to reduce memory usage (~50%) while maintaining precision. High benchmarking performance on tasks like MMLU, MATH, CMMLU, C-Eval, etc. -
24
GPT-SoVITS
1 min voice data can also be used to train a good TTS model
GPT‑SoVITS is a state-of-the-art voice conversion and TTS system that enables zero‑shot and few‑shot synthesis based on a short vocal sample (e.g., 5 seconds). It supports cross‑lingual speech synthesis across English, Chinese, Japanese, Korean, Cantonese, and more. It's powered by VITS architecture enhanced for few‑sample adaptation and real‑time usability. -
25
OpenVINO Training Extensions
Trainable models and NN optimization tools
OpenVINO™ Training Extensions provide a convenient environment to train Deep Learning models and convert them using the OpenVINO™ toolkit for optimized inference. When ote_cli is installed in the virtual environment, you can use the ote command line interface to perform various actions for templates related to the chosen task type, such as running, training, evaluating, exporting, etc. ote train trains a model (a particular model template) on a dataset and saves results in two files. ote optimize optimizes a pre-trained model using NNCF or POT depending on the model format. ...
