Search Results for "inference engine" - Page 3
Sort By:
Showing 92 open source projects for "inference engine"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
PasteGuard
Masks sensitive data and secrets before they reach AI
...PasteGuard supports two primary modes: mask mode, which anonymizes data and still uses external APIs; and route mode, which forwards sensitive requests to a local LLM inference engine while sending the rest to the cloud. It can be self-hosted via Docker, works with a wide range of SDKs and tools, and includes a browser extension for automatic protection in everyday AI chats. -
2
Transcoder
Hardware-accelerated video transcoding using Android MediaCodec APIs
Transcoder by DeepMedia is an AI-powered video-to-video speech translation engine that enables fully automated multilingual dubbing. Unlike traditional speech translation systems that rely on multi-stage pipelines, Transcoder directly translates one speaker’s video into another language while preserving facial expressions, lip-sync, and vocal identity. Designed for real-time use and production-grade pipelines, Transcoder combines advanced deep learning models with GPU acceleration to deliver... -
3
node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama
node-llama-cpp is a JavaScript and Node.js binding that allows developers to run large language models locally using the high-performance inference engine provided by llama.cpp. The library enables applications built with Node.js to interact directly with local LLM models without requiring a remote API or external service. By using native bindings and optimized model execution, the framework allows developers to integrate advanced language model capabilities into desktop applications, server software, and command-line tools. ... -
4
ClearScript
A library for adding scripting to .NET applications
ClearScript is a library that makes it easy to add scripting to your .NET applications. It currently supports JavaScript (via V8 and JScript) and VBScript. Simple usage; create a script engine, add your objects and/or types, run scripts. Support for several script engines: Google's V8, Microsoft's JScript and VBScript. Exposed resources require no modification, decoration, or special coding of any kind. Scripts get simple access to most of the features of exposed objects and types. Full support for generic types and methods, including C#-like type inference and explicit type arguments. ... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
llama2.c
Inference Llama 2 in one file of pure C
llama2.c is a minimalist implementation of the Llama 2 language model architecture designed to run entirely in pure C. Created by Andrej Karpathy, this project offers an educational and lightweight framework for performing inference on small Llama 2 models without external dependencies. It provides a full training and inference pipeline: models can be trained in PyTorch and later executed using a concise 700-line C program (run.c). While it can technically load Meta’s official Llama 2... -
6
Secret Llama
Fully private LLM chatbot that runs entirely with a browser
Secret Llama is a privacy-first large-language-model chatbot that runs entirely inside your web browser, meaning no server is required and your conversation data never leaves your device. It focuses on open-source model support, letting you load families like Llama and Mistral directly in the client for fully local inference. Because everything happens in-browser, it can work offline once models are cached, which is helpful for air-gapped environments or travel. The interface mirrors the modern chat UX you’d expect—streaming responses, markdown, and a clean layout—so there’s no usability tradeoff to gain privacy. Under the hood it uses a web-native inference engine to accelerate model execution with GPU/WebGPU when available, keeping responses responsive even without a backend. ... -
7
OnnxStream
Lightweight inference library for ONNX files, written in C++
The challenge is to run Stable Diffusion 1.5, which includes a large transformer model with almost 1 billion parameters, on a Raspberry Pi Zero 2, which is a microcomputer with 512MB of RAM, without adding more swap space and without offloading intermediate results on disk. The recommended minimum RAM/VRAM for Stable Diffusion 1.5 is typically 8GB. Generally, major machine learning frameworks and libraries are focused on minimizing inference latency and/or maximizing throughput, all of which at the cost of RAM usage. So I decided to write a super small and hackable inference library specifically focused on minimizing memory consumption: OnnxStream. OnnxStream is based on the idea of decoupling the inference engine from the component responsible for providing the model weights, which is a class derived from WeightsProvider. ... -
8
Llama 2 Everywhere (L2E)
Llama 2 Everywhere (L2E)
Llama 2 Everywhere (L2E) is an open-source implementation of the LLaMA-2 large language model architecture designed to demonstrate how transformer-based language models can be executed with extremely minimal code. The project focuses on simplicity and educational clarity by implementing inference for LLaMA-style models in a compact C program rather than relying on large machine learning frameworks. Developers can train models using a Python training pipeline and then run inference using a... -
9
Lightning Bolts
Toolbox of models, callbacks, and datasets for AI/ML researchers
Bolts package provides a variety of components to extend PyTorch Lightning, such as callbacks & datasets, for applied research and production. Torch ORT converts your model into an optimized ONNX graph, speeding up training & inference when using NVIDIA or AMD GPUs. We can introduce sparsity during fine-tuning with SparseML, which ultimately allows us to leverage the DeepSparse engine to see performance improvements at inference time. -
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
10
FrankMocap
A Strong and Easy-to-use Single View 3D Hand+Body Pose Estimator
FrankMocap is a monocular 3D human capture system that estimates body, hand, and optionally face pose from a single RGB image or video. It regresses parametric human models (e.g., SMPL/SMPL-X) directly, producing temporally stable meshes and joint angles suitable for animation or analytics. The pipeline couples a robust 2D keypoint detector with 3D mesh regression networks and priors that keep results anatomically plausible. It can run frame-by-frame or with temporal smoothing, and includes... -
11
Real-ESRGAN ncnn Vulkan
NCNN implementation of Real-ESRGAN
Real-ESRGAN ncnn Vulkan is an optimized, cross-platform implementation of Real-ESRGAN using the ncnn neural network inference engine and Vulkan for hardware acceleration. Unlike the standard PyTorch-based Real-ESRGAN code, this variant is written in C/C++ and designed to run efficiently on many platforms (including Windows, Linux, and possibly Android) without requiring heavy frameworks like CUDA or Python. It provides command-line tools for upscaling images with selected models, allowing users to specify input/output paths, scaling factors, tile sizes, and model names from a compressed model set, which is particularly helpful for larger images or automated workflows. ... -
12
Yoha
A practical hand tracking engine
Yoha is a browser-based hand tracking engine designed to enable real-time gesture recognition and interaction using standard webcams, making it accessible for web applications without specialized hardware. Built using JavaScript and TensorFlow.js, it runs directly in the browser and performs inference on-device, eliminating the need for server-side processing. The engine is capable of detecting 21 two-dimensional hand landmarks, allowing developers to build applications that respond to gestures such as pinching or forming a fist. ... -
13
MACE
Deep learning inference framework optimized for mobile platforms
Mobile AI Compute Engine (or MACE for short) is a deep learning inference framework optimized for mobile heterogeneous computing on Android, iOS, Linux and Windows devices. Runtime is optimized with NEON, OpenCL and Hexagon, and Winograd algorithm is introduced to speed up convolution operations. The initialization is also optimized to be faster. Chip-dependent power options like big.LITTLE scheduling, Adreno GPU hints are included as advanced APIs. -
14
An automatic knowledge inference engine. Given a set of statements it can derive actions and other statements from the the set of assumptions.
-
15
TensorFlowTTS
Real-Time State-of-the-art Speech Synthesis for Tensorflow 2
TensorFlowTTS is a state-of-the-art, open-source speech synthesis library built on TensorFlow 2. It offers a variety of architectures for text-to-speech, including classic and modern models such as Tacotron‑2, FastSpeech / FastSpeech2, and neural vocoders like MelGAN and Multiband‑MelGAN. Because it’s based on TensorFlow 2, it can leverage optimizations such as fake-quantization aware training and pruning — which allow models to run faster than real time and to be deployable on mobile or... -
16
Minkowski Engine
Auto-diff neural network library for high-dimensional sparse tensors
The Minkowski Engine is an auto-differentiation library for sparse tensors. It supports all standard neural network layers such as convolution, pooling, unspooling, and broadcasting operations for sparse tensors. The Minkowski Engine supports various functions that can be built on a sparse tensor. We list a few popular network architectures and applications here. -
17
DeepSpeech
Open source embedded speech-to-text engine
...If you want to use the pre-trained English model for performing speech-to-text, you can download it (along with other important inference material) from the DeepSpeech releases page. -
18
PyText
A natural language modeling framework based on PyTorch
...It achieves this by providing simple and extensible interfaces and abstractions for model components, and by using PyTorch’s capabilities of exporting models for inference via the optimized Caffe2 execution engine. We use PyText at Facebook to iterate quickly on new modeling ideas and then seamlessly ship them at scale. Distributed-training support built on the new C10d backend in PyTorch 1.0. Mixed precision training support through APEX (trains faster with less GPU memory on NVIDIA Tensor Cores). ... -
19
survol
RDF-based framework monitoring business systems activity
...all communicating with each other, manipulating your data, and whose software architecture has become, with time, complicated, difficult to understand, and undocumented. Data are aggregated with an RDF inference engine, creating a global vision of the business information processing. -
20
DarkForestGo
DarkForest, the Facebook Go engine
darkforestGo is an early deep-reinforcement-learning Go engine that combined a convolutional policy/value network with Monte Carlo Tree Search (MCTS) to play the full 19×19 game at a strong amateur level. The system couples fast GPU policy inference with CPU or GPU-assisted tree search so priors from the network guide exploration while search refines local tactics. Training pipelines mix supervised learning from human professional games and self-play fine-tuning, allowing the model to learn opening patterns and endgame tactics beyond simple pattern libraries. ... -
21
heartdroid
HeaRTDroid is a rule inference engine for Android and desktop
Key features are: Inference support for XTT2 rule representation that is based on the Attributive Logic with Set Values and its textual representation called HMR+. Pure Java implementation that allows for integration with any Java code, including Android applications. Integration with AWARE framework in a form ofplugin. Callbacks mechanism based on Java reflection, that allows for easy integration with other systems. Uncertainty management mechanism based on modified certainty factors... -
22
CTL-RP
CTL-RP is a theorem prover for Computation Tree Logic (CTL)
...Computation Tree Logic (CTL) is a branching-time temporal logic. CTL-RP is a resolution based theorem prover for CTL, which utilises a first-order theorem prover, SPASS, as a core engine for inference. Please see the following link for more details. http://cueb.science/web/software/ (if you are inside China.) http://ctlrp.sourceforge.net (if you are not inside China.) -
23
fusie
Fuzzy scala inference engine
Rewriting from Java to Scala and improving my university project on course Artifical Intelligence Systems. About project in russian: http://brunneng.blogspot.com/2013/06/fuzzy-scala-inference-engine-fusie.html Can be used from maven: <dependency> <groupId>net.sf.brunneng.fusie</groupId> <artifactId>fusie</artifactId> <version>0.8.1</version> </dependency> -
24
IRIS is an extensible reasoning engine for Datalog extended with function symbols, unsafe rules, negation, locally stratified or non-stratified programs, XML schema data types and a comprehensive and extensible set of built-in predicates.
-
25
Aliza Gaming API
An extensible development framework for roleplay games.
AlizaGameAPI is a robust, open-source Java-based framework designed to streamline and enhance the development of 2D and 3D games. It offers a comprehensive set of tools, utilities, and libraries, empowering developers to create immersive and dynamic gaming experiences with ease. Key Features: Modular Architecture, Rich Graphics and UI Components, Comprehensive Game Logic and Character Management, Environment and World-Building Tools, Statistical and Mathematical Utilities, Enhanced...
