Search Results for "inference"
Sort By:
Showing 17 open source projects for "inference"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
bitnet.cpp
Official inference framework for 1-bit LLMs
bitnet.cpp is the official open-source inference framework and ecosystem designed to enable ultra-efficient execution of 1-bit large language models (LLMs), which quantize most model parameters to ternary values (-1, 0, +1) while maintaining competitive performance with full-precision counterparts. At its core is bitnet.cpp, a highly optimized C++ backend that supports fast, low-memory inference on both CPUs and GPUs, enabling models such as BitNet b1.58 to run without requiring enormous compute infrastructure. ... -
2
BentoML
Unified Model Serving Framework
...Standard .bento format for packaging code, models and dependencies for easy versioning and deployment. Integrate with any training pipeline or ML experimentation platform. Parallelize compute-intense model inference workloads to scale separately from the serving logic. Adaptive batching dynamically groups inference requests for optimal performance. Orchestrate distributed inference graph with multiple models via Yatai on Kubernetes. Easily configure CUDA dependencies for running inference with GPU. Automatically generate docker images for production deployment. -
3
NNCF
Neural Network Compression Framework for enhanced OpenVINO
NNCF (Neural Network Compression Framework) is an optimization toolkit for deep learning models, designed to apply quantization, pruning, and other techniques to improve inference efficiency. -
4
Seldon Core
An MLOps framework to package, deploy, monitor and manage models
The de facto standard open-source platform for rapidly deploying machine learning models on Kubernetes. Seldon Core, our open-source framework, makes it easier and faster to deploy your machine learning models and experiments at scale on Kubernetes. Seldon Core serves models built in any open-source or commercial model building framework. You can make use of powerful Kubernetes features like custom resource definitions to manage model graphs. And then connect your continuous integration and... -
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
5
Superduper
Superduper: Integrate AI models and machine learning workflows
Superduper is a Python-based framework for building end-2-end AI-data workflows and applications on your own data, integrating with major databases. It supports the latest technologies and techniques, including LLMs, vector-search, RAG, and multimodality as well as classical AI and ML paradigms. Developers may leverage Superduper by building compositional and declarative objects that out-source the details of deployment, orchestration versioning, and more to the Superduper engine. This... -
6
Meridian
Meridian is an MMM framework
Meridian is a comprehensive, open source marketing mix modeling (MMM) framework developed by Google to help advertisers analyze and optimize the impact of their marketing investments. Built on Bayesian causal inference principles, Meridian enables organizations to evaluate how different marketing channels influence key performance indicators (KPIs) such as revenue or conversions while accounting for external factors like seasonality or economic trends. The framework provides a robust foundation for constructing in-house MMM pipelines capable of handling both national and geo-level data, with built-in support for calibration using experimental data or prior knowledge. ... -
7
tinygrad
Deep learning framework
This may not be the best deep learning framework, but it is a deep learning framework. Due to its extreme simplicity, it aims to be the easiest framework to add new accelerators to, with support for both inference and training. If XLA is CISC, tinygrad is RISC. -
8
Ray
A unified framework for scalable computing
Modern workloads like deep learning and hyperparameter tuning are compute-intensive and require distributed or parallel execution. Ray makes it effortless to parallelize single machine code — go from a single CPU to multi-core, multi-GPU or multi-node with minimal code changes. Accelerate your PyTorch and Tensorflow workload with a more resource-efficient and flexible distributed execution framework powered by Ray. Accelerate your hyperparameter search workloads with Ray Tune. Find the best... -
9
pep484 stubs for Django
PEP-484 stubs for Django
This package contains type stubs and a custom mypy plugin to provide more precise static types and type inference for Django framework. Django uses some Python "magic" that makes having precise types for some code patterns problematic. This is why we need this project. The final goal is to be able to get precise types for the most common patterns. We are independent from Django at the moment. There's a proposal to merge our project into the Django itself. -
Add Two Lines of Code. Get Full APM.Works out of the box for Rails, Django, Express, Phoenix, and more. Monitoring exceptions and performance in no time.
-
10
TorchQuantum
A PyTorch-based framework for Quantum Classical Simulation
A PyTorch-based framework for Quantum Classical Simulation, Quantum Machine Learning, Quantum Neural Networks, Parameterized Quantum Circuits with support for easy deployments on real quantum computers. Researchers on quantum algorithm design, parameterized quantum circuit training, quantum optimal control, quantum machine learning, and quantum neural networks. Dynamic computation graph, automatic gradient computation, fast GPU support, batch model terrorized processing. -
11
MMDeploy
OpenMMLab Model Deployment Framework
...Models can be exported and run in several backends, and more will be compatible. All kinds of modules in the SDK can be extended, such as Transform for image processing, Net for Neural Network inference, Module for postprocessing and so on. Install and build your target backend. ONNX Runtime is a cross-platform inference and training accelerator compatible with many popular ML/DNN frameworks. Please read getting_started for the basic usage of MMDeploy. -
12
pipeless
A computer vision framework to create and deploy apps in minutes
...You provide some functions that are executed for new video frames and Pipeless takes care of everything else. You can easily use industry-standard models, such as YOLO, or load your custom model in one of the supported inference runtimes. Pipeless ships some of the most popular inference runtimes, such as the ONNX Runtime, allowing you to run inference with high performance on CPU or GPU out-of-the-box. You can deploy your Pipeless application with a single command to edge and IoT devices or the cloud. -
13
towhee
Framework that is dedicated to making neural data processing
...From images to text to 3D molecular structures, Towhee supports data transformation for nearly 20 different unstructured data modalities. We provide end-to-end pipeline optimizations, covering everything from data decoding/encoding, to model inference, making your pipeline execution 10x faster. Towhee provides out-of-the-box integration with your favorite libraries, tools, and frameworks, making development quick and easy. Towhee includes a pythonic method-chaining API for describing custom data processing pipelines. We also support schemas, making processing unstructured data as easy as handling tabular data. -
14
MMdnn
Tools to help users inter-operate among deep learning frameworks
...We implement a universal converter to convert DL models between frameworks, which means you can train a model with one framework and deploy it with another. During the model conversion, we generate some code snippets to simplify later retraining or inference. We provide a model collection to help you find some popular models. We provide a model visualizer to display the network architecture more intuitively. We provide some guidelines to help you deploy DL models to another hardware platform. -
15
PyText
A natural language modeling framework based on PyTorch
...It achieves this by providing simple and extensible interfaces and abstractions for model components, and by using PyTorch’s capabilities of exporting models for inference via the optimized Caffe2 execution engine. We use PyText at Facebook to iterate quickly on new modeling ideas and then seamlessly ship them at scale. Distributed-training support built on the new C10d backend in PyTorch 1.0. Mixed precision training support through APEX (trains faster with less GPU memory on NVIDIA Tensor Cores). ... -
16
CrypTen
A framework for Privacy Preserving Machine Learning
...The framework supports both encryption and decryption of tensors and operations such as addition and multiplication over encrypted values. Although not yet production-ready, CrypTen focuses on advancing real-world secure ML applications, such as training and inference over private datasets, without exposing sensitive data. -
17
seq2seq
A general-purpose encoder-decoder framework for Tensorflow
seq2seq is an early, influential TensorFlow reference implementation for sequence-to-sequence learning with attention, covering tasks like neural machine translation, summarization, and dialogue. It packaged encoders, decoders, attention mechanisms, and beam search into a modular training and inference framework. The codebase showcased best practices for batching, bucketing by sequence length, and handling variable-length sequences efficiently on GPUs. Researchers used it as a baseline to reproduce classic results and to prototype new attention variants and training tricks. It also offered scripts for data preprocessing, evaluation, and exporting models for serving. ...
- Previous
- You're on page 1
- Next
