Search Results for "loading"
Sort By:
Showing 27 open source projects for "loading"
View related business solutions-
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
1
DALI
A GPU-accelerated library containing highly optimized building blocks
The NVIDIA Data Loading Library (DALI) is a library for data loading and pre-processing to accelerate deep learning applications. It provides a collection of highly optimized building blocks for loading and processing image, video and audio data. It can be used as a portable drop-in replacement for built-in data loaders and data iterators in popular deep learning frameworks. -
2
High-Level Training Utilities Pytorch
High-level training, data augmentation, and utilities for Pytorch
...This package provides a few things. A high-level module for Keras-like training with callbacks, constraints, and regularizers. Comprehensive data augmentation, transforms, sampling, and loading. Utility tensor and variable functions so you don't need numpy as often. Have any feature requests? Submit an issue! I'll make it happen. Specifically, any data augmentation, data loading, or sampling functions. ModuleTrainer. The ModuleTrainer class provides a high-level training interface that abstracts away the training loop while providing callbacks, constraints, initializers, regularizers, and more. ... -
3
TorchAudio
Data manipulation and transformation for audio signal processing
The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the same philosophy of providing strong GPU acceleration, having a focus on trainable features through the autograd system, and having consistent style (tensor names and dimension names). Therefore, it is primarily a machine learning library and not a general signal processing library. The benefits of PyTorch can be seen in torchaudio through having all the computations be through PyTorch... -
4
Pedalboard
A Python library for audio
pedalboard is a Python library for working with audio: reading, writing, rendering, adding effects, and more. It supports the most popular audio file formats and a number of common audio effects out of the box and also allows the use of VST3® and Audio Unit formats for loading third-party software instruments and effects. pedalboard was built by Spotify’s Audio Intelligence Lab to enable using studio-quality audio effects from within Python and TensorFlow. Internally at Spotify, pedalboard is used for data augmentation to improve machine learning models and to help power features like Spotify’s AI DJ and AI Voice Translation. pedalboard also helps in the process of content creation, making it possible to add effects to audio without using a Digital Audio Workstation. -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
5
mosaicml composer
Supercharge Your Model Training
...At the center of the project is a highly optimized Trainer abstraction that simplifies the management of training loops, parallelization, metrics, logging, and data loading. The framework is intended for modern workloads that may span anything from a single GPU to very large distributed training environments, which makes it suitable for both experimentation and production-scale development. It includes built-in support for distributed training strategies such as Fully Sharded Data Parallelism and standard Distributed Data Parallel execution, helping teams scale models without having to assemble as much infrastructure by hand. -
6
Datasets
Hub of ready-to-use datasets for ML models
Datasets is a library for easily accessing and sharing datasets, and evaluation metrics for Natural Language Processing (NLP), computer vision, and audio tasks. Load a dataset in a single line of code, and use our powerful data processing methods to quickly get your dataset ready for training in a deep learning model. Backed by the Apache Arrow format, process large datasets with zero-copy reads without any memory constraints for optimal speed and efficiency. We also feature a deep... -
7
TorchIO
Medical imaging toolkit for deep learning
TorchIO is an open-source Python library for efficient loading, preprocessing, augmentation and patch-based sampling of 3D medical images in deep learning, following the design of PyTorch. It includes multiple intensity and spatial transforms for data augmentation and preprocessing. These transforms include typical computer vision operations such as random affine transformations and also domain-specific ones such as simulation of intensity artifacts due to MRI magnetic field inhomogeneity (bias) or k-space motion artifacts. ... -
8
Determined
Determined, deep learning training platform
The fastest and easiest way to build deep learning models. Distributed training without changing your model code. Determined takes care of provisioning machines, networking, data loading, and fault tolerance. Build more accurate models faster with scalable hyperparameter search, seamlessly orchestrated by Determined. Use state-of-the-art algorithms and explore results with our hyperparameter search visualizations. Interpret your experiment results using the Determined UI and TensorBoard, and reproduce experiments with artifact tracking. ... -
9
IVY
The Unified Machine Learning Framework
...For example, an existing TensorFlow model, and some useful functions from both PyTorch and NumPy libraries. Choose any framework for writing your higher-level pipeline, including data loading, distributed training, analytics, logging, visualization etc. Choose any backend framework which should be used under the hood, for running this entire pipeline. Choose the most appropriate device or combination of devices for your needs. DeepMind releases an awesome model on GitHub, written in JAX. We'll use PerceiverIO as an example. ... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
TensorFlow Hub
A library for transfer learning by reusing parts of TensorFlow models
TensorFlow Hub is a repository that provides a library and platform for publishing, discovering, and reusing pre-trained machine learning models built with TensorFlow. The project enables developers to integrate high-quality models into their applications without needing to train them from scratch. Through TensorFlow Hub, researchers and practitioners can share reusable model components such as image classifiers, text embedding models, and object detection networks. These models can be... -
11
OnnxStream
Lightweight inference library for ONNX files, written in C++
...OnnxStream is based on the idea of decoupling the inference engine from the component responsible for providing the model weights, which is a class derived from WeightsProvider. A WeightsProvider specialization can implement any type of loading, caching, and prefetching of the model parameters. -
12
Datapipe
Real-time, incremental ETL library for ML with record-level depend
Datapipe is a real-time, incremental ETL library for Python with record-level dependency tracking. Datapipe is designed to streamline the creation of data processing pipelines. It excels in scenarios where data is continuously changing, requiring pipelines to adapt and process only the modified data efficiently. This library tracks dependencies for each record in the pipeline, ensuring minimal and efficient data processing. -
13
Alink
Alink is the Machine Learning algorithm platform based on Flink
Alink is Alibaba’s scalable machine learning algorithm platform built on Apache Flink, designed for batch and stream data processing. It provides a wide variety of ready-to-use ML algorithms for tasks like classification, regression, clustering, recommendation, and more. Written in Java and Scala, Alink is suitable for enterprise-grade big data applications where performance and scalability are crucial. It supports model training, evaluation, and deployment in real-time environments and... -
14
LLM Applications
A comprehensive guide to building RAG-based LLM applications
LLM Applications is a practical reference repository that demonstrates how to build production-grade applications powered by large language models. The project focuses particularly on Retrieval-Augmented Generation architectures, which combine language models with external knowledge sources to improve accuracy and reliability. It provides step-by-step guidance for constructing systems that ingest documents, split them into chunks, generate embeddings, index them in vector databases, and... -
15
Lightning Flash
Flash enables you to easily configure and run complex AI recipes
Your PyTorch AI Factory, Flash enables you to easily configure and run complex AI recipes for over 15 tasks across 7 data domains. In a nutshell, Flash is the production-grade research framework you always dreamed of but didn't have time to build. All data loading in Flash is performed via a from_* classmethod on a DataModule. Which DataModule to use and which from_* methods are available depends on the task you want to perform. For example, for image segmentation where your data is stored in folders, you would use the from_folders method of the SemanticSegmentationData class. Our tasks come loaded with pre-trained backbones and (where applicable) heads. ... -
16
sense2vec
Contextually-keyed word vectors
sense2vec (Trask et. al, 2015) is a nice twist on word2vec that lets you learn more interesting and detailed word vectors. This library is a simple Python implementation for loading, querying and training sense2vec models. For more details, check out our blog post. To explore the semantic similarities across all Reddit comments of 2015 and 2019, see the interactive demo. -
17
FFCV
Fast Forward Computer Vision (and other ML workloads!)
ffcv is a drop-in data loading system that dramatically increases data throughput in model training. From gridding to benchmarking to fast research iteration, there are many reasons to want faster model training. Below we present premade codebases for training on ImageNet and CIFAR, including both (a) extensible codebases and (b) numerous premade training configurations. -
18
Merlion
A Machine Learning Framework for Time Series Intelligence
Merlion is a Python library for time series intelligence. It provides an end-to-end machine learning framework that includes loading and transforming data, building and training models, post-processing model outputs, and evaluating model performance. It supports various time series learning tasks, including forecasting, anomaly detection, and change point detection for both univariate and multivariate time series. This library aims to provide engineers and researchers a one-stop solution to rapidly develop models for their specific time series needs, and benchmark them across multiple time series datasets. -
19
OpenPrompt
An Open-Source Framework for Prompt-Learning
Prompt-learning is the latest paradigm to adapt pre-trained language models (PLMs) to downstream NLP tasks, which modifies the input text with a textual template and directly uses PLMs to conduct pre-trained tasks. OpenPrompt is a library built upon PyTorch and provides a standard, flexible and extensible framework to deploy the prompt-learning pipeline. OpenPrompt supports loading PLMs directly from huggingface transformers. In the future, we will also support PLMs implemented by other libraries. The template is one of the most important modules in prompt learning, which wraps the original input with textual or soft-encoding sequence. Use the implementations of current prompt-learning approaches.* We have implemented various of prompting methods, including templating, verbalizing and optimization strategies under a unified standard. ... -
20
LayoutParser
A Unified Toolkit for Deep Learning Based Document Image Analysis
With the help of state-of-the-art deep learning models, Layout Parser enables extracting complicated document structures using only several lines of code. This method is also more robust and generalizable as no sophisticated rules are involved in this process. A complete instruction for installing the main Layout Parser library and auxiliary components. Learn how to load DL Layout models and use them for layout detection. The full list of layout models currently available in Layout Parser.... -
21
BlazingSQL
BlazingSQL is a lightweight, GPU accelerated, SQL engine for Python
BlazingSQL is a GPU-accelerated SQL engine built on top of the RAPIDS ecosystem. RAPIDS is based on the Apache Arrow columnar memory format, and cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data. BlazingSQL is a SQL interface for cuDF, with various features to support large-scale data science workflows and enterprise datasets. -
22
fastNLP
fastNLP: A Modularized and Extensible NLP Framework
...A unified Tabular data container simplifies the data preprocessing process. Built-in Loader and Pipe for multiple datasets, eliminating the need for preprocessing code. Various convenient NLP tools, such as Embedding loading (including ELMo and BERT), intermediate data cache, etc.. Provide a variety of neural network components and recurrence models (covering tasks such as Chinese word segmentation, named entity recognition, syntactic analysis, text classification, text matching, metaphor resolution, summarization, etc.). Trainer provides a variety of built-in Callback functions to facilitate experiment recording, exception capture, etc. ... -
23
Machine Learning Homework
Matlab Coding homework for Machine Learning
The Machine-Learning-homework repository by user “Ayatans” is a collection of MATLAB code intended to solve or illustrate assignments in machine learning courses. It includes implementations of standard machine learning algorithms (such as regression, classification, etc.), scripts for data loading and preprocessing, and evaluation routines (e.g. accuracy, error metrics). Because it is structured as homework or practice material, the code is likely intended more for didactic use than for production deployment. It may contain comments, example datasets, and perhaps test scripts. The repository does not seem to be heavily maintained as a software project; rather, it functions as a library of solved problems and educational examples. ... -
24
SINGA
A distributed deep learning platform
...SINGA records the computation graph and applies the backward propagation automatically after forward propagation. The optimization of memory are implemented in the Device class. SINGA supports loading ONNX format models and saving models defined using SINGA APIs into ONNX format, which enables AI developers to use models across different libraries and tools. SINGA supports the time profiling of each of the operators buffered in the graph. Half precision is supported to bring benefits. -
25
Tensorpack
A Neural Net Training Interface on TensorFlow, with focus on speed
...Your training can probably gets faster if written with Tensorpack. Scalable data-parallel multi-GPU / distributed training strategy is off-the-shelf to use. Squeeze the best data loading performance of Python with tensorpack.dataflow. Symbolic programming (e.g. tf.data) does not offer the data processing flexibility needed in research. Tensorpack squeezes the most performance out of pure Python with various auto parallelization strategies. There are too many symbolic function wrappers already. Tensorpack includes only a few common layers. ...
