Search Results for "loading"
Sort By:
Showing 21 open source projects for "loading"
View related business solutions-
One Monitoring Tool for IT, OT and Cloud | Free TrialModern infrastructure spans data centers, cloud platforms and factory floors, and every blind spot between them is a risk. PRTG supports SNMP, WMI, SSH and other standard protocols to monitor IT, OT and hybrid environments through one customizable dashboard. Build the views your team needs, from network health to application performance, without switching tools. Try PRTG free for 30 days now.
-
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
1
Olmo-core
PyTorch building blocks for the OLMo ecosystem
OLMo-core is a Python library from AI2 that provides reusable PyTorch building blocks for training large language models at scale. It underpins the broader OLMo ecosystem and includes components for models, data loading, distributed execution, optimization, checkpointing, evaluation, and generation. Official scripts reproduce training configurations for released OLMo models. The library supports advanced attention backends, mixture-of-experts architectures, float8 training, and fused low-memory operations through optional dependencies. Jobs can run with standard torchrun or AI2’s Beaker infrastructure. ... -
2
Pedalboard
A Python library for audio
pedalboard is a Python library for working with audio: reading, writing, rendering, adding effects, and more. It supports the most popular audio file formats and a number of common audio effects out of the box and also allows the use of VST3® and Audio Unit formats for loading third-party software instruments and effects. pedalboard was built by Spotify’s Audio Intelligence Lab to enable using studio-quality audio effects from within Python and TensorFlow. Internally at Spotify, pedalboard is used for data augmentation to improve machine learning models and to help power features like Spotify’s AI DJ and AI Voice Translation. pedalboard also helps in the process of content creation, making it possible to add effects to audio without using a Digital Audio Workstation. -
3
Determined
Determined, deep learning training platform
The fastest and easiest way to build deep learning models. Distributed training without changing your model code. Determined takes care of provisioning machines, networking, data loading, and fault tolerance. Build more accurate models faster with scalable hyperparameter search, seamlessly orchestrated by Determined. Use state-of-the-art algorithms and explore results with our hyperparameter search visualizations. Interpret your experiment results using the Determined UI and TensorBoard, and reproduce experiments with artifact tracking. ... -
4
TorchAudio
Data manipulation and transformation for audio signal processing
The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the same philosophy of providing strong GPU acceleration, having a focus on trainable features through the autograd system, and having consistent style (tensor names and dimension names). Therefore, it is primarily a machine learning library and not a general signal processing library. The benefits of PyTorch can be seen in torchaudio through having all the computations be through PyTorch... -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
5
mosaicml composer
Supercharge Your Model Training
...At the center of the project is a highly optimized Trainer abstraction that simplifies the management of training loops, parallelization, metrics, logging, and data loading. The framework is intended for modern workloads that may span anything from a single GPU to very large distributed training environments, which makes it suitable for both experimentation and production-scale development. It includes built-in support for distributed training strategies such as Fully Sharded Data Parallelism and standard Distributed Data Parallel execution, helping teams scale models without having to assemble as much infrastructure by hand. -
6
Datasets
Hub of ready-to-use datasets for ML models
Datasets is a library for easily accessing and sharing datasets, and evaluation metrics for Natural Language Processing (NLP), computer vision, and audio tasks. Load a dataset in a single line of code, and use our powerful data processing methods to quickly get your dataset ready for training in a deep learning model. Backed by the Apache Arrow format, process large datasets with zero-copy reads without any memory constraints for optimal speed and efficiency. We also feature a deep... -
7
TorchIO
Medical imaging toolkit for deep learning
TorchIO is an open-source Python library for efficient loading, preprocessing, augmentation and patch-based sampling of 3D medical images in deep learning, following the design of PyTorch. It includes multiple intensity and spatial transforms for data augmentation and preprocessing. These transforms include typical computer vision operations such as random affine transformations and also domain-specific ones such as simulation of intensity artifacts due to MRI magnetic field inhomogeneity (bias) or k-space motion artifacts. ... -
8
IVY
The Unified Machine Learning Framework
...For example, an existing TensorFlow model, and some useful functions from both PyTorch and NumPy libraries. Choose any framework for writing your higher-level pipeline, including data loading, distributed training, analytics, logging, visualization etc. Choose any backend framework which should be used under the hood, for running this entire pipeline. Choose the most appropriate device or combination of devices for your needs. DeepMind releases an awesome model on GitHub, written in JAX. We'll use PerceiverIO as an example. ... -
9
Datapipe
Real-time, incremental ETL library for ML with record-level depend
Datapipe is a real-time, incremental ETL library for Python with record-level dependency tracking. Datapipe is designed to streamline the creation of data processing pipelines. It excels in scenarios where data is continuously changing, requiring pipelines to adapt and process only the modified data efficiently. This library tracks dependencies for each record in the pipeline, ensuring minimal and efficient data processing. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
10
High-Level Training Utilities Pytorch
High-level training, data augmentation, and utilities for Pytorch
...This package provides a few things. A high-level module for Keras-like training with callbacks, constraints, and regularizers. Comprehensive data augmentation, transforms, sampling, and loading. Utility tensor and variable functions so you don't need numpy as often. Have any feature requests? Submit an issue! I'll make it happen. Specifically, any data augmentation, data loading, or sampling functions. ModuleTrainer. The ModuleTrainer class provides a high-level training interface that abstracts away the training loop while providing callbacks, constraints, initializers, regularizers, and more. ... -
11
TensorFlow Hub
A library for transfer learning by reusing parts of TensorFlow models
TensorFlow Hub is a repository that provides a library and platform for publishing, discovering, and reusing pre-trained machine learning models built with TensorFlow. The project enables developers to integrate high-quality models into their applications without needing to train them from scratch. Through TensorFlow Hub, researchers and practitioners can share reusable model components such as image classifiers, text embedding models, and object detection networks. These models can be... -
12
LLM Applications
A comprehensive guide to building RAG-based LLM applications
LLM Applications is a practical reference repository that demonstrates how to build production-grade applications powered by large language models. The project focuses particularly on Retrieval-Augmented Generation architectures, which combine language models with external knowledge sources to improve accuracy and reliability. It provides step-by-step guidance for constructing systems that ingest documents, split them into chunks, generate embeddings, index them in vector databases, and... -
13
Lightning Flash
Flash enables you to easily configure and run complex AI recipes
Your PyTorch AI Factory, Flash enables you to easily configure and run complex AI recipes for over 15 tasks across 7 data domains. In a nutshell, Flash is the production-grade research framework you always dreamed of but didn't have time to build. All data loading in Flash is performed via a from_* classmethod on a DataModule. Which DataModule to use and which from_* methods are available depends on the task you want to perform. For example, for image segmentation where your data is stored in folders, you would use the from_folders method of the SemanticSegmentationData class. Our tasks come loaded with pre-trained backbones and (where applicable) heads. ... -
14
sense2vec
Contextually-keyed word vectors
sense2vec (Trask et. al, 2015) is a nice twist on word2vec that lets you learn more interesting and detailed word vectors. This library is a simple Python implementation for loading, querying and training sense2vec models. For more details, check out our blog post. To explore the semantic similarities across all Reddit comments of 2015 and 2019, see the interactive demo. -
15
FFCV
Fast Forward Computer Vision (and other ML workloads!)
ffcv is a drop-in data loading system that dramatically increases data throughput in model training. From gridding to benchmarking to fast research iteration, there are many reasons to want faster model training. Below we present premade codebases for training on ImageNet and CIFAR, including both (a) extensible codebases and (b) numerous premade training configurations. -
16
Merlion
A Machine Learning Framework for Time Series Intelligence
Merlion is a Python library for time series intelligence. It provides an end-to-end machine learning framework that includes loading and transforming data, building and training models, post-processing model outputs, and evaluating model performance. It supports various time series learning tasks, including forecasting, anomaly detection, and change point detection for both univariate and multivariate time series. This library aims to provide engineers and researchers a one-stop solution to rapidly develop models for their specific time series needs, and benchmark them across multiple time series datasets. -
17
OpenPrompt
An Open-Source Framework for Prompt-Learning
Prompt-learning is the latest paradigm to adapt pre-trained language models (PLMs) to downstream NLP tasks, which modifies the input text with a textual template and directly uses PLMs to conduct pre-trained tasks. OpenPrompt is a library built upon PyTorch and provides a standard, flexible and extensible framework to deploy the prompt-learning pipeline. OpenPrompt supports loading PLMs directly from huggingface transformers. In the future, we will also support PLMs implemented by other libraries. The template is one of the most important modules in prompt learning, which wraps the original input with textual or soft-encoding sequence. Use the implementations of current prompt-learning approaches.* We have implemented various of prompting methods, including templating, verbalizing and optimization strategies under a unified standard. ... -
18
LayoutParser
A Unified Toolkit for Deep Learning Based Document Image Analysis
With the help of state-of-the-art deep learning models, Layout Parser enables extracting complicated document structures using only several lines of code. This method is also more robust and generalizable as no sophisticated rules are involved in this process. A complete instruction for installing the main Layout Parser library and auxiliary components. Learn how to load DL Layout models and use them for layout detection. The full list of layout models currently available in Layout Parser.... -
19
fastNLP
fastNLP: A Modularized and Extensible NLP Framework
...A unified Tabular data container simplifies the data preprocessing process. Built-in Loader and Pipe for multiple datasets, eliminating the need for preprocessing code. Various convenient NLP tools, such as Embedding loading (including ELMo and BERT), intermediate data cache, etc.. Provide a variety of neural network components and recurrence models (covering tasks such as Chinese word segmentation, named entity recognition, syntactic analysis, text classification, text matching, metaphor resolution, summarization, etc.). Trainer provides a variety of built-in Callback functions to facilitate experiment recording, exception capture, etc. ... -
20
Tensorpack
A Neural Net Training Interface on TensorFlow, with focus on speed
...Your training can probably gets faster if written with Tensorpack. Scalable data-parallel multi-GPU / distributed training strategy is off-the-shelf to use. Squeeze the best data loading performance of Python with tensorpack.dataflow. Symbolic programming (e.g. tf.data) does not offer the data processing flexibility needed in research. Tensorpack squeezes the most performance out of pure Python with various auto parallelization strategies. There are too many symbolic function wrappers already. Tensorpack includes only a few common layers. ... -
21
auto_ml
Automated machine learning for analytics & production
auto_ml is designed for production. Here's an example that includes serializing and loading the trained model, then getting predictions on single dictionaries, roughly the process you'd likely follow to deploy the trained model. Before you go any further, try running the code. Load up some data (either a DataFrame, or a list of dictionaries, where each dictionary is a row of data). Make a column_descriptions dictionary that tells us which attribute name in each row represents the value we’re trying to predict. ...
- Previous
- You're on page 1
- Next
