Search Results for "data processing"
Sort By:
Showing 65 open source projects for "data processing"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
1
Bytewax
Python Stream Processing
...Bytewax is a Python framework and Rust distributed processing engine that uses a dataflow computational model to provide parallelizable stream processing and event processing capabilities similar to Flink, Spark, and Kafka Streams. You can use Bytewax for a variety of workloads from moving data à la Kafka Connect style all the way to advanced online machine learning workloads. Bytewax is not limited to streaming applications but excels anywhere that data can be distributed at the input and output. -
2
Diffgram
Training data (data labeling, annotation, workflow) for all data types
...Training Data is the art of supervising machines through data. This includes the activities of annotation, which produces structured data; ready to be consumed by a machine learning model. Annotation is required because raw media is considered to be unstructured and not usable without it. That’s why training data is required for many modern machine learning use cases including computer vision, natural language processing and speech recognition. -
3
Awesome Fraud Detection Research Papers
A curated list of data mining papers about fraud detection
A curated list of data mining papers about fraud detection from several conferences. -
4
Datasets
Hub of ready-to-use datasets for ML models
Datasets is a library for easily accessing and sharing datasets, and evaluation metrics for Natural Language Processing (NLP), computer vision, and audio tasks. Load a dataset in a single line of code, and use our powerful data processing methods to quickly get your dataset ready for training in a deep learning model. Backed by the Apache Arrow format, process large datasets with zero-copy reads without any memory constraints for optimal speed and efficiency. ... -
AI-powered service management for IT and enterprise teamsGive your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
-
5
python-small-examples
Focus on creating classic Python small examples and cases
python-small-examples is an open-source educational repository that contains hundreds of concise Python programming examples designed to illustrate practical coding techniques. The project focuses on teaching programming concepts through small, focused scripts that demonstrate common tasks in data processing, visualization, and general programming. Each example highlights a specific function or programming pattern so that learners can quickly understand how to apply Python features in real-world scenarios. The repository includes examples covering topics such as file processing, JSON manipulation, data visualization, and library usage. ... -
6
Kornia
Open Source Differentiable Computer Vision Library
Kornia is a differentiable computer vision library for PyTorch. It consists of a set of routines and differentiable modules to solve generic computer vision problems. At its core, the package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions. Inspired by existing packages, this library is composed by a subset of packages containing operators that can be inserted within... -
7
TorchAudio
Data manipulation and transformation for audio signal processing
The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the same philosophy of providing strong GPU acceleration, having a focus on trainable features through the autograd system, and having consistent style (tensor names and dimension names). Therefore, it is primarily a machine learning library and not a general signal processing library. The benefits of PyTorch can be seen in torchaudio through having all the computations be through PyTorch... -
8
NVIDIA NeMo
Toolkit for conversational AI
NVIDIA NeMo, part of the NVIDIA AI platform, is a toolkit for building new state-of-the-art conversational AI models. NeMo has separate collections for Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Text-to-Speech (TTS) models. Each collection consists of prebuilt modules that include everything needed to train on your data. Every module can easily be customized, extended, and composed to create new conversational AI model architectures. Conversational AI architectures are typically large and require a lot of data and compute for training. ... -
9
Quantitative Trading System
A comprehensive quantitative trading system with AI-powered analysis
Quantitative Trading System is a comprehensive quantitative trading platform that integrates artificial intelligence, financial data analysis, and automated strategy execution within a unified software system. The project is designed to provide an end-to-end infrastructure for building and operating algorithmic trading strategies in financial markets. It includes tools for collecting and processing market data from multiple sources, performing statistical and machine learning analysis, and generating trading signals based on quantitative models. ... -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
10
POT
Python Optimal Transport
This open source Python library provides several solvers for optimization problems related to Optimal Transport for signal, image processing and machine learning. -
11
AutoGluon
AutoGluon: AutoML for Image, Text, and Tabular Data
AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on automated stack ensembling, deep learning, and real-world applications spanning image, text, and tabular data. Intended for both ML beginners and experts, AutoGluon enables you to quickly prototype deep learning and classical ML solutions for your raw data with a few lines of code. Automatically utilize state-of-the-art techniques (where appropriate) without expert knowledge. Leverage automatic hyperparameter tuning, model selection/ensembling, architecture search, and data processing. ... -
12
docext
An on-premises, OCR-free unstructured data extraction
docext is a document intelligence toolkit that uses vision-language models to extract structured information from documents such as PDFs, forms, and scanned images. The system is designed to operate entirely on-premises, allowing organizations to process sensitive documents without relying on external cloud services. Unlike traditional document processing pipelines that rely heavily on optical character recognition, docext leverages multimodal AI models capable of understanding both visual... -
13
Natural Language Toolkit
NLTK Source
The Natural Language Toolkit (NLTK) is a widely used open-source Python library designed for working with human language data and building natural language processing (NLP) applications. It provides a comprehensive suite of modules, datasets, and tutorials that support both symbolic and statistical approaches to language processing. The toolkit includes implementations of many foundational NLP algorithms and utilities, enabling developers to perform tasks such as tokenization, stemming, parsing, classification, and semantic reasoning. ... -
14
Advanced NLP with spaCy
Advanced NLP with spaCy: A free online course
...It also demonstrates how spaCy pipelines work and how developers can extend them with custom components and training data. The course is structured as a hands-on learning environment where students can run code examples, experiment with NLP techniques, and build practical language processing applications. Because spaCy is widely used in production environments, the course emphasizes industrial-strength NLP workflows and best practices. -
15
Triton Inference Server
The Triton Inference Server provides an optimized cloud
...Triton enables teams to deploy any AI model from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more. Triton supports inference across cloud, data center, edge, and embedded devices on NVIDIA GPUs, x86 and ARM CPU, or AWS Inferentia. Triton delivers optimized performance for many query types, including real-time, batched, ensembles, and audio/video streaming. Provides Backend API that allows adding custom backends and pre/post-processing operations. Model pipelines using Ensembling or Business Logic Scripting (BLS). ... -
16
TorchRL
A modular, primitive-first, python-first PyTorch library
TorchRL is an open-source Reinforcement Learning (RL) library for PyTorch. TorchRL provides PyTorch and python-first, low and high-level abstractions for RL that are intended to be efficient, modular, documented, and properly tested. The code is aimed at supporting research in RL. Most of it is written in Python in a highly modular way, such that researchers can easily swap components, transform them, or write new ones with little effort. -
17
Kaggle Solutions
Collection of Kaggle Solutions and Ideas
...The repository also highlights important machine learning concepts such as feature engineering, cross-validation strategies, ensemble modeling, and post-processing methods commonly used in winning solutions. Because the content is organized by competition categories such as computer vision, natural language processing, tabular data, and time-series forecasting, users can explore techniques relevant to specific problem types. -
18
FlexLLMGen
Running large language models on a single GPU
FlexLLMGen is an open-source inference engine designed to run large language models efficiently on limited hardware resources such as a single GPU. The system focuses on high-throughput generation workloads where large batches of text must be processed quickly, such as large-scale data extraction or document analysis tasks. Instead of requiring expensive multi-GPU systems, the framework uses techniques such as memory offloading, compression, and optimized batching to run large models on... -
19
Amazing-Python-Scripts
Curated collection of Amazing Python scripts
Amazing-Python-Scripts is a collaborative repository that collects a wide variety of Python scripts designed to demonstrate practical programming techniques and automation tasks. The project includes scripts ranging from beginner-level utilities to more advanced applications involving machine learning, data processing, and system automation. Its goal is to provide developers with useful coding examples that can solve everyday problems, automate repetitive tasks, or serve as learning exercises. The repository encourages community contributions, allowing developers to add their own scripts and improve existing ones through pull requests. ... -
20
C3
The goal of CLAIMED is to enable low-code/no-code rapid prototyping
C3 is an open-source framework designed to simplify the development and deployment of data science and machine learning workflows through reusable components and low-code development techniques. The framework focuses on enabling rapid prototyping while maintaining a path to production through automated CI/CD integration. CLAIMED provides a component-based architecture where data processing steps, models, and workflows can be packaged into reusable operators. -
21
ComfyUI-3D-Pack
An extensive node suite that enables ComfyUI to process 3D inputs
ComfyUI-3D-Pack is an extension package for the ComfyUI visual AI workflow environment that enables users to generate and manipulate 3D assets using advanced machine learning techniques. ComfyUI itself is a node-based interface for designing and executing generative AI pipelines, and this extension expands its capabilities by introducing nodes specifically designed for working with three-dimensional data. The package allows the platform to process inputs such as meshes and UV textures and... -
22
TensorFlow
TensorFlow is an open source library for machine learning
Originally developed by Google for internal use, TensorFlow is an open source platform for machine learning. Available across all common operating systems (desktop, server and mobile), TensorFlow provides stable APIs for Python and C as well as APIs that are not guaranteed to be backwards compatible or are 3rd party for a variety of other languages. The platform can be easily deployed on multiple CPUs, GPUs and Google's proprietary chip, the tensor processing unit (TPU). TensorFlow... -
23
fastdup
An unsupervised and free tool for image and video dataset analysis
fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale. -
24
hls4ml
Machine learning on FPGAs using HLS
...This approach allows machine learning algorithms to run directly on specialized hardware, making them suitable for applications that require extremely fast response times and minimal power consumption. The framework was originally developed for high-energy physics experiments where real-time decision systems must process large volumes of data with strict latency constraints. Over time, it has expanded to support a variety of scientific and industrial applications including signal processing, embedded systems, and biomedical monitoring. -
25
torchtext
Data loaders and abstractions for text and NLP
We recommend Anaconda as a Python package management system. Please refer to pytorch.org for the details of PyTorch installation. LTS versions are distributed through a different channel than the other versioned releases. Alternatively, you might want to use the Moses tokenizer port in SacreMoses (split from NLTK). You have to install SacreMoses. To build torchtext from source, you need git, CMake and C++11 compiler such as g++. When building from source, make sure that you have the same C++...
