Search Results for "data processing" - Page 2
Sort By:
Showing 65 open source projects for "data processing"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
Semantic Router
Superfast AI decision making and processing of multi-modal data
Semantic Router is a superfast decision-making layer for your LLMs and agents. Rather than waiting for slow, unreliable LLM generations to make tool-use or safety decisions, we use the magic of semantic vector space — routing our requests using semantic meaning. Combining LLMs with deterministic rules means we can be confident that our AI systems behave as intended. Cramming agent tools into the limited context window is expensive, slow, and fundamentally limited. Semantic Router enables... -
2
Jina
Build cross-modal and multimodal applications on the cloud
Jina is a framework that empowers anyone to build cross-modal and multi-modal applications on the cloud. It uplifts a PoC into a production-ready service. Jina handles the infrastructure complexity, making advanced solution engineering and cloud-native technologies accessible to every developer. Build applications that deliver fresh insights from multiple data types such as text, image, audio, video, 3D mesh, PDF with Jina AI’s DocArray. Polyglot gateway that supports gRPC, Websockets, HTTP,... -
3
fastai
Deep learning library
...It aims to do both things without substantial compromises in ease of use, flexibility, or performance. This is possible thanks to a carefully layered architecture, which expresses common underlying patterns of many deep learning and data processing techniques in terms of decoupled abstractions. These abstractions can be expressed concisely and clearly by leveraging the dynamism of the underlying Python language and the flexibility of the PyTorch library. fastai is organized around two main design goals: to be approachable and rapidly productive, while also being deeply hackable and configurable. ... -
4
Ray
A unified framework for scalable computing
Modern workloads like deep learning and hyperparameter tuning are compute-intensive and require distributed or parallel execution. Ray makes it effortless to parallelize single machine code — go from a single CPU to multi-core, multi-GPU or multi-node with minimal code changes. Accelerate your PyTorch and Tensorflow workload with a more resource-efficient and flexible distributed execution framework powered by Ray. Accelerate your hyperparameter search workloads with Ray Tune. Find the best... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
OpenVINO Notebooks
Jupyter notebook tutorials for OpenVINO
...Many notebooks include end-to-end examples that show how to prepare input data, load optimized models, run inference, and visualize results. The project is particularly useful for developers who want to learn how to optimize machine learning inference pipelines for production environments. -
6
crème de la crème of AI courses
This repository is a curated collection of links to various courses
...Topics covered include deep learning, natural language processing, computer vision, large language models, linear algebra, reinforcement learning, and machine learning engineering. Because the repository links to well-known educational content such as university lecture series and professional training materials, it functions as a structured roadmap for individuals who want to develop expertise in artificial intelligence. -
7
Python Code Tutorials
The Python Code Tutorials
Python Code Tutorials is a large educational repository that aggregates programming tutorials from the “The Python Code” website into a structured collection of Python projects and learning materials. The repository covers a wide range of programming topics including cybersecurity, networking, web scraping, machine learning, GUI development, and automation scripts. Each tutorial typically includes complete Python code examples and explanations that demonstrate how to build real tools and... -
8
Datapipe
Real-time, incremental ETL library for ML with record-level depend
Datapipe is a real-time, incremental ETL library for Python with record-level dependency tracking. Datapipe is designed to streamline the creation of data processing pipelines. It excels in scenarios where data is continuously changing, requiring pipelines to adapt and process only the modified data efficiently. This library tracks dependencies for each record in the pipeline, ensuring minimal and efficient data processing. -
9
SMILI
Scientific Visualisation Made Easy
The Simple Medical Imaging Library Interface (SMILI), pronounced 'smilie', is an open-source, light-weight and easy-to-use medical imaging viewer and library for all major operating systems. The main sMILX application features for viewing n-D images, vector images, DICOMs, anonymizing, shape analysis and models/surfaces with easy drag and drop functions. It also features a number of standard processing algorithms for smoothing, thresholding, masking etc. images and models, both with... -
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
10
PoseidonQ - AI/ML Based QSAR Modeling
ML based QSAR Modelling And Translation of Model to Deployable WebApps
- This Software was made with an intention to make QSAR/QSPR development more efficient and reproducible. - Published in ACS, Journal of Chemical Information and Modeling . Link : https://pubs.acs.org/doi/10.1021/acs.jcim.4c02372 - Simple to use and no compromise on essential features necessary to make reliable QSAR models. - From Generating Reliable ML Based QSAR Models to Developing Your Own QSAR WebApp. For any feedback or queries, contact kabeermuzammil614@gmail.com - Available on... -
11
Lightning-Hydra-Template
PyTorch Lightning + Hydra. A very user-friendly template
...A collection of best practices for efficient workflow and reproducibility. Thoroughly commented - you can use this repo as a reference and educational resource. Not fitted for data engineering - the template configuration setup is not designed for building data processing pipelines that depend on each other. PyTorch Lightning, a lightweight PyTorch wrapper for high-performance AI research. Think of it as a framework for organizing your PyTorch code. Hydra, a framework for elegantly configuring complex applications. ... -
12
find-similar
User-friendly library to find similar objects
The mission of the FindSimilar project is to provide a powerful and versatile open source library that empowers developers to efficiently find similar objects and perform comparisons across a variety of data types. Whether dealing with texts, images, audio, or more, our project aims to simplify the process of identifying similarities and enhancing decision-making. https://github.com/findsimilar/find-similar - GitHub repo http://demo.findsimilar.org/ - Demo project and... -
13
Horovod
Distributed training framework for TensorFlow, Keras, PyTorch, etc.
...Horovod can be installed on-premise or run out-of-the-box in cloud platforms, including AWS, Azure, and Databricks. Horovod can additionally run on top of Apache Spark, making it possible to unify data processing and model training into a single pipeline. Once Horovod has been configured, the same infrastructure can be used to train models with any framework, making it easy to switch between TensorFlow, PyTorch, MXNet, and future frameworks as machine learning tech stacks continue to evolve. Start scaling your model training with just a few lines of Python code. ... -
14
AB3DMOT
Official Python Implementation for "3D Multi-Object Tracking
...The system processes detection results from 3D object detectors that analyze LiDAR point clouds and uses them to track multiple objects across consecutive frames. Its tracking pipeline relies on a combination of classical algorithms, including a Kalman filter for state estimation and the Hungarian algorithm for data association between detected objects and existing tracks. This relatively simple design allows the tracker to achieve very high processing speeds while maintaining competitive tracking accuracy. The project also introduces new evaluation metrics specifically designed for assessing performance in 3D tracking benchmarks. The framework has been evaluated on widely used datasets such as KITTI and nuScenes and demonstrates strong performance compared with more complex tracking systems. -
15
OGB
Benchmark datasets, data loaders, and evaluators for graph machine
...OGB fully automates dataset processing. The OGB data loaders automatically download and process graphs, provide graph objects that are fully compatible with Pytorch Geometric and DGL. OGB provides standardized dataset splits and evaluators that allow for easy and reliable comparison of different models in a unified manner. -
16
Merlion
A Machine Learning Framework for Time Series Intelligence
Merlion is a Python library for time series intelligence. It provides an end-to-end machine learning framework that includes loading and transforming data, building and training models, post-processing model outputs, and evaluating model performance. It supports various time series learning tasks, including forecasting, anomaly detection, and change point detection for both univariate and multivariate time series. This library aims to provide engineers and researchers a one-stop solution to rapidly develop models for their specific time series needs, and benchmark them across multiple time series datasets. -
17
Mars Framework
Mars is a tensor-based unified framework for large-scale data
...Its architecture automatically divides large computational tasks into smaller chunks that can be executed across multiple nodes in a cluster, allowing complex analytics, machine learning workflows, and data transformations to run efficiently at scale. Mars is particularly useful for workloads that exceed the memory capacity of a single machine or require high levels of parallel processing. -
18
Pattern
Web mining module for Python, with tools for scraping
Pattern is an open-source Python library that provides tools for web mining, natural language processing, machine learning, and network analysis. The project integrates multiple capabilities into a single framework that allows developers to collect, process, and analyze textual data from the web. It includes modules for web scraping and crawling that can retrieve information from sources such as social media platforms, search engines, and online knowledge bases. -
19
pyntcloud
pyntcloud is a Python library for working with 3D point clouds
This page will introduce the general concept of point clouds and illustrate the capabilities of pyntcloud as a point cloud processing tool. Point clouds are one of the most relevant entities for representing three dimensional data these days, along with polygonal meshes (which are just a special case of point clouds with connectivity graph attached). In its simplest form, a point cloud is a set of points in a cartesian coordinate system. Accurate 3D point clouds can nowadays be (easily and cheaply) acquired from different sources. pyntcloud enables simple and interactive exploration of point cloud data, regardless of which sensor was used to generate it or what the use case is. ... -
20
TensorFlowOnSpark
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters
By combining salient features from the TensorFlow deep learning framework with Apache Spark and Apache Hadoop, TensorFlowOnSpark enables distributed deep learning on a cluster of GPU and CPU servers. It enables both distributed TensorFlow training and inferencing on Spark clusters, with a goal to minimize the amount of code changes required to run existing TensorFlow programs on a shared grid. -
21
MeshCNN in PyTorch
Convolutional Neural Network for 3D meshes in PyTorch
MeshCNN is a deep learning framework designed specifically for processing 3D triangular mesh data using convolutional neural networks. Unlike traditional CNNs that operate on images or voxel grids, MeshCNN performs convolution operations directly on the edges of mesh structures. This design allows the model to capture geometric relationships between mesh elements while preserving the underlying topology of 3D shapes. -
22
AI Platform Training and Prediction
Cloud ML Engine repo
AI Platform Training and Prediction is a collection of machine learning example projects that demonstrate how to train, deploy, and serve models using Google Cloud AI Platform and related services. It includes a wide variety of implementations across frameworks such as TensorFlow, PyTorch, scikit-learn, and XGBoost, allowing developers to explore different approaches to building ML solutions. The repository covers the full machine learning lifecycle, including data preprocessing, model... -
23
igel
Machine learning tool that allows you to train and test models
A delightful machine learning tool that allows you to train/fit, test, and use models without writing code. The goal of the project is to provide machine learning for everyone, both technical and non-technical users. I sometimes needed a tool sometimes, which I could use to fast create a machine learning prototype. Whether to build some proof of concept, create a fast draft model to prove a point or use auto ML. I find myself often stuck writing boilerplate code and thinking too much about... -
24
SparrowRecSys
A Deep Learning Recommender System
SparrowRecSys is an open-source deep learning recommendation system framework designed to demonstrate the architecture and implementation of modern industrial-scale recommender systems. The project integrates multiple machine learning models and data processing pipelines to simulate how real-world recommendation platforms operate. It includes components for offline data processing, feature engineering, model training, real-time data updates, and online recommendation services. SparrowRecSys supports a wide range of state-of-the-art recommendation algorithms, including models for click-through rate prediction and user behavior modeling that are widely used in advertising and content recommendation systems. ... -
25
tensorflow_template_application
TensorFlow template application for deep learning
tensorflow_template_application is a template project that demonstrates how to structure scalable applications built with TensorFlow. The repository provides a standardized architecture that helps developers organize machine learning code into clear components such as data processing, model training, evaluation, and deployment. Instead of focusing on a specific algorithm, the project emphasizes software engineering practices that make machine learning systems easier to maintain and extend. The template includes configuration files, scripts, and project structures that help teams build reproducible experiments and production-ready pipelines. ...
