Search Results for "data integration"
Sort By:
Showing 33 open source projects for "data integration"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild, govern, and optimize agents and models with Gemini Enterprise Agent Platform.
-
Compliant and Reliable File Transfers Backed by Top Security CertificationsStop relying on non-certified, legacy file transfer tools that creak under the weight of modern security demands. Get full audit trails, advanced access controls and more supported by an award-winning team of experts. Start your free 25-day trial today.
-
1
RAGFlow
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding. It offers a streamlined RAG workflow for businesses of any scale, combining LLM (Large Language Models) to provide truthful question-answering capabilities, backed by well-founded citations from various complex formatted data. -
2
X-AnyLabeling
Effortless data labeling with AI support from Segment Anything
X-AnyLabeling is an open-source data annotation platform designed to streamline the process of labeling datasets for computer vision and multimodal AI applications. The software integrates an AI-powered labeling engine that allows users to generate annotations automatically with the assistance of modern vision models such as Segment Anything and various object detection frameworks. It supports labeling tasks across images and videos and enables developers to prepare training datasets for... -
3
C3
The goal of CLAIMED is to enable low-code/no-code rapid prototyping
C3 is an open-source framework designed to simplify the development and deployment of data science and machine learning workflows through reusable components and low-code development techniques. The framework focuses on enabling rapid prototyping while maintaining a path to production through automated CI/CD integration. CLAIMED provides a component-based architecture where data processing steps, models, and workflows can be packaged into reusable operators. -
4
Apache Hamilton
Helps data scientists define testable self-documenting dataflows
Apache Hamilton is an open-source Python framework designed to simplify the creation and management of dataflows used in analytics, machine learning pipelines, and data engineering workflows. The framework enables developers to define data transformations as simple Python functions, where each function represents a node in a dataflow graph and its parameters define dependencies on other nodes. Hamilton automatically analyzes these functions and constructs a directed acyclic graph... -
Stop Storing Third-Party Tokens in Your DatabaseRolling your own OAuth token storage can be a security liability. Token Vault securely stores access and refresh tokens from federated providers and handles exchange and renewal automatically. Connected accounts, refresh exchange, and privileged worker flows included.
-
5
Ploomber
The fastest way to build data pipelines
Ploomber is an open-source framework designed to simplify the development and deployment of data science and machine learning pipelines. It allows developers to transform exploratory data analysis workflows into production-ready pipelines without rewriting large portions of code. The system integrates with common development environments such as Jupyter Notebook, VS Code, and PyCharm, enabling data scientists to continue working with familiar tools while building scalable workflows. Ploomber... -
6
ComfyUI-3D-Pack
An extensive node suite that enables ComfyUI to process 3D inputs
ComfyUI-3D-Pack is an extension package for the ComfyUI visual AI workflow environment that enables users to generate and manipulate 3D assets using advanced machine learning techniques. ComfyUI itself is a node-based interface for designing and executing generative AI pipelines, and this extension expands its capabilities by introducing nodes specifically designed for working with three-dimensional data. The package allows the platform to process inputs such as meshes and UV textures and... -
7
PySINDy
A package for the sparse identification of nonlinear dynamical systems
PySINDy is a Python library that implements the Sparse Identification of Nonlinear Dynamics (SINDy) method for discovering mathematical models of dynamical systems from data. The framework focuses on identifying governing equations that describe the behavior of complex physical systems by selecting sparse combinations of candidate functions. Instead of fitting a purely predictive machine learning model, PySINDy attempts to recover interpretable differential equations that explain how a... -
8
Robyn
Experimental, AI/ML-powered and open sourced Marketing Mix Modeling
Robyn is an open-source, AI/ML-powered Marketing Mix Modeling (MMM) toolkit developed by Meta Marketing Science under the “facebookexperimental” GitHub umbrella. Its goal is to democratize rigorous MMM: what traditionally required expert statisticians and expensive consulting becomes accessible to any company with data. Robyn takes in historical data (spends on different marketing channels, conversions, or revenue, and optional context or organic-media variables) and uses a combination of... -
9
docext
An on-premises, OCR-free unstructured data extraction
docext is a document intelligence toolkit that uses vision-language models to extract structured information from documents such as PDFs, forms, and scanned images. The system is designed to operate entirely on-premises, allowing organizations to process sensitive documents without relying on external cloud services. Unlike traditional document processing pipelines that rely heavily on optical character recognition, docext leverages multimodal AI models capable of understanding both visual... -
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
10
skfolio
Python library for portfolio optimization built on top of scikit-learn
skfolio is a Python library designed for portfolio optimization and financial risk management that integrates closely with the scikit-learn ecosystem. The project provides a unified machine learning-style framework for building, validating, and comparing portfolio allocation strategies using financial data. By following the familiar scikit-learn API design, the library allows quantitative researchers and developers to apply techniques such as model selection, cross-validation, and... -
11
Datasets
Hub of ready-to-use datasets for ML models
Datasets is a library for easily accessing and sharing datasets, and evaluation metrics for Natural Language Processing (NLP), computer vision, and audio tasks. Load a dataset in a single line of code, and use our powerful data processing methods to quickly get your dataset ready for training in a deep learning model. Backed by the Apache Arrow format, process large datasets with zero-copy reads without any memory constraints for optimal speed and efficiency. We also feature a deep integration with the Hugging Face Hub, allowing you to easily load and share a dataset with the wider NLP community. ... -
12
Jina
Build cross-modal and multimodal applications on the cloud
...Jina handles the infrastructure complexity, making advanced solution engineering and cloud-native technologies accessible to every developer. Build applications that deliver fresh insights from multiple data types such as text, image, audio, video, 3D mesh, PDF with Jina AI’s DocArray. Polyglot gateway that supports gRPC, Websockets, HTTP, GraphQL protocols with TLS. Intuitive design pattern for high-performance microservices. Seamless Docker container integration: sharing, exploring, sandboxing, versioning and dependency control via Jina Hub. ... -
13
GluonTS
Probabilistic time series modeling in Python
GluonTS is a Python package for probabilistic time series modeling, focusing on deep learning based models. GluonTS requires Python 3.6 or newer, and the easiest way to install it is via pip. We train a DeepAR-model and make predictions using the simple "airpassengers" dataset. The dataset consists of a single time-series, containing monthly international passengers between the years 1949 and 1960, a total of 144 values (12 years * 12 months). We split the dataset into train and test parts,... -
14
TNT
A lightweight library for PyTorch training tools and utilities
TNT is a lightweight training framework developed by Meta that simplifies the process of building and managing machine learning training loops using PyTorch. The project focuses on providing a flexible yet structured environment for implementing training pipelines without the complexity of large deep learning frameworks. It introduces modular abstractions that allow developers to organize training logic into reusable components such as trainers, evaluators, and callbacks. This design helps... -
15
OpenVINO Notebooks
Jupyter notebook tutorials for OpenVINO
openvino_notebooks is a collection of interactive Jupyter notebooks designed to demonstrate how to build, optimize, and deploy artificial intelligence applications using the OpenVINO toolkit. The repository provides practical tutorials that guide developers through various AI workflows including computer vision, natural language processing, and generative AI tasks. Each notebook demonstrates how to run pre-trained models, optimize inference performance, and deploy models across hardware such... -
16
PyG
Graph Neural Network Library for PyTorch
PyG (PyTorch Geometric) is a library built upon PyTorch to easily write and train Graph Neural Networks (GNNs) for a wide range of applications related to structured data. It consists of various methods for deep learning on graphs and other irregular structures, also known as geometric deep learning, from a variety of published papers. In addition, it consists of easy-to-use mini-batch loaders for operating on many small and single giant graphs, multi GPU-support, DataPipe support,... -
17
VoxelMorph
Unsupervised Learning for Image Registration
VoxelMorph is an open-source deep learning framework designed for medical image registration, a process that aligns multiple medical scans into a common spatial coordinate system. Traditional image registration techniques typically rely on optimization procedures that must be executed separately for each pair of images, which can be computationally expensive and slow. VoxelMorph approaches the problem using neural networks that learn to predict deformation fields that transform one image so... -
18
Natural Language Toolkit
NLTK Source
The Natural Language Toolkit (NLTK) is a widely used open-source Python library designed for working with human language data and building natural language processing (NLP) applications. It provides a comprehensive suite of modules, datasets, and tutorials that support both symbolic and statistical approaches to language processing. The toolkit includes implementations of many foundational NLP algorithms and utilities, enabling developers to perform tasks such as tokenization, stemming,... -
19
TensorFlow Quantum
Open-source Python framework for hybrid quantum-classical ml learning
TensorFlow Quantum is an open-source software framework designed for building and training hybrid quantum-classical machine learning models within the TensorFlow ecosystem. The framework enables researchers and developers to represent quantum circuits as data and integrate them directly into machine learning workflows. By combining classical deep learning techniques with quantum algorithms, the platform allows experimentation with quantum machine learning methods that may offer advantages... -
20
higgsfield
Fault-tolerant, highly scalable GPU orchestration
Higgsfield is an open-source, fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters, such as Large Language Models (LLMs). -
21
Datapipe
Real-time, incremental ETL library for ML with record-level depend
Datapipe is a real-time, incremental ETL library for Python with record-level dependency tracking. Datapipe is designed to streamline the creation of data processing pipelines. It excels in scenarios where data is continuously changing, requiring pipelines to adapt and process only the modified data efficiently. This library tracks dependencies for each record in the pipeline, ensuring minimal and efficient data processing. -
22
CLIP-as-service
Embed images and sentences into fixed-length vectors
...Smooth integration with neural search ecosystem including Jina and DocArray. Build cross-modal and multi-modal solutions in no time. -
23
DPM-Solver
Fast ODE Solver for Diffusion Probabilistic Model Sampling
DPM-Solver is a machine learning research implementation focused on accelerating the sampling process in diffusion probabilistic models used for generative AI tasks. Diffusion models are powerful generative systems capable of producing high-quality images and other data, but traditional sampling methods often require hundreds or thousands of computational steps. The project introduces a specialized numerical solver designed to approximate the diffusion process using a small number of high-order integration steps. By reformulating the sampling problem as the solution of a diffusion-related ordinary differential equation, the solver can produce high-quality samples much more efficiently. ... -
24
fastquant
Backtest and optimize your ML trading strategies with only 3 lines
fastquant is a Python library designed to simplify quantitative financial analysis and algorithmic trading strategy development. The project focuses on making backtesting accessible by providing a high-level interface that allows users to test investment strategies with only a few lines of code. It integrates historical market data sources and trading frameworks so that users can quickly build experiments without constructing complex data pipelines. The framework enables users to test common... -
25
TensorFlow Ranking
Learning to rank in TensorFlow
TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform. Commonly used loss functions including pointwise, pairwise, and listwise losses. Commonly used ranking metrics like Mean Reciprocal Rank (MRR) and Normalized Discounted Cumulative Gain (NDCG). Multi-item (also known as groupwise) scoring functions. LambdaLoss implementation for direct ranking metric optimization. Unbiased Learning-to-Rank from biased feedback data. We envision that this library...
