Search Results for "data processing"
Sort By:
Showing 86 open source projects for "data processing"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Earn up to 16% annual interest with Nexo.Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
-
1
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
2
Lesan
New way to create web server and NoSQL data model
Lesan is a multilingual text processing and translation library designed for natural language processing (NLP) applications. It provides tools for text normalization, tokenization, and translation across multiple languages. -
3
GridDB
GridDB is a next-generation open source database
...Multi-model architecture capable of supporting various data stores with time-series data-oriented and pluggable data stores for efficient real-time processing and management of huge amounts of time-series data at high frequency. Various architectural innovations, such as in-memory orientation with "memory as the main unit and disk as the secondary unit" and event-driven design with minimal overhead, have been incorporated to achieve processing capabilities that can handle petabyte-scale applications. -
4
Addax
Addax is a versatile open-source ETL tool
Addax is a data integration and ETL (Extract, Transform, Load) tool designed for high-performance data migration tasks. It simplifies the process of moving data between different systems and formats. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
KCloud‑Platform‑IoT
KCloud-Platform-IoT
KCloud-Platform-IoT is a comprehensive open-source IoT management platform built with Spring Cloud and Vue.js. It supports device registration, data collection, rule-based processing, and dashboard visualization. Designed for scalability and modularity, the platform is ideal for managing large IoT fleets in industrial or smart city environments. -
6
Flink CDC
Flink CDC is a streaming data integration tool
Apache Flink CDC is a distributed data integration tool that captures data changes in real-time from various databases. It leverages Change Data Capture (CDC) technology to stream data changes into Apache Flink, enabling real-time analytics and data processing. Flink CDC simplifies data pipeline development with its declarative YAML configurations. -
7
Memgraph
Open-source graph database, tuned for dynamic analytics environments
A high-performance, in-memory graph database designed for real-time analytics and complex network data processing. -
8
TDengine
Open-source time-series database with high-performance and scalability
Enables efficient, real-time data ingestion, processing and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine can be widely applied to IoT, Industrial Internet, Connected Vehicles, DevOps, Energy , Finance and many other use-cases. TDengine’s innovative design and purpose-built storage engine outperforms other time-series databases for data ingestion, querying and data compression while significantly reducing storage and computing costs. ... -
9
HStreamDB
HStreamDB is an open-source, cloud-native streaming database
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications. By subscribing to streams in HStreamDB, any update of the data stream will be pushed to your apps in real-time, and this promotes your apps to be more responsive. You can also replace message brokers with HStreamDB and everything you do with message brokers can be done better with HStreamDB. HStreamDB provides built-in support for event time-based stream processing. ... -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
10
eXist-db
eXist Native XML Database and Application Platform
eXist-db is an open-source, native XML database and application platform that provides a powerful environment for storing, querying, and managing XML documents. It is designed for complex data management needs, offering XQuery, XSLT, and RESTful web services for interacting with structured data. -
11
pg_analytics
DuckDB-powered analytics for Postgres
pg_analytics (formerly named pg_lakehouse) puts DuckDB inside Postgres. With pg_analytics installed, Postgres can query foreign object stores like AWS S3 and table formats like Iceberg or Delta Lake. Queries are pushed down to DuckDB, a high-performance analytical query engine. By transforming Postgres into a performant search and analytics engine, ParadeDB frees your team from the pain of scaling and syncing Elasticsearch. -
12
Logstash
Centralize, transform and stash your data
Logstash is a server-side data processing pipeline that dynamically ingests data from numerous sources, transforms it, and ships it to your favorite “stash” regardless of format or complexity. It supports and ingests data of all shapes, sizes and sources, dynamically transforms and prepares this data, and transports it to the output of your choice. Logstash is extensible, with over 200 plugins available to let you create and configure your pipeline how you choose. -
13
Bitalosdb
Bitalosdb is a high-performance KV storage engine
BitalosDB is a distributed, high-performance key-value database designed for cloud-native applications. It is optimized for scalability, supporting large workloads while maintaining low latency and high availability. -
14
CogDB
Micro Graph Database for Python Applications
Cog is a lightweight, embedded graph database for Go that provides a simple interface for storing and querying graph-based data structures, making it useful for knowledge representation and graph analytics. -
15
AlaSQL
JavaScript SQL database for browser and Node.js for relational tables
AlaSQL.js - JavaScript SQL database for browser and Node.js. Handles both traditional relational tables and nested JSON data (NoSQL). Export, store, and import data from localStorage, IndexedDB, or Excel. We focus on speed by taking advantage of the dynamic nature of JavaScript when building up queries. Real-world solutions demand flexibility regarding where data comes from and where it is to be stored. We focus on flexibility by making sure you can import/export and query directly on data... -
16
StarRocks
StarRocks is a next-gen sub-second MPP database for full analytics
StarRocks is the next generation of real-time SQL engines for enterprise analytics. Real-time analytics is notoriously difficult. Complex data pipelines and de-normalized tables have always been a necessary evil. Processing any updates or deletes once data arrives has not been possible- until now. StarRocks solves these challenges and makes real-time analytics easy. Get amazing query performance on Star or Snowflake Schemas directly. From canceled orders to updated items, your analytics applications can easily handle them with StarRocks. ... -
17
DuckDB
DuckDB is an in-process SQL OLAP Database Management System
...DuckDB supports arbitrary and nested correlated subqueries, window functions, collations, complex types (arrays, structs), and more. For more information on the goals of DuckDB, please refer to the Why DuckDB page on our website. Processing and storing tabular datasets, e.g. from CSV or Parquet files. Interactive data analysis, e.g. Joining & aggregate multiple large tables. Concurrent large changes, to multiple large tables, e.g. appending rows, adding/removing/updating columns. Large result set transfer to client. For development, DuckDB requires CMake, Python3 and a C++11 compliant compiler. ... -
18
Tarantool
Get your data in RAM, get compute close to data, enjoy the performance
In OLTP scenarios, Tarantool can be used instead of relational databases. Such a solution will work many times faster. With Tarantool, you can replace the traditional bundle of database & cache and reduce operational costs. Tarantool is tolerant of write-heavy loads. It also allows keeping full-featured applications close to the data, thus reducing data access network latency to zero. The open-source Community Edition lets you develop applications and speed up a system in operation. It... -
19
GreptimeDB
An open-source, cloud-native, unified time series database for metrics
GreptimeDB treats all time series as contextual events with timestamps, and thus unifies the processing of metrics, logs, and events. It supports analyzing metrics, logs, and events with SQL, PromQL, and streaming with continuous aggregation. GreptimeDB is a time-series database optimized for storing and querying large amounts of time-series data, commonly used in monitoring and IoT applications. -
20
Lantern Database
PostgreSQL vector database extension for building AI applications
Lantern is a real-time data transformation engine that enables data engineers to build, run, and monitor streaming data pipelines with SQL. It’s designed to process events in motion, offering low-latency stream transformations, aggregations, and enrichment in a declarative way. Lantern is especially suited for modern data infrastructure and analytics platforms. -
21
KuzuDB
Embeddable property graph database management system
KuzuDB is a high-performance graph database optimized for analytical queries, built from the ground up with a columnar storage engine. It is designed to efficiently process large-scale graph workloads, making it ideal for data science, machine learning, and knowledge graph applications. -
22
SuperDuperDB
Integrate, train and manage any AI models and APIs with your database
...Integrate and combine models from Sklearn, PyTorch, HuggingFace with AI APIs such as OpenAI to build even the most complex AI applications and workflows. Train models on your data in your datastore simply by querying without additional ingestion and pre-processing. -
23
PolarDB-X
PolarDB-X is a cloud native distributed SQL Database
PolarDB-X is a cloud-native distributed SQL database designed to handle high concurrency, massive storage, and complex querying scenarios. It features a shared-nothing architecture that decouples computing from storage, providing scalability and flexibility for various applications. -
24
CDC Connectors for Apache Flink
CDC Connectors for Apache Flink
This project provides a set of source connectors for Apache Flink® directly ingesting changes coming from different databases using Change Data Capture(CDC). CDC Connectors for Apache Flink® is a set of source connectors for Apache Flink®, ingesting changes from different databases using change data capture (CDC). CDC Connectors for Apache Flink® integrates Debezium as the engine to capture data changes. So it can fully leverage the ability of Debezium. Supports reading database snapshot and continues to read transaction logs with exactly-once processing even failures happen. ... -
25
MongoDB PHP Library
The Official MongoDB PHP library
...Built on top of the underlying MongoDB PHP extension, the library handles serialization, connection pooling, and error handling in a way that feels natural in idiomatic PHP. It supports rich query expressions, bulk writes, change streams, transactions, and GridFS, making it suitable for everything from simple content apps to complex data processing services. The project also includes helpers for working with BSON types such as ObjectId, UTC datetime, and decimals, which helps bridge the gap between native PHP types and MongoDB’s storage model.
