Search Results for "data processing"
Sort By:
Showing 93 open source projects for "data processing"
View related business solutions-
Earn up to 16% annual interest with Nexo.Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
-
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
1
Flink CDC
Flink CDC is a streaming data integration tool
Apache Flink CDC is a distributed data integration tool that captures data changes in real-time from various databases. It leverages Change Data Capture (CDC) technology to stream data changes into Apache Flink, enabling real-time analytics and data processing. Flink CDC simplifies data pipeline development with its declarative YAML configurations. -
2
KCloud‑Platform‑IoT
KCloud-Platform-IoT
KCloud-Platform-IoT is a comprehensive open-source IoT management platform built with Spring Cloud and Vue.js. It supports device registration, data collection, rule-based processing, and dashboard visualization. Designed for scalability and modularity, the platform is ideal for managing large IoT fleets in industrial or smart city environments. -
3
GridDB
GridDB is a next-generation open source database
...Multi-model architecture capable of supporting various data stores with time-series data-oriented and pluggable data stores for efficient real-time processing and management of huge amounts of time-series data at high frequency. Various architectural innovations, such as in-memory orientation with "memory as the main unit and disk as the secondary unit" and event-driven design with minimal overhead, have been incorporated to achieve processing capabilities that can handle petabyte-scale applications. -
4
Lesan
New way to create web server and NoSQL data model
Lesan is a multilingual text processing and translation library designed for natural language processing (NLP) applications. It provides tools for text normalization, tokenization, and translation across multiple languages. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
6
Memgraph
Open-source graph database, tuned for dynamic analytics environments
A high-performance, in-memory graph database designed for real-time analytics and complex network data processing. -
7
Addax
Addax is a versatile open-source ETL tool
Addax is a data integration and ETL (Extract, Transform, Load) tool designed for high-performance data migration tasks. It simplifies the process of moving data between different systems and formats. -
8
InfluxDB
The open source time series database
...Time series is currently the fastest growing database category there is, and InfluxDB is here to ensure businesses can keep up. InfluxDB provides infrastructure and application monitoring, IoT monitoring and analytics and more. It has APIs for storing and querying data, processing it in the background for ETL or monitoring and alerting purposes. This data can also be visualized, explored and more to help businesses seize opportunities and make the best decisions. InfluxDB is easy to start and easy to scale. Learn more about it on https://www.influxdata.com/ -
9
eXist-db
eXist Native XML Database and Application Platform
eXist-db is an open-source, native XML database and application platform that provides a powerful environment for storing, querying, and managing XML documents. It is designed for complex data management needs, offering XQuery, XSLT, and RESTful web services for interacting with structured data. -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
Bitalosdb
Bitalosdb is a high-performance KV storage engine
BitalosDB is a distributed, high-performance key-value database designed for cloud-native applications. It is optimized for scalability, supporting large workloads while maintaining low latency and high availability. -
11
ClickHouse
A fast open-source OLAP database management system
...ClickHouse also has exceptional hardware efficiency and a host of other features, including a feature-rich SQL database, vectorized query execution, real-time query processing and data ingestion, and more. -
12
TDengine
Open-source time-series database with high-performance and scalability
Enables efficient, real-time data ingestion, processing and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine can be widely applied to IoT, Industrial Internet, Connected Vehicles, DevOps, Energy , Finance and many other use-cases. TDengine’s innovative design and purpose-built storage engine outperforms other time-series databases for data ingestion, querying and data compression while significantly reducing storage and computing costs. ... -
13
AlaSQL
JavaScript SQL database for browser and Node.js for relational tables
AlaSQL.js - JavaScript SQL database for browser and Node.js. Handles both traditional relational tables and nested JSON data (NoSQL). Export, store, and import data from localStorage, IndexedDB, or Excel. We focus on speed by taking advantage of the dynamic nature of JavaScript when building up queries. Real-world solutions demand flexibility regarding where data comes from and where it is to be stored. We focus on flexibility by making sure you can import/export and query directly on data... -
14
pg_analytics
DuckDB-powered analytics for Postgres
pg_analytics (formerly named pg_lakehouse) puts DuckDB inside Postgres. With pg_analytics installed, Postgres can query foreign object stores like AWS S3 and table formats like Iceberg or Delta Lake. Queries are pushed down to DuckDB, a high-performance analytical query engine. By transforming Postgres into a performant search and analytics engine, ParadeDB frees your team from the pain of scaling and syncing Elasticsearch. -
15
DuckDB
DuckDB is an in-process SQL OLAP Database Management System
...DuckDB supports arbitrary and nested correlated subqueries, window functions, collations, complex types (arrays, structs), and more. For more information on the goals of DuckDB, please refer to the Why DuckDB page on our website. Processing and storing tabular datasets, e.g. from CSV or Parquet files. Interactive data analysis, e.g. Joining & aggregate multiple large tables. Concurrent large changes, to multiple large tables, e.g. appending rows, adding/removing/updating columns. Large result set transfer to client. For development, DuckDB requires CMake, Python3 and a C++11 compliant compiler. ... -
16
KuzuDB
Embeddable property graph database management system
KuzuDB is a high-performance graph database optimized for analytical queries, built from the ground up with a columnar storage engine. It is designed to efficiently process large-scale graph workloads, making it ideal for data science, machine learning, and knowledge graph applications. -
17
StarRocks
StarRocks is a next-gen sub-second MPP database for full analytics
StarRocks is the next generation of real-time SQL engines for enterprise analytics. Real-time analytics is notoriously difficult. Complex data pipelines and de-normalized tables have always been a necessary evil. Processing any updates or deletes once data arrives has not been possible- until now. StarRocks solves these challenges and makes real-time analytics easy. Get amazing query performance on Star or Snowflake Schemas directly. From canceled orders to updated items, your analytics applications can easily handle them with StarRocks. ... -
18
GreptimeDB
An open-source, cloud-native, unified time series database for metrics
GreptimeDB treats all time series as contextual events with timestamps, and thus unifies the processing of metrics, logs, and events. It supports analyzing metrics, logs, and events with SQL, PromQL, and streaming with continuous aggregation. GreptimeDB is a time-series database optimized for storing and querying large amounts of time-series data, commonly used in monitoring and IoT applications. -
19
Lantern Database
PostgreSQL vector database extension for building AI applications
Lantern is a real-time data transformation engine that enables data engineers to build, run, and monitor streaming data pipelines with SQL. It’s designed to process events in motion, offering low-latency stream transformations, aggregations, and enrichment in a declarative way. Lantern is especially suited for modern data infrastructure and analytics platforms. -
20
Logstash
Centralize, transform and stash your data
Logstash is a server-side data processing pipeline that dynamically ingests data from numerous sources, transforms it, and ships it to your favorite “stash” regardless of format or complexity. It supports and ingests data of all shapes, sizes and sources, dynamically transforms and prepares this data, and transports it to the output of your choice. Logstash is extensible, with over 200 plugins available to let you create and configure your pipeline how you choose. -
21
CogDB
Micro Graph Database for Python Applications
Cog is a lightweight, embedded graph database for Go that provides a simple interface for storing and querying graph-based data structures, making it useful for knowledge representation and graph analytics. -
22
SuperDuperDB
Integrate, train and manage any AI models and APIs with your database
...Integrate and combine models from Sklearn, PyTorch, HuggingFace with AI APIs such as OpenAI to build even the most complex AI applications and workflows. Train models on your data in your datastore simply by querying without additional ingestion and pre-processing. -
23
Tarantool
Get your data in RAM, get compute close to data, enjoy the performance
In OLTP scenarios, Tarantool can be used instead of relational databases. Such a solution will work many times faster. With Tarantool, you can replace the traditional bundle of database & cache and reduce operational costs. Tarantool is tolerant of write-heavy loads. It also allows keeping full-featured applications close to the data, thus reducing data access network latency to zero. The open-source Community Edition lets you develop applications and speed up a system in operation. It... -
24
PolarDB-X
PolarDB-X is a cloud native distributed SQL Database
PolarDB-X is a cloud-native distributed SQL database designed to handle high concurrency, massive storage, and complex querying scenarios. It features a shared-nothing architecture that decouples computing from storage, providing scalability and flexibility for various applications. -
25
CDC Connectors for Apache Flink
CDC Connectors for Apache Flink
This project provides a set of source connectors for Apache Flink® directly ingesting changes coming from different databases using Change Data Capture(CDC). CDC Connectors for Apache Flink® is a set of source connectors for Apache Flink®, ingesting changes from different databases using change data capture (CDC). CDC Connectors for Apache Flink® integrates Debezium as the engine to capture data changes. So it can fully leverage the ability of Debezium. Supports reading database snapshot and continues to read transaction logs with exactly-once processing even failures happen. ...
