Search Results for "data"
Sort By:
Showing 44 open source projects for "data"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Go from Data Warehouse to Data and AI platform with BigQueryBigQuery is more than a data warehouse—it's an autonomous data-to-AI platform. Use familiar SQL to train ML models, run time-series forecasts, and generate AI-powered insights with native Gemini integration. Built-in agents handle data engineering and data science workflows automatically. Get $300 in free credit, query 1 TB, and store 10 GB free monthly.
-
1
Conduit
Conduit streams data between data stores. Kafka Connect replacement
Conduit is a data streaming tool written in Go. It aims to provide the best user experience for building and running real-time data pipelines. Conduit comes with batteries included, it provides a UI, common connectors, processors and observability data out of the box. Sync data between your production systems using an extensible, event-first experience with minimal dependencies that fit within your existing workflow. -
2
Arize Phoenix
Uncover insights, surface problems, monitor, and fine tune your LLM
Phoenix provides ML insights at lightning speed with zero-config observability for model drift, performance, and data quality. Phoenix is an Open Source ML Observability library designed for the Notebook. The toolset is designed to ingest model inference data for LLMs, CV, NLP and tabular datasets. It allows Data Scientists to quickly visualize their model data, monitor performance, track down issues & insights, and easily export to improve. Deep Learning Models (CV, LLM, and Generative) are an amazing technology that will power many of future ML use cases. ... -
3
Grafana
Leading open-source visualization and observability platform
Grafana OSS is a leading open-source visualization and observability platform that lets you query, visualize, alert on, and explore your data—regardless of where it’s stored. With support for 100+ data source plugins (such as Prometheus, Loki, Elasticsearch, InfluxDB, SQL/NoSQL databases, OTel, and more), you can unify metrics, logs, traces, and other observability signals in one place. Grafana OSS empowers you to build dynamic, reusable dashboards with rich visualizations, template variables, interactive filtering, and cross-panel linking. ... -
4
fluentbit
Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX
...No more OOM errors! Integration with all your technology, cloud-native services, containers, streaming processors, and data backends. Fully event-driven design leverages the operating system API for performance and reliability. All operations to collect and deliver data are asynchronous. -
Build AI Apps with Gemini 3 on Vertex AIVertex AI gives developers access to Gemini 3—Google’s most advanced reasoning and coding model—plus 200+ foundation models including Claude, Llama, and Gemma. Build generative AI apps with Vertex AI Studio, customize with fine-tuning, and deploy to production with enterprise-grade MLOps. New customers get $300 in free credits.
-
5
Elementary
Open-source data observability for analytics engineers
Elementary is an open-source data observability solution for data & analytics engineers. Monitor your dbt project and data in minutes, and be the first to know of data issues. Gain immediate visibility, detect data issues, send actionable alerts, and understand the impact and root cause. Generate a data observability report, host it or share with your team. Monitoring of data quality metrics, freshness, volume and schema changes, including anomaly detection. ... -
6
Fluid
Fluid, elastic data abstraction and acceleration for BigData/AI apps
Fluid, elastic data abstraction and acceleration for BigData/AI applications in the cloud. Provide DataSet abstraction for underlying heterogeneous data sources with multidimensional management in a cloud environment. Enable dataset warmup and acceleration for data-intensive applications by using a distributed cache in Kubernetes with observability, portability, and scalability. -
7
ODD Platform
First open-source data discovery and observability platform
Unlock the power of big data with OpenDataDiscovery Platform. Experience seamless end-to-end insights, powered by unprecedented observability and trust - from ingestion to production - while building your ideal tech stack! Democratize data and accelerate insights. Find data that fits your use case and discover hints left by your peers to leverage existing knowledge. -
8
Dagster
An orchestration platform for the development, production
...Dagster as a unified control plane: The ‘single plane of glass’ data teams love to use. Rein in the chaos and maintain control over your data as the complexity scales. Centralize your metadata in one tool with built-in observability, diagnostics, cataloging, and lineage. Spot any issues and identify performance improvement opportunities. -
9
Elastic APM Node.js Agent
Elastic APM Node.js Agent
This is the official Node.js application performance monitoring (APM) agent for the Elastic Observability solution. It is a Node.js package that runs with your Node.js application to automatically capture errors, tracing data, and performance metrics. APM data is sent to your Elastic Observability deployment -- hosted in Elastic's cloud or in your own on-premises deployment -- where you can monitor your application, create alerts, and quick identify root causes of service issues. First, you will need an Elastic Stack deployment. This is a deployment of APM Server (which receives APM data from the APM agent running in your application), Elasticsearch (the database that stores all APM data), and Kibana (the application that provides the interface to visualize and analyze the data). ... -
Managed MySQL, PostgreSQL, and SQL Databases on Google CloudCloud SQL is a fully managed relational database for MySQL, PostgreSQL, and SQL Server. We handle patching, backups, replication, encryption, and failover—so you can focus on your app. Migrate from on-prem or other clouds with free Database Migration Service. IDC found customers achieved 246% ROI. New customers get $300 in credits plus a 30-day free trial.
-
10
Grafana Pyroscope
Continuous Profiling Platform. Debug performance issues
...Collect, store, and analyze profiles from various external profiling tools in one central location. Link to your Open Telemetry tracing data and get request-specific or span-specific profiles to enhance other observability data like traces and logs. -
11
Alibaba iLogtail
Fast and Lightweight Observability Data Collector
iLogtail was born for observable scenarios and has many production-level features such as lightweight, high performance, and automated configuration, which are widely used internally by Alibaba Group and tens of thousands of external Alibaba Cloud customers. You can deploy it in physical machines, Kubernetes and other environments to collect telemetry data, such as logs, traces and metrics. Supports a variety of Logs, Traces, and Metrics data collection, and is friendly to container and Kubernetes environment support. The resource cost of data collection is quite low, 5-20 times better than similar telemetry data collection Agent performance. High stability, used in the production of Alibaba and tens of thousands of Alibaba Cloud customers, and collecting dozens of petabytes of observable data every day with nearly tens of millions deployments. -
12
memphis
Next-Generation Event Processing Platform
Memphis enables building modern queue-based applications that require large volumes of streamed and enriched data, modern protocols, zero ops, up to x9 faster development, up to x46 fewer costs, and significantly lower dev time for data-oriented developers and data engineers. Queues and brokers are a mission-critical component in the modern application architecture and should be highly available and stable as possible. Provide great performance while maintaining efficient resource consumption. ... -
13
Loggie
A lightweight, cloud-native data transfer agent and aggregator
Loggie is a lightweight, high-performance, cloud-native agent and aggregator based on Golang. Loggie includes LogConfig/ClusterLogConfig/Interceptor/Sink CRDs, allowing for the creation of data collection, transfer, processing, and sending pipelines through simple YAML file creation. Supports deployment as an independent intermediate machine, which can receive aggregated data sent by Loggie Agent and can also be used to consume and process various data sources. Configure Filebeat and Loggie to collect logs, and send them to a Kafka topic without using client compression, with the Kafka topic partition configured as 3. ... -
14
OpenTelemetry Collector
OpenTelemetry Collector
The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry data. In addition, it removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source telemetry data formats (e.g. Jaeger, Prometheus, etc.) to multiple open-source or commercial back-ends. -
15
qryn
All-in-one Polyglot Observability stack with ClickHouse storage
...Let's get Polyglot. qryn independently implements popular observability standards, protocols and query languages. Make sure you have sufficient memory and disk resources allocated for your node service and clickhouse server when dealing with large amounts of data and fingerprints. We suggest 8GB RAM or higher for most setups with 100k-1M fingerprints. Observe your daily and weekly data consumption to forecast your disk usage requirements. Compression codecs and other optimizations can be performed at the ClickHouse level. -
16
OpenObserve
Elasticsearch/Splunk/Datadog alternative for (logs, metrics, traces)
...OpenObserve stored data in Amazon s3 and Elasticsearch stored data on Amazon EBS volumes. -
17
QuestDB
An open source SQL database designed to process time series data
...It includes endpoints for PostgreSQL wire protocol, high-throughput schema-agnostic ingestion using InfluxDB Line Protocol, and a REST API for queries, bulk imports, and exports. QuestDB implements ANSI SQL with native extensions for time-oriented language features. These extensions make it simple to correlate data from multiple sources using relational and time series joins. QuestDB achieves high performance from a column-oriented storage model, massively-parallelized vector execution, SIMD instructions, and various low-latency techniques. The entire codebase was built from the ground up in Java and C++, with no dependencies, and is 100% free from garbage collection. ... -
18
SigNoz
SigNoz is an open-source APM. It helps developers monitor their apps
...You can find the root cause of the problem by going to the exact traces which are causing the problem and see detailed flamegraphs of individual request traces. Run aggregates on trace data to get business relevant metrics. Filter and query logs, build dashboards and alerts based on attributes in logs. -
19
HyperDX
An open source observability platform unifying session replays & logs
HyperDX helps engineers figure out why production is broken faster by centralizing and correlating logs, metrics, traces, exceptions and session replays in one place. An open-source and developer-friendly alternative to Datadog and New Relic. The HyperDX stack ingests, stores, and searches/graphs your telemetry data. After standing up the Docker Compose stack, you'll want to instrument your app to send data over to HyperDX. -
20
Vector
A high-performance observability data pipeline
Vector is a Rust‑based, high‑performance observability data pipeline tool (agent + aggregator) designed to collect, transform, and route logs and metrics at scale. Created by Datadog, it aims to be the only tool needed from ingestion to vendor output, providing cost-efficient, safe, and flexible telemetry processing. -
21
OpenTelemetry
OpenTelemetry Go API and SDK
OpenTelemetry-Go is the Go implementation of OpenTelemetry. It provides a set of APIs to directly measure the performance and behavior of your software and send this data to observability platforms. High-quality, ubiquitous, and portable telemetry to enable effective observability. OpenTelemetry is a collection of APIs, SDKs, and tools. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior. -
22
Jaeger
Monitor and troubleshoot transactions in complex distributed systems
...Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system released as open source by Uber Technologies. It is used for monitoring and troubleshooting microservices-based distributed systems. OpenTracing compatible data model and instrumentation libraries include Go, Java, Node, Python, C++ and C#. Jaeger uses consistent upfront sampling with individual per service/endpoint probabilities and it has multiple storage backends: Cassandra, Elasticsearch, memory. -
23
BFE
A modern layer 7 load balancer from baidu
...BFE provides a rich set of plugins for traffic management, security, observability, etc. BFE includes detailed built-in metrics for all subsystems. BFE writes various logs for trouble shooting, data analysis and visualization. BFE also supports distributed tracing. -
24
Jina
Build cross-modal and multimodal applications on the cloud
...Fast deployment to Kubernetes, Docker Compose and Jina Cloud. Improved engineering efficiency thanks to the Jina AI ecosystem, so you can focus on innovating with the data applications you build. -
25
Pixie
Instant Kubernetes-Native Application Observability
...Use Pixie to view the high-level state of your cluster (service maps, cluster resources, application traffic) and also drill down into more detailed views (pod state, flame graphs, individual full-body application requests). Pixie uses eBPF to automatically collect telemetry data such as full-body requests, resource and network metrics, application profiles, and more. Pixie collects, stores and queries all telemetry data locally in the cluster. Pixie uses less than 5% of cluster CPU and in most cases less than 2%. PxL, Pixie’s flexible Pythonic query language, can be used across Pixie’s UI, CLI, and client APIs.
