Alternatives to Amazon MWAA
Compare Amazon MWAA alternatives for your business or organization using the curated list below. SourceForge ranks the best alternatives to Amazon MWAA in 2026. Compare features, ratings, user reviews, pricing, and more from Amazon MWAA competitors and alternatives in order to make an informed decision for your business.
-
1
DataBahn
DataBahn
DataBahn.ai is redefining how enterprises manage the explosion of security and operational data in the AI era. Our AI-powered data pipeline and fabric platform helps organizations securely collect, enrich, orchestrate, and optimize enterprise data—including security, application, observability, and IoT/OT telemetry—for analytics, automation, and AI. With native support for over 400 integrations and built-in enrichment capabilities, DataBahn streamlines fragmented data workflows and reduces SIEM and infrastructure costs from day one. The platform requires no specialist training, enabling security and IT teams to extract insights in real time and adapt quickly to new demands. We've helped Fortune 500 and Global 2000 companies reduce data processing costs by over 50% and automate more than 80% of their data engineering workloads. -
2
Managed Service for Apache Airflow is a fully managed workflow orchestration platform from Google Cloud built on the open-source Apache Airflow project. It allows users to author, schedule, and monitor data pipelines using Python-based workflows known as DAGs. The platform eliminates the need to manage infrastructure, enabling teams to focus on building and running pipelines. It integrates seamlessly with Google Cloud services such as BigQuery, Dataflow, and Managed Service for Apache Spark. It also supports hybrid and multi-cloud environments, allowing workflows to span across different systems. Users benefit from built-in monitoring, logging, and troubleshooting tools for reliability. The service is designed to simplify complex data workflows, including ETL, MLOps, and automation tasks. Overall, it provides a scalable and flexible solution for orchestrating modern data pipelines.Starting Price: $0.074 per vCPU hour
-
3
Apache Airflow
The Apache Software Foundation
Airflow is a platform created by the community to programmatically author, schedule and monitor workflows. Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to infinity. Airflow pipelines are defined in Python, allowing for dynamic pipeline generation. This allows for writing code that instantiates pipelines dynamically. Easily define your own operators and extend libraries to fit the level of abstraction that suits your environment. Airflow pipelines are lean and explicit. Parametrization is built into its core using the powerful Jinja templating engine. No more command-line or XML black-magic! Use standard Python features to create your workflows, including date time formats for scheduling and loops to dynamically generate tasks. This allows you to maintain full flexibility when building your workflows. -
4
Astro by Astronomer
Astronomer
For data teams looking to increase the availability of trusted data, Astronomer provides Astro, a modern data orchestration platform, powered by Apache Airflow, that enables the entire data team to build, run, and observe data pipelines-as-code. Astronomer is the commercial developer of Airflow, the de facto standard for expressing data flows as code, used by hundreds of thousands of teams across the world. -
5
Yandex Data Proc
Yandex
You select the size of the cluster, node capacity, and a set of services, and Yandex Data Proc automatically creates and configures Spark and Hadoop clusters and other components. Collaborate by using Zeppelin notebooks and other web apps via a UI proxy. You get full control of your cluster with root permissions for each VM. Install your own applications and libraries on running clusters without having to restart them. Yandex Data Proc uses instance groups to automatically increase or decrease computing resources of compute subclusters based on CPU usage indicators. Data Proc allows you to create managed Hive clusters, which can reduce the probability of failures and losses caused by metadata unavailability. Save time on building ETL pipelines and pipelines for training and developing models, as well as describing other iterative tasks. The Data Proc operator is already built into Apache Airflow.Starting Price: $0.19 per hour -
6
DoubleCloud
DoubleCloud
Save time & costs by streamlining data pipelines with zero-maintenance open source solutions. From ingestion to visualization, all are integrated, fully managed, and highly reliable, so your engineers will love working with data. You choose whether to use any of DoubleCloud’s managed open source services or leverage the full power of the platform, including data storage, orchestration, ELT, and real-time visualization. We provide leading open source services like ClickHouse, Kafka, and Airflow, with deployment on Amazon Web Services or Google Cloud. Our no-code ELT tool allows real-time data syncing between systems, fast, serverless, and seamlessly integrated with your existing infrastructure. With our managed open-source data visualization you can simply visualize your data in real time by building charts and dashboards. We’ve designed our platform to make the day-to-day life of engineers more convenient.Starting Price: $0.024 per 1 GB per month -
7
TensorStax
TensorStax
TensorStax is an AI-powered platform that automates data engineering tasks, enabling businesses to efficiently manage data pipelines, database migrations, ETL/ELT processes, and data ingestion within their cloud infrastructure. Its autonomous agents integrate seamlessly with existing tools like Airflow and dbt, facilitating end-to-end pipeline development and proactive issue detection to minimize downtime. Deployed within a company's Virtual Private Cloud (VPC), TensorStax ensures data security and privacy. By automating complex data workflows, it allows teams to focus on strategic analysis and decision-making. -
8
CData Python Connectors
CData Software
CData Python Connectors simplify the way that Python users connect to SaaS, Big Data, NoSQL, and relational data sources. Our Python Connectors offer simple Python database interfaces (DB-API), making it easy to connect with popular tooling like Jupyter Notebook, SQLAlchemy, pandas, Dash, Apache Airflow, petl, and more. CData Python Connectors create a SQL wrapper around APIs and data protocols, simplifying data access from within Python and enabling Python users to easily connect more than 150 SaaS, Big Data, NoSQL, and relational data sources with advanced Python processing. The CData Python Connectors fill a critical gap in Python tooling by providing consistent connectivity with data-centric interfaces to hundreds of different SaaS/Cloud, NoSQL, and Big Data sources. Download a 30-day free trial or learn more at: https://www.cdata.com/python/ -
9
Locus
EQ Works
With multiple environments to work in, Locus provides a streamlined method of deep analysis of geospatial data for everyone from tech-challenged marketers, to deep query analysis for data scientists and analysts, to top-level metrics for data-driven execs hungry to find their next success. This provides for the most secure, efficient and seamless way to connect other data sources or your data lake to LOCUS. Connection Hub has integrated data lineage governance and transformation capabilities built-in to allow for further integration with tools such as LOCUS Notebook and LOCUS QL. EQ builds its own directed acyclical graph processor on top of the popular Apache Airflow framework. The DAG Builder has been engineered to crunch (and munch) your geospatial workflows with over twenty (20) built-in helper stages. -
10
Prophecy
Prophecy
Prophecy enables many more users - including visual ETL developers and Data Analysts. All you need to do is point-and-click and write a few SQL expressions to create your pipelines. As you use the Low-Code designer to build your workflows - you are developing high quality, readable code for Spark and Airflow that is committed to your Git. Prophecy gives you a gem builder - for you to quickly develop and rollout your own Frameworks. Examples are Data Quality, Encryption, new Sources and Targets that extend the built-in ones. Prophecy provides best practices and infrastructure as managed services – making your life and operations simple! With Prophecy, your workflows are high performance and use scale-out performance & scalability of the cloud.Starting Price: $299 per month -
11
AKL FlowDesigner
AKL
AKL FlowDesigner is a computational fluid dynamics (CFD) simulation software that enables wind analysis by easily importing 3D models of buildings or urban blocks developed by modeling tools such as Autodesk Revit, GRAPHISOFT ARCHICAD, Rhinoceros and SketchUp. AKL FlowDesigner is BIM capable in IFC format. Architects, designers, engineers and consultants can use AKL FlowDesigner early in the design process to understand and visualize airflow around their projects. Early analysis reduces design time and gives a powerful impression to clients. CFD simulation of AKL FlowDesigner gives you the maximum benefit in the application of AEC (architecture, engineering and construction) industry. Airflow simulation used to be a complex and time-consuming task requiring deep technical knowledge. No more! AKL FlowDesigner allows any user to create simulations and analyze airflow in minutes. No engineering degree or complex calculations are required! -
12
Conduktor
Conduktor
We created Conduktor, the all-in-one friendly interface to work with the Apache Kafka ecosystem. Develop and manage Apache Kafka with confidence. With Conduktor DevTools, the all-in-one Apache Kafka desktop client. Develop and manage Apache Kafka with confidence, and save time for your entire team. Apache Kafka is hard to learn and to use. Made by Kafka lovers, Conduktor best-in-class user experience is loved by developers. Conduktor offers more than just an interface over Apache Kafka. It provides you and your teams the control of your whole data pipeline, thanks to our integration with most technologies around Apache Kafka. Provide you and your teams the most complete tool on top of Apache Kafka. -
13
Prefect
Prefect
Prefect is a workflow orchestration and automation platform designed for the modern context-driven era. It enables teams to turn Python functions into production-ready workflows with minimal effort. Prefect provides open-source foundations alongside managed platforms for enterprise-scale automation. The platform supports building and orchestrating data pipelines, workflows, and AI applications with full observability. Prefect Cloud offers managed orchestration with autoscaling, enterprise authentication, and built-in governance. Prefect Horizon extends automation to AI infrastructure by enabling deployment of MCP servers for AI agents. Trusted by leading organizations, Prefect helps teams scale automation without operational complexity. -
14
Amazon MSK
Amazon
Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that makes it easy for you to build and run applications that use Apache Kafka to process streaming data. Apache Kafka is an open-source platform for building real-time streaming data pipelines and applications. With Amazon MSK, you can use native Apache Kafka APIs to populate data lakes, stream changes to and from databases, and power machine learning and analytics applications. Apache Kafka clusters are challenging to setup, scale, and manage in production. When you run Apache Kafka on your own, you need to provision servers, configure Apache Kafka manually, replace servers when they fail, orchestrate server patches and upgrades, architect the cluster for high availability, ensure data is durably stored and secured, setup monitoring and alarms, and carefully plan scaling events to support load changes.Starting Price: $0.0543 per hour -
15
Cake AI
Cake AI
Cake AI is a comprehensive AI infrastructure platform that enables teams to build and deploy AI applications using hundreds of pre-integrated open source components, offering complete visibility and control. It provides a curated, end-to-end selection of fully managed, best-in-class commercial and open source AI tools, with pre-built integrations across the full breadth of components needed to move an AI application into production. Cake supports dynamic autoscaling, comprehensive security measures including role-based access control and encryption, advanced monitoring, and infrastructure flexibility across various environments, including Kubernetes clusters and cloud services such as AWS. Its data layer equips teams with tools for data ingestion, transformation, and analytics, leveraging tools like Airflow, DBT, Prefect, Metabase, and Superset. For AI operations, Cake integrates with model catalogs like Hugging Face and supports modular workflows using LangChain, LlamaIndex, and more. -
16
OpenSnowcat
OpenSnowcat
OpenSnowcat is an open source fork of Snowplow under the Apache 2.0 License that delivers a full event data pipeline for collection, enrichment, routing, and loading, remaining fully compatible with Snowplow and Segment SDKs. It provides an end-to-end solution to collect behavioral data from web and mobile sources, enrich it with customizable processes, route events through modern integrations, and load enriched data into destinations such as Snowflake, Redshift, S3, Amplitude, Kinesis, and more, with support for JSON and TSV output formats. OpenSnowcat emphasizes being free and open source forever with a trusted license, aiming for security, stability, and backward compatibility so existing Snowplow implementations can continue without disruption. Its architecture is designed for high performance, minimal latency, and dynamic scalability, integrating with cloud services to simplify management and cost efficiency at scale.Starting Price: Free -
17
Data Flow Manager
Ksolves
Data Flow Manager is an Agentic AI Control Plane for Apache NiFi Operations, built for enterprises running NiFi at real scale. Run, manage, and fix NiFi challenges across all clusters, environments, and flows using simple natural-language prompts. One platform. One control plane. Zero firefighting. DFM replaces fragmented UIs, brittle scripts, and reactive operations with centralized, AI-driven control, enabling NiFi teams to transition from manual operations to governed, autonomous execution. What DFM delivers: • Centralized control across all NiFi clusters and environments • Prompt-driven flow deployment and promotion • Pre-deploy flow validation & sanity checks • Scheduled and controlled flow deployments • Centralized controller service management • Built-in approval workflows and RBAC • Immutable, detailed audit logs • Unified visibility into flow health and runtime state -
18
Power IQ
Raritan
Power IQ® DCIM Monitoring Software enables data center and facility managers to closely monitor and efficiently utilize their existing data center power infrastructure. Data center health maps, power analytics, cooling charts, and reports alert you to potential trouble and help you understand real-time power load, trends, and capacity at all levels of infrastructure. A configurable dashboard provides vendor-agnostic views of power capacity, environmental health, and energy consumption. Get single-click access to rack power, cooling, airflow, events, and much more. A complete environment management solution that helps you to identify potential trouble areas, save energy, and maintain a safe environment for your IT equipment. Consolidate the names, polling status, locations, models and firmware for all your rack power distribution units (PDUs) onto one screen and save valuable management time that can be allocated to other priorities. -
19
Azure Event Hubs
Microsoft
Event Hubs is a fully managed, real-time data ingestion service that’s simple, trusted, and scalable. Stream millions of events per second from any source to build dynamic data pipelines and immediately respond to business challenges. Keep processing data during emergencies using the geo-disaster recovery and geo-replication features. Integrate seamlessly with other Azure services to unlock valuable insights. Allow existing Apache Kafka clients and applications to talk to Event Hubs without any code changes—you get a managed Kafka experience without having to manage your own clusters. Experience real-time data ingestion and microbatching on the same stream. Focus on drawing insights from your data instead of managing infrastructure. Build real-time big data pipelines and respond to business challenges right away.Starting Price: $0.03 per hour -
20
Kestra
Kestra
Kestra is an open-source, event-driven orchestrator that simplifies data operations and improves collaboration between engineers and business users. By bringing Infrastructure as Code best practices to data pipelines, Kestra allows you to build reliable workflows and manage them with confidence. Thanks to the declarative YAML interface for defining orchestration logic, everyone who benefits from analytics can participate in the data pipeline creation process. The UI automatically adjusts the YAML definition any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is defined declaratively in code, even if some workflow components are modified in other ways. -
21
GlassFlow
GlassFlow
GlassFlow is a serverless, event-driven data pipeline platform designed for Python developers. It enables users to build real-time data pipelines without the need for complex infrastructure like Kafka or Flink. By writing Python functions, developers can define data transformations, and GlassFlow manages the underlying infrastructure, offering auto-scaling, low latency, and optimal data retention. The platform supports integration with various data sources and destinations, including Google Pub/Sub, AWS Kinesis, and OpenAI, through its Python SDK and managed connectors. GlassFlow provides a low-code interface for quick pipeline setup, allowing users to create and deploy pipelines within minutes. It also offers features such as serverless function execution, real-time API connections, and alerting and reprocessing capabilities. The platform is designed to simplify the creation and management of event-driven data pipelines, making it accessible for Python developers.Starting Price: $350 per month -
22
Managed Service for Apache Spark is a Google Cloud solution that simplifies running Apache Spark workloads with either serverless execution or fully managed clusters. It allows users to process large-scale data without needing to manage infrastructure, reducing operational complexity. The platform features Lightning Engine, which accelerates Spark performance by up to 4.9 times compared to open-source Spark. It supports data engineering, data science, and machine learning workflows at scale. Integration with Gemini enables AI-powered development, including automated code generation and troubleshooting. The service works seamlessly with open data formats like Apache Iceberg and integrates with tools like BigQuery and Knowledge Catalog. It offers flexible deployment options to suit different workloads and use cases. Overall, it provides a faster, smarter, and more efficient way to run Spark workloads in the cloud.
-
23
rudol
rudol
Unify your data catalog, reduce communication overhead and enable quality control to any member of your company, all without deploying or installing anything. rudol is a data quality platform that helps companies understand all their data sources, no matter where they come from; reduces excessive communication in reporting processes or urgencies; and enables data quality diagnosing and issue prevention to all the company, through easy steps With rudol, each organization is able to add data sources from a growing list of providers and BI tools with a standardized structure, including MySQL, PostgreSQL, Airflow, Redshift, Snowflake, Kafka, S3*, BigQuery*, MongoDB*, Tableau*, PowerBI*, Looker* (* in development). So, regardless of where it’s coming from, people can understand where and how the data is stored, read and collaborate with its documentation, or easily contact data owners using our integrations.Starting Price: $0 -
24
Amazon EMR
Amazon
Amazon EMR is the industry-leading cloud big data platform for processing vast amounts of data using open-source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, and Presto. With EMR you can run Petabyte-scale analysis at less than half of the cost of traditional on-premises solutions and over 3x faster than standard Apache Spark. For short-running jobs, you can spin up and spin down clusters and pay per second for the instances used. For long-running workloads, you can create highly available clusters that automatically scale to meet demand. If you have existing on-premises deployments of open-source tools such as Apache Spark and Apache Hive, you can also run EMR clusters on AWS Outposts. Analyze data using open-source ML frameworks such as Apache Spark MLlib, TensorFlow, and Apache MXNet. Connect to Amazon SageMaker Studio for large-scale model training, analysis, and reporting. -
25
Apache Kafka
The Apache Software Foundation
Apache Kafka® is an open-source, distributed streaming platform. Scale production clusters up to a thousand brokers, trillions of messages per day, petabytes of data, hundreds of thousands of partitions. Elastically expand and contract storage and processing. Stretch clusters efficiently over availability zones or connect separate clusters across geographic regions. Process streams of events with joins, aggregations, filters, transformations, and more, using event-time and exactly-once processing. Kafka’s out-of-the-box Connect interface integrates with hundreds of event sources and event sinks including Postgres, JMS, Elasticsearch, AWS S3, and more. Read, write, and process streams of events in a vast array of programming languages. -
26
BigBI
BigBI
BigBI enables data specialists to build their own powerful big data pipelines interactively & efficiently, without any coding! BigBI unleashes the power of Apache Spark enabling: Scalable processing of real Big Data (up to 100X faster) Integration of traditional data (SQL, batch files) with modern data sources including semi-structured (JSON, NoSQL DBs, Elastic, Hadoop), and unstructured (Text, Audio, video), Integration of streaming data, cloud data, AI/ML & graphs -
27
Google Cloud Dataflow
Google
Unified stream and batch data processing that's serverless, fast, and cost-effective. Fully managed data processing service. Automated provisioning and management of processing resources. Horizontal autoscaling of worker resources to maximize resource utilization. OSS community-driven innovation with Apache Beam SDK. Reliable and consistent exactly-once processing. Streaming data analytics with speed. Dataflow enables fast, simplified streaming data pipeline development with lower data latency. Allow teams to focus on programming instead of managing server clusters as Dataflow’s serverless approach removes operational overhead from data engineering workloads. Allow teams to focus on programming instead of managing server clusters as Dataflow’s serverless approach removes operational overhead from data engineering workloads. Dataflow automates provisioning and management of processing resources to minimize latency and maximize utilization. -
28
SmartAC
SmartAC
SmartAC is an end-to-end HVAC monitoring and membership growth platform that connects heating and cooling systems to cloud-based analytics through wireless sensors and mobile software. It enables contractors and homeowners to move from reactive repairs to proactive system management by continuously tracking performance metrics such as temperature, airflow, filter health, and potential water leaks. SmartAC’s sensor network installs quickly without wiring and sends real-time data to the cloud, where machine learning analyzes system behavior and alerts users before failures occur. It includes a white-labeled homeowner app for scheduling, service credits, and system visibility, along with technician tools and a contractor dashboard that surfaces revenue opportunities and maintenance needs. By combining predictive maintenance, automated alerts, and customer engagement features, SmartAC helps HVAC businesses increase membership adoption, improve retention, and more.Starting Price: Free -
29
Datavolo
Datavolo
Capture all your unstructured data for all your LLM needs. Datavolo replaces single-use, point-to-point code with fast, flexible, reusable pipelines, freeing you to focus on what matters most, doing incredible work. Datavolo is the dataflow infrastructure that gives you a competitive edge. Get fast, unencumbered access to all of your data, including the unstructured files that LLMs rely on, and power up your generative AI. Get pipelines that grow with you, in minutes, not days, without custom coding. Instantly configure from any source to any destination at any time. Trust your data because lineage is built into every pipeline. Make single-use pipelines and expensive configurations a thing of the past. Harness your unstructured data and unleash AI innovation with Datavolo, powered by Apache NiFi and built specifically for unstructured data. Our founders have spent a lifetime helping organizations make the most of their data.Starting Price: $36,000 per year -
30
StreamNative
StreamNative
StreamNative redefines streaming infrastructure by seamlessly integrating Kafka, MQ, and other protocols into a single, unified platform, providing unparalleled flexibility and efficiency for modern data processing needs. StreamNative offers a unified solution that adapts to the diverse requirements of streaming and messaging in a microservices-driven environment. By providing a comprehensive and intelligent approach to messaging and streaming, StreamNative empowers organizations to navigate the complexities and scalability of the modern data ecosystem with efficiency and agility. Apache Pulsar’s unique architecture decouples the message serving layer from the message storage layer to deliver a mature cloud-native data-streaming platform. Scalable and elastic to adapt to rapidly changing event traffic and business needs. Scale-up to millions of topics with architecture that decouples computing and storage.Starting Price: $1,000 per month -
31
Airplane
Airplane
Let your customer-facing teams delete accounts, change emails, issue refunds, and more. Empower your customer success team to configure accounts for new customers. Make sure you're not the only one who knows how to run that script you wrote. Make sure sensitive operations are approved by a manager or admin before being executed. Run daily reports and other periodic operations without the headache of maintaining cron or Airflow. Kick-off data backfills and other long-running tasks and get notified when they’re complete. Go beyond security checkboxes. Audit logs show who ran what so you can stop guessing and stay informed. Give teammates access upon request. Require signoff for sensitive actions. Get notifications, approve requests, and execute runbooks without leaving Slack. Go beyond security checkboxes. Audit logs show who ran what so you can stop guessing and stay informed.Starting Price: $10 per user per month -
32
Dataplane
Dataplane
The concept behind Dataplane is to make it quicker and easier to construct a data mesh with robust data pipelines and automated workflows for businesses and teams of all sizes. In addition to being more user friendly, there has been an emphasis on scaling, resilience, performance and security.Starting Price: Free -
33
Ansys Icepak
Ansys
Ansys Icepak is a CFD solver for electronics thermal management. It predicts airflow, temperature, and heat transfer in IC packages, PCBs, electronic assemblies/enclosures, and power electronics. Ansys Icepak provides powerful electronic cooling solutions that utilize the industry-leading Ansys Fluent computational fluid dynamics (CFD) solver for thermal and fluid flow analyses of integrated circuits (ICs), packages, printed circuit boards (PCBs), and electronic assemblies. The Ansys Icepak CFD solver uses the Ansys Electronics Desktop (AEDT) graphical user interface (GUI). Perform conduction, convection, and radiation conjugate heat transfer analyses, with many advanced capabilities to model laminar and turbulent flows, and species analysis including radiation and convection. Ansys’ complete PCB design solution enables you to simulate PCBs, ICs, and packages and accurately evaluate an entire system. -
34
Dagster
Dagster Labs
Dagster is a next-generation orchestration platform for the development, production, and observation of data assets. Unlike other data orchestration solutions, Dagster provides you with an end-to-end development lifecycle. Dagster gives you control over your disparate data tools and empowers you to build, test, deploy, run, and iterate on your data pipelines. It makes you and your data teams more productive, your operations more robust, and puts you in complete control of your data processes as you scale. Dagster brings a declarative approach to the engineering of data pipelines. Your team defines the data assets required, quickly assessing their status and resolving any discrepancies. An assets-based model is clearer than a tasks-based one and becomes a unifying abstraction across the whole workflow.Starting Price: $0 -
35
Parameter
Parameter
Parameter is a comprehensive monitoring tool built to help mission-critical facilities detect, analyze, and act on environmental, leak, and equipment health issues before they cause failures. It integrates data from RLE leak detection systems, battery monitoring (Cellwatch), and environmental monitoring into a unified system that feeds real-time information to building management systems (BMS) using native protocols like SNMP, BACnet, and Modbus. It supports scalable monitoring from single racks to hyperscale operations, offering precise detection of fluid leaks with location mapping, continuous tracking of temperature, humidity, airflow, CO2, and power conditions, and advanced battery health insights for UPS and backup systems. Parameter’s solutions provide real-time alerts and actionable data through web interfaces accessible from any browser or device, enabling rapid response and preemptive maintenance to maximize uptime. -
36
Thousands of customers use Amazon Managed Service for Apache Flink to run stream processing applications. With Amazon Managed Service for Apache Flink, you can transform and analyze streaming data in real-time using Apache Flink and integrate applications with other AWS services. There are no servers and clusters to manage, and there is no computing and storage infrastructure to set up. You pay only for the resources you use. Build and run Apache Flink applications, without setting up infrastructure and managing resources and clusters. Process gigabytes of data per second with subsecond latencies and respond to events in real-time. Deploy highly available and durable applications with Multi-AZ deployments and APIs for application lifecycle management. Develop applications that transform and deliver data to Amazon Simple Storage Service (Amazon S3), Amazon OpenSearch Service, and more.Starting Price: $0.11 per hour
-
37
CloverDX
CloverDX
Design, debug, run and troubleshoot data transformations and jobflows in a developer-friendly visual designer. Orchestrate data workloads that require tasks to be carried out in the right sequence, orchestrate multiple systems with the transparency of visual workflows. Deploy data workloads easily into a robust enterprise runtime environment. In cloud or on-premise. Make data available to people, applications and storage under a single unified platform. Manage your data workloads and related processes together in a single platform. No task is too complex. We’ve built CloverDX on years of experience with large enterprise projects. Developer-friendly open architecture and flexibility lets you package and hide the complexity for non-technical users. Manage the entire lifecycle of a data pipeline from design, deployment to evolution and testing. Get things done fast with the help of our in-house customer success teams.Starting Price: $5000.00/one-time -
38
Dataform
Google
Dataform enables data analysts and data engineers to develop and operationalize scalable data transformation pipelines in BigQuery using only SQL from a single, unified environment. Its open source core language lets teams define table schemas, configure dependencies, add column descriptions, and set up data quality assertions within a shared code repository while applying software development best practices, version control, environments, testing, and documentation. A fully managed, serverless orchestration layer automatically handles workflow dependencies, tracks lineage, and executes SQL pipelines on demand or via schedules in Cloud Composer, Workflows, BigQuery Studio, or third-party services. In the browser-based development interface, users get real-time error feedback, visualize dependency graphs, connect to GitHub or GitLab for commits and code reviews, and launch production-grade pipelines in minutes without leaving BigQuery Studio.Starting Price: Free -
39
Chalk
Chalk
Powerful data engineering workflows, without the infrastructure headaches. Complex streaming, scheduling, and data backfill pipelines, are all defined in simple, composable Python. Make ETL a thing of the past, fetch all of your data in real-time, no matter how complex. Incorporate deep learning and LLMs into decisions alongside structured business data. Make better predictions with fresher data, don’t pay vendors to pre-fetch data you don’t use, and query data just in time for online predictions. Experiment in Jupyter, then deploy to production. Prevent train-serve skew and create new data workflows in milliseconds. Instantly monitor all of your data workflows in real-time; track usage, and data quality effortlessly. Know everything you computed and data replay anything. Integrate with the tools you already use and deploy to your own infrastructure. Decide and enforce withdrawal limits with custom hold times.Starting Price: Free -
40
Bluon
Bluon
Bluon is the leading support app for HVAC technicians with the largest and fastest growing HVAC community! Easily find manuals, tech specs, standard and optional controls and discuss specific equipment for over 45,000 HVAC models. The Bluon app and support platform was created by technicians, for technicians. We know what you need to thrive in the rapidly changing HVAC industry. Access our searchable database of virtually every piece of HVAC equipment on planet Earth. Our expert technicians have extracted and organized every bit of useful information about each system. Easily bid each retrofit job. Man hours, refrigerant amount, and optional valves and control panel, customized for each unit. Get a breakdown of controls available on this unit, and possible adjustments in a Bluon retrofit. Get useful tools like superheat and subcool calcs, airflow diagnostics, pressure setpoint converter, and more. -
41
Apache Spark
Apache Software Foundation
Apache Spark™ is a unified analytics engine for large-scale data processing. Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. Spark offers over 80 high-level operators that make it easy to build parallel apps. And you can use it interactively from the Scala, Python, R, and SQL shells. Spark powers a stack of libraries including SQL and DataFrames, MLlib for machine learning, GraphX, and Spark Streaming. You can combine these libraries seamlessly in the same application. Spark runs on Hadoop, Apache Mesos, Kubernetes, standalone, or in the cloud. It can access diverse data sources. You can run Spark using its standalone cluster mode, on EC2, on Hadoop YARN, on Mesos, or on Kubernetes. Access data in HDFS, Alluxio, Apache Cassandra, Apache HBase, Apache Hive, and hundreds of other data sources. -
42
Airbyte
Airbyte
Airbyte is an open-source data integration platform designed to help businesses synchronize data from various sources to their data warehouses, lakes, or databases. The platform provides over 550 pre-built connectors and enables users to easily create custom connectors using low-code or no-code tools. Airbyte's solution is optimized for large-scale data movement, enhancing AI workflows by seamlessly integrating unstructured data into vector databases like Pinecone and Weaviate. It offers flexible deployment options, ensuring security, compliance, and governance across all models.Starting Price: $2.50 per credit -
43
Apache Accumulo
Apache Corporation
With Apache Accumulo, users can store and manage large data sets across a cluster. Accumulo uses Apache Hadoop's HDFS to store its data and Apache ZooKeeper for consensus. While many users interact directly with Accumulo, several open source projects use Accumulo as their underlying store. To learn more about Accumulo, take the Accumulo tour, read the user manual and run the Accumulo example code. Feel free to contact us if you have any questions. Accumulo has a programming mechanism (called Iterators) that can modify key/value pairs at various points in the data management process. Every Accumulo key/value pair has its own security label which limits query results based off user authorizations. Accumulo runs on a cluster using one or more HDFS instances. Nodes can be added or removed as the amount of data stored in Accumulo changes. -
44
Dafne
Adastra
Dafne is a data workflow & orchestration engine mainly designed for data warehouse automation (DWA). It simplifies the process of building, defining, scheduling, managing, and monitoring production workflows & ETLs, offering visibility, reliability, dependencies, priorities, and internal constraints to improve SLAs and performance. -
45
AWS Data Pipeline
Amazon
AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals. With AWS Data Pipeline, you can regularly access your data where it’s stored, transform and process it at scale, and efficiently transfer the results to AWS services such as Amazon S3, Amazon RDS, Amazon DynamoDB, and Amazon EMR. AWS Data Pipeline helps you easily create complex data processing workloads that are fault tolerant, repeatable, and highly available. You don’t have to worry about ensuring resource availability, managing inter-task dependencies, retrying transient failures or timeouts in individual tasks, or creating a failure notification system. AWS Data Pipeline also allows you to move and process data that was previously locked up in on-premises data silos.Starting Price: $1 per month -
46
Apache Hive
Apache Software Foundation
The Apache Hive data warehouse software facilitates reading, writing, and managing large datasets residing in distributed storage using SQL. Structure can be projected onto data already in storage. A command line tool and JDBC driver are provided to connect users to Hive. Apache Hive is an open source project run by volunteers at the Apache Software Foundation. Previously it was a subproject of Apache® Hadoop®, but has now graduated to become a top-level project of its own. We encourage you to learn about the project and contribute your expertise. Traditional SQL queries must be implemented in the MapReduce Java API to execute SQL applications and queries over distributed data. Hive provides the necessary SQL abstraction to integrate SQL-like queries (HiveQL) into the underlying Java without the need to implement queries in the low-level Java API. -
47
Nextflow
Seqera Labs
Data-driven computational pipelines. Nextflow enables scalable and reproducible scientific workflows using software containers. It allows the adaptation of pipelines written in the most common scripting languages. Its fluent DSL simplifies the implementation and deployment of complex parallel and reactive workflows on clouds and clusters. Nextflow is built around the idea that Linux is the lingua franca of data science. Nextflow allows you to write a computational pipeline by making it simpler to put together many different tasks. You may reuse your existing scripts and tools and you don't need to learn a new language or API to start using it. Nextflow supports Docker and Singularity containers technology. This, along with the integration of the GitHub code-sharing platform, allows you to write self-contained pipelines, manage versions, and rapidly reproduce any former configuration. Nextflow provides an abstraction layer between your pipeline's logic and the execution layer.Starting Price: Free -
48
Actifio
Google
Automate self-service provisioning and refresh of enterprise workloads, integrate with existing toolchain. High-performance data delivery and re-use for data scientists through a rich set of APIs and automation. Recover any data across any cloud from any point in time – at the same time – at scale, beyond legacy solutions. Minimize the business impact of ransomware / cyber attacks by recovering quickly with immutable backups. Unified platform to better protect, secure, retain, govern, or recover your data on-premises or in the cloud. Actifio’s patented software platform turns data silos into data pipelines. Virtual Data Pipeline (VDP) delivers full-stack data management — on-premises, hybrid or multi-cloud – from rich application integration, SLA-based orchestration, flexible data movement, and data immutability and security. -
49
SAS Studio
SAS
SAS Studio provides a web browser-based programming environment, so writing and interacting with SAS code is easier and faster, wherever you are. It helps teams build efficient data pipelines with a data engineering experience designed for seamless collaboration, low-code work, and open source integration. SAS Studio connects to leading cloud data platforms such as AWS Redshift and S3, Google BigQuery and Cloud Storage, and Azure Data Lake Storage, as well as relational and nonrelational databases, including Oracle, Snowflake, Teradata, SingleStore, MongoDB, and other sources. It also works with file formats such as Excel, text, Parquet, and ORC. Users can choose no code, low code, or code by creating end-to-end data pipelines with drag-and-drop steps, developing Python and SAS code assets in SAS Studio or another IDE, and embedding them into SAS Studio flows for secure, centralized access to data sources and governed execution. SAS Studio supports ELT and ETL approaches. -
50
Upsolver
Upsolver
Upsolver makes it incredibly simple to build a governed data lake and to manage, integrate and prepare streaming data for analysis. Define pipelines using only SQL on auto-generated schema-on-read. Easy visual IDE to accelerate building pipelines. Add Upserts and Deletes to data lake tables. Blend streaming and large-scale batch data. Automated schema evolution and reprocessing from previous state. Automatic orchestration of pipelines (no DAGs). Fully-managed execution at scale. Strong consistency guarantee over object storage. Near-zero maintenance overhead for analytics-ready data. Built-in hygiene for data lake tables including columnar formats, partitioning, compaction and vacuuming. 100,000 events per second (billions daily) at low cost. Continuous lock-free compaction to avoid “small files” problem. Parquet-based tables for fast queries.
