Business Software for Apache Spark - Page 2
View:
Top Software that integrates with Apache Spark as of July 2025 - Page 2
Sort By:
-
1
BentoML
BentoML
Serve your ML model in any cloud in minutes. Unified model packaging format enabling both online and offline serving on any platform. 100x the throughput of your regular flask-based model server, thanks to our advanced micro-batching mechanism. Deliver high-quality prediction services that speak the DevOps language and integrate perfectly with common infrastructure tools. Unified format for deployment. High-performance model serving. DevOps best practices baked in. The service uses the BERT model trained with the TensorFlow framework to predict movie reviews' sentiment. DevOps-free BentoML workflow, from prediction service registry, deployment automation, to endpoint monitoring, all configured automatically for your team. A solid foundation for running serious ML workloads in production. Keep all your team's models, deployments, and changes highly visible and control access via SSO, RBAC, client authentication, and auditing logs.Starting Price: Free -
2
Flyte
Union.ai
The workflow automation platform for complex, mission-critical data and ML processes at scale. Flyte makes it easy to create concurrent, scalable, and maintainable workflows for machine learning and data processing. Flyte is used in production at Lyft, Spotify, Freenome, and others. At Lyft, Flyte has been serving production model training and data processing for over four years, becoming the de-facto platform for teams like pricing, locations, ETA, mapping, autonomous, and more. In fact, Flyte manages over 10,000 unique workflows at Lyft, totaling over 1,000,000 executions every month, 20 million tasks, and 40 million containers. Flyte has been battle-tested at Lyft, Spotify, Freenome, and others. It is entirely open-source with an Apache 2.0 license under the Linux Foundation with a cross-industry overseeing committee. Configuring machine learning and data workflows can get complex and error-prone with YAML.Starting Price: Free -
3
Vertex AI Notebooks
Google
Vertex AI Notebooks is a fully managed, scalable solution from Google Cloud that accelerates machine learning (ML) development. It provides a seamless, interactive environment for data scientists and developers to explore data, prototype models, and collaborate in real-time. With integration into Google Cloud’s vast data and ML tools, Vertex AI Notebooks supports rapid prototyping, automated workflows, and deployment, making it easier to scale ML operations. The platform’s support for both Colab Enterprise and Vertex AI Workbench ensures a flexible and secure environment for diverse enterprise needs.Starting Price: $10 per GB -
4
Comet
Comet
Manage and optimize models across the entire ML lifecycle, from experiment tracking to monitoring models in production. Achieve your goals faster with the platform built to meet the intense demands of enterprise teams deploying ML at scale. Supports your deployment strategy whether it’s private cloud, on-premise servers, or hybrid. Add two lines of code to your notebook or script and start tracking your experiments. Works wherever you run your code, with any machine learning library, and for any machine learning task. Easily compare experiments—code, hyperparameters, metrics, predictions, dependencies, system metrics, and more—to understand differences in model performance. Monitor your models during every step from training to production. Get alerts when something is amiss, and debug your models to address the issue. Increase productivity, collaboration, and visibility across all teams and stakeholders.Starting Price: $179 per user per month -
5
DQOps
DQOps
DQOps is an open-source data quality platform designed for data quality and data engineering teams that makes data quality visible to business sponsors. The platform provides an efficient user interface to quickly add data sources, configure data quality checks, and manage issues. DQOps comes with over 150 built-in data quality checks, but you can also design custom checks to detect any business-relevant data quality issues. The platform supports incremental data quality monitoring to support analyzing data quality of very big tables. Track data quality KPI scores using our built-in or custom dashboards to show progress in improving data quality to business sponsors. DQOps is DevOps-friendly, allowing you to define data quality definitions in YAML files stored in Git, run data quality checks directly from your data pipelines, or automate any action with a Python Client. DQOps works locally or as a SaaS platform.Starting Price: $499 per month -
6
ELCA Smart Data Lake Builder
ELCA Group
Classical Data Lakes are often reduced to basic but cheap raw data storage, neglecting significant aspects like transformation, data quality and security. These topics are left to data scientists, who end up spending up to 80% of their time acquiring, understanding and cleaning data before they can start using their core competencies. In addition, classical Data Lakes are often implemented by separate departments using different standards and tools, which makes it harder to implement comprehensive analytical use cases. Smart Data Lakes solve these various issues by providing architectural and methodical guidelines, together with an efficient tool to build a strong high-quality data foundation. Smart Data Lakes are at the core of any modern analytics platform. Their structure easily integrates prevalent Data Science tools and open source technologies, as well as AI and ML. Their storage is cheap and scalable, supporting both unstructured data and complex data structures.Starting Price: Free -
7
BigLake
Google
BigLake is a storage engine that unifies data warehouses and lakes by enabling BigQuery and open-source frameworks like Spark to access data with fine-grained access control. BigLake provides accelerated query performance across multi-cloud storage and open formats such as Apache Iceberg. Store a single copy of data with uniform features across data warehouses & lakes. Fine-grained access control and multi-cloud governance over distributed data. Seamless integration with open-source analytics tools and open data formats. Unlock analytics on distributed data regardless of where and how it’s stored, while choosing the best analytics tools, open source or cloud-native over a single copy of data. Fine-grained access control across open source engines like Apache Spark, Presto, and Trino, and open formats such as Parquet. Performant queries over data lakes powered by BigQuery. Integrates with Dataplex to provide management at scale, including logical data organization.Starting Price: $5 per TB -
8
HStreamDB
EMQ
A streaming database is purpose-built to ingest, store, process, and analyze massive data streams. It is a modern data infrastructure that unifies messaging, stream processing, and storage to help get value out of your data in real-time. Ingest massive amounts of data continuously generated from various sources, such as IoT device sensors. Store millions of data streams reliably in a specially designed distributed streaming data storage cluster. Consume data streams in real-time as fast as from Kafka by subscribing to topics in HStreamDB. With the permanent data stream storage, you can playback and consume data streams anytime. Process data streams based on event-time with the same familiar SQL syntax you use to query data in a relational database. You can use SQL to filter, transform, aggregate, and even join multiple data streams.Starting Price: Free -
9
Apache PredictionIO
Apache
Apache PredictionIO® is an open-source machine learning server built on top of a state-of-the-art open-source stack for developers and data scientists to create predictive engines for any machine learning task. It lets you quickly build and deploy an engine as a web service on production with customizable templates. Respond to dynamic queries in real-time once deployed as a web service, evaluate and tune multiple engine variants systematically, and unify data from multiple platforms in batch or in real-time for comprehensive predictive analytics. Speed up machine learning modeling with systematic processes and pre-built evaluation measures, support machine learning and data processing libraries such as Spark MLLib and OpenNLP. Implement your own machine learning models and seamlessly incorporate them into your engine. Simplify data infrastructure management. Apache PredictionIO® can be installed as a full machine learning stack, bundled with Apache Spark, MLlib, HBase, Akka HTTP, etc.Starting Price: Free -
10
Akira AI
Akira AI
Akira.ai provides businesses with Agentic AI, a set of specialized AI agents designed to optimize and automate complex workflows across various industries. These AI agents collaborate with human teams, enhancing productivity, making real-time decisions, and automating repetitive tasks, such as data analysis, incident management, and HR processes. The platform integrates smoothly with existing systems, including CRMs and ERPs, ensuring a disruption-free transition to AI-enhanced operations. Akira’s AI agents help businesses streamline their operations, increase decision-making speed, and boost overall efficiency, driving innovation across sectors like manufacturing, finance, and IT.Starting Price: $15 per month -
11
ZenML
ZenML
Simplify your MLOps pipelines. Manage, deploy, and scale on any infrastructure with ZenML. ZenML is completely free and open-source. See the magic with just two simple commands. Set up ZenML in a matter of minutes, and start with all the tools you already use. ZenML standard interfaces ensure that your tools work together seamlessly. Gradually scale up your MLOps stack by switching out components whenever your training or deployment requirements change. Keep up with the latest changes in the MLOps world and easily integrate any new developments. Define simple and clear ML workflows without wasting time on boilerplate tooling or infrastructure code. Write portable ML code and switch from experimentation to production in seconds. Manage all your favorite MLOps tools in one place with ZenML's plug-and-play integrations. Prevent vendor lock-in by writing extensible, tooling-agnostic, and infrastructure-agnostic code.Starting Price: Free -
12
Kedro
Kedro
Kedro is the foundation for clean data science code. It borrows concepts from software engineering and applies them to machine-learning projects. A Kedro project provides scaffolding for complex data and machine-learning pipelines. You spend less time on tedious "plumbing" and focus instead on solving new problems. Kedro standardizes how data science code is created and ensures teams collaborate to solve problems easily. Make a seamless transition from development to production with exploratory code that you can transition to reproducible, maintainable, and modular experiments. A series of lightweight data connectors is used to save and load data across many different file formats and file systems.Starting Price: Free -
13
Tabular
Tabular
Tabular is an open table store from the creators of Apache Iceberg. Connect multiple computing engines and frameworks. Decrease query time and storage costs by up to 50%. Centralize enforcement of data access (RBAC) policies. Connect any query engine or framework, including Athena, BigQuery, Redshift, Snowflake, Databricks, Trino, Spark, and Python. Smart compaction, clustering, and other automated data services reduce storage costs and query times by up to 50%. Unify data access at the database or table. RBAC controls are simple to manage, consistently enforced, and easy to audit. Centralize your security down to the table. Tabular is easy to use plus it features high-powered ingestion, performance, and RBAC under the hood. Tabular gives you the flexibility to work with multiple “best of breed” compute engines based on their strengths. Assign privileges at the data warehouse database, table, or column level.Starting Price: $100 per month -
14
Apache Doris
The Apache Software Foundation
Apache Doris is a modern data warehouse for real-time analytics. It delivers lightning-fast analytics on real-time data at scale. Push-based micro-batch and pull-based streaming data ingestion within a second. Storage engine with real-time upsert, append and pre-aggregation. Optimize for high-concurrency and high-throughput queries with columnar storage engine, MPP architecture, cost based query optimizer, vectorized execution engine. Federated querying of data lakes such as Hive, Iceberg and Hudi, and databases such as MySQL and PostgreSQL. Compound data types such as Array, Map and JSON. Variant data type to support auto data type inference of JSON data. NGram bloomfilter and inverted index for text searches. Distributed design for linear scalability. Workload isolation and tiered storage for efficient resource management. Supports shared-nothing clusters as well as separation of storage and compute.Starting Price: Free -
15
Hue
Hue
Hue brings the best querying experience with the most intelligent autocomplete and query editor components. The tables and storage browsers leverage your existing data catalog knowledge transparently. Help users find the correct data among thousands of databases and self-document it. Assist users with their SQL queries and leverage rich previews for links, sharing from the editor directly in Slack. Several apps, each one specialized in a certain type of querying are available. Data sources can be explored first via the browsers. The editor shines for SQL queries. It comes with an intelligent autocomplete, risk alerts, and self-service troubleshooting. Dashboards focus on visualizing indexed data but can also query SQL databases. You can now search for certain cell values in the table and the results are highlighted. To make your SQL editing experience, Hue comes with one of the best SQL autocomplete on the planet.Starting Price: Free -
16
Yandex Data Proc
Yandex
You select the size of the cluster, node capacity, and a set of services, and Yandex Data Proc automatically creates and configures Spark and Hadoop clusters and other components. Collaborate by using Zeppelin notebooks and other web apps via a UI proxy. You get full control of your cluster with root permissions for each VM. Install your own applications and libraries on running clusters without having to restart them. Yandex Data Proc uses instance groups to automatically increase or decrease computing resources of compute subclusters based on CPU usage indicators. Data Proc allows you to create managed Hive clusters, which can reduce the probability of failures and losses caused by metadata unavailability. Save time on building ETL pipelines and pipelines for training and developing models, as well as describing other iterative tasks. The Data Proc operator is already built into Apache Airflow.Starting Price: $0.19 per hour -
17
Tonic Ephemeral
Tonic
Stop wasting time provisioning and maintaining databases yourself. Effortlessly create isolated test databases to ship features faster. Equip your developers with the ready-to-go data they need to keep fast-paced projects on track. Spin up pre-populated databases for testing purposes as part of your CI/CD pipeline, and automatically tear them down once the tests are done. Quickly and painlessly spin up databases at the click of a button for testing, bug reproduction, demos, and more with built-in container orchestration. Use our patented subsetter to shrink PBs down to GBs without breaking referential integrity, then leverage Tonic Ephemeral to spin up a database with only the data needed for development to cut cloud costs and maximize efficiency. Pair our patented subsetted with Tonic Ephemeral to get all the data subsets you need for only as long as you need them. Maximize efficiency by getting your developers access to one-off datasets for local development.Starting Price: $199 per month -
18
Spark NLP
John Snow Labs
Experience the power of large language models like never before, unleashing the full potential of Natural Language Processing (NLP) with Spark NLP, the open source library that delivers scalable LLMs. The full code base is open under the Apache 2.0 license, including pre-trained models and pipelines. The only NLP library built natively on Apache Spark. The most widely used NLP library in the enterprise. Spark ML provides a set of machine learning applications that can be built using two main components, estimators and transformers. The estimators have a method that secures and trains a piece of data to such an application. The transformer is generally the result of a fitting process and applies changes to the target dataset. These components have been embedded to be applicable to Spark NLP. Pipelines are a mechanism for combining multiple estimators and transformers in a single workflow. They allow multiple chained transformations along a machine-learning task.Starting Price: Free -
19
StarRocks
StarRocks
Whether you're working with a single table or multiple, you'll experience at least 300% better performance on StarRocks compared to other popular solutions. From streaming data to data capture, with a rich set of connectors, you can ingest data into StarRocks in real time for the freshest insights. A query engine that adapts to your use cases. Without moving your data or rewriting SQL, StarRocks provides the flexibility to scale your analytics on demand with ease. StarRocks enables a rapid journey from data to insight. StarRocks' performance is unmatched and provides a unified OLAP solution covering the most popular data analytics scenarios. Whether you're working with a single table or multiple, you'll experience at least 300% better performance on StarRocks compared to other popular solutions. StarRocks' built-in memory-and-disk-based caching framework is specifically designed to minimize the I/O overhead of fetching data from external storage to accelerate query performance.Starting Price: Free -
20
Speedb
Speedb
The next-generation key-value storage engine.bSpeedb is 100% RocksDB compatible enhancing stability, efficiency, and overall performance. Join the Hive, Speedb’s open-source community, to interact, improve, and share knowledge and best practices on RocksDB. Speedb is a compatible alternative for LevelDB and RocksDB users who would like to take their application to the next level. When using event streaming platforms like Kafka, Flink, Spark, Splunk, Elastic, or others, consider using Speedb to enhance its performance. The increase in metadata in modern data sets is causing significant performance issues for many applications. With Speedb you can keep costs low and ensure your applications continue to run smoothly even under heavy loads. When it comes to making a choice to upgrade or deploy a new key-value store with your platform, Speedb is up for the challenge. By seamlessly integrating Speedb's advanced key-value storage engine with your projects, you'll experience immediate relief.Starting Price: Free -
21
Apache Phoenix
Apache Software Foundation
Apache Phoenix enables OLTP and operational analytics in Hadoop for low-latency applications by combining the best of both worlds. The power of standard SQL and JDBC APIs with full ACID transaction capabilities and the flexibility of late-bound, schema-on-read capabilities from the NoSQL world by leveraging HBase as its backing store. Apache Phoenix is fully integrated with other Hadoop products such as Spark, Hive, Pig, Flume, and Map Reduce. Become the trusted data platform for OLTP and operational analytics for Hadoop through well-defined, industry-standard APIs. Apache Phoenix takes your SQL query, compiles it into a series of HBase scans, and orchestrates the running of those scans to produce regular JDBC result sets. Direct use of the HBase API, along with coprocessors and custom filters, results in performance on the order of milliseconds for small queries, or seconds for tens of millions of rows.Starting Price: Free -
22
Stackable
Stackable
The Stackable data platform was designed with openness and flexibility in mind. It provides you with a curated selection of the best open source data apps like Apache Kafka, Apache Druid, Trino, and Apache Spark. While other current offerings either push their proprietary solutions or deepen vendor lock-in, Stackable takes a different approach. All data apps work together seamlessly and can be added or removed in no time. Based on Kubernetes, it runs everywhere, on-prem or in the cloud. stackablectl and a Kubernetes cluster are all you need to run your first stackable data platform. Within minutes, you will be ready to start working with your data. Configure your one-line startup command right here. Similar to kubectl, stackablectl is designed to easily interface with the Stackable Data Platform. Use the command line utility to deploy and manage stackable data apps on Kubernetes. With stackablectl, you can create, delete, and update components.Starting Price: Free -
23
Inferyx
Inferyx
Move past application silos, cost overrun, and skill obsolescence to scale faster with our intelligent data and analytics platform. An intelligent platform built to perform data management and advanced analytics. Helps you scale across the technology landscape. Our architecture understands how data flows and transforms throughout its lifecycle. Enabling the development of future-proof enterprise AI applications. A highly modular and extensible platform that enables the handling of multifold components. Designed to scale with a multi-tenant architecture. Analyzing complex data structures is made easy using advanced data visualization. Resulting in enhanced enterprise AI app development in an intuitive and low-code predictive platform. Our unique hybrid multi-cloud platform is built using open source community software which makes it immensely adaptive, highly secure, and essentially low-cost.Starting Price: Free -
24
ScaleOps
ScaleOps
Reduce Kubernetes costs by up to 80% and enhance cluster reliability by using real-time, application context-aware, automation for your most critical production environments. We are bringing a new era of cloud resource management by using our proprietary technology of real-time automation & application context awareness, unlocking the full potential of cloud-native applications. Cut your Kubernetes costs by up to 80% through our intelligent resource optimization and automated workload management, ensuring you only pay for what you need without sacrificing performance. Enhance your Kubernetes environments for peak application performance and improve cluster reliability with proactive and reactive mechanisms that automatically mitigate issues caused by sudden, unexpected bursts and stressed nodes, ensuring stability and performance. Installation takes just 2 minutes. Starting with read-only permissions, you will immediately discover the potential our platform can bring to your apps.Starting Price: $5 per month -
25
DataHub
DataHub
DataHub is an open source metadata platform designed to streamline data discovery, observability, and governance across diverse data ecosystems. It enables organizations to effortlessly discover trustworthy data, with experiences tailored for each person and eliminates breaking changes with detailed cross-platform and column-level lineage. DataHub builds confidence in your data by providing a comprehensive view of business, operational, and technical context, all in one place. The platform offers automated data quality checks and AI-driven anomaly detection, notifying teams when issues arise and centralizing incident tracking. With detailed lineage, documentation, and ownership information, DataHub facilitates swift issue resolution. It also automates governance programs by classifying assets as they evolve, minimizing manual work through GenAI documentation, AI-driven classification, and smart propagation. DataHub's extensible architecture supports over 70 native integrations.Starting Price: Free -
26
Alteryx
Alteryx
Step into a new era of analytics with the Alteryx AI Platform. Empower your organization with automated data preparation, AI-powered analytics, and approachable machine learning — all with embedded governance and security. Welcome to the future of data-driven decisions for every user, every team, every step of the way. Empower your teams with an easy, intuitive user experience allowing everyone to create analytic solutions that improve productivity, efficiency, and the bottom line. Build an analytics culture with an end-to-end cloud analytics platform and transform data into insights with self-service data prep, machine learning, and AI-generated insights. Reduce risk and ensure your data is fully protected with the latest security standards and certifications. Connect to your data and applications with open API standards. -
27
Protegrity
Protegrity
Our platform allows businesses to use data—including its application in advanced analytics, machine learning, and AI—to do great things without worrying about putting customers, employees, or intellectual property at risk. The Protegrity Data Protection Platform doesn't just secure data—it simultaneously classifies and discovers data while protecting it. You can't protect what you don't know you have. Our platform first classifies data, allowing users to categorize the type of data that can mostly be in the public domain. With those classifications established, the platform then leverages machine learning algorithms to discover that type of data. Classification and discovery finds the data that needs to be protected. Whether encrypting, tokenizing, or applying privacy methods, the platform secures the data behind the many operational systems that drive the day-to-day functions of business, as well as the analytical systems behind decision-making. -
28
RazorThink
RazorThink
RZT aiOS offers all of the benefits of a unified artificial intelligence platform and more, because it's not just a platform — it's a comprehensive Operating System that fully connects, manages and unifies all of your AI initiatives. And, AI developers now can do in days what used to take them months, because aiOS process management dramatically increases the productivity of AI teams. This Operating System offers an intuitive environment for AI development, letting you visually build models, explore data, create processing pipelines, run experiments, and view analytics. What's more is that you can do it all even without advanced software engineering skills. -
29
Querona
YouNeedIT
We make BI & Big Data analytics work easier and faster. Our goal is to empower business users and make always-busy business and heavily loaded BI specialists less dependent on each other when solving data-driven business problems. If you have ever experienced a lack of data you needed, time to consuming report generation or long queue to your BI expert, consider Querona. Querona uses a built-in Big Data engine to handle growing data volumes. Repeatable queries can be cached or calculated in advance. Optimization needs less effort as Querona automatically suggests query improvements. Querona empowers business analysts and data scientists by putting self-service in their hands. They can easily discover and prototype data models, add new data sources, experiment with query optimization and dig in raw data. Less IT is needed. Now users can get live data no matter where it is stored. If databases are too busy to be queried live, Querona will cache the data. -
30
geoblink
geoblink
Gain strategic insights about your business instantly and roll out tailored action plans to maximise success. Geoblink's Location Management Platform was designed to help professionals with different business profiles achieve their goals and make their locations reach their full potential. Monitor and manage your network’s health and ensure it reaches its full sales potential. Open in places where the market conditions match your best-performing stores. Reinforce your product mix and launch campaigns at the right time and place. Geoblink is a SaaS-based Location Intelligence solution that helps professionals from the retail, real estate, and FMCG industries make informed decisions about their business strategies. It combines traditional and non-traditional advanced analytics techniques over big and small data, together with a rich map-based UI to display multiple types of statistics in a way that is simple to use and easy to understand.