Business Software for Apache Spark - Page 6
View:
Top Software that integrates with Apache Spark as of July 2025 - Page 6
Sort By:
-
1
Kestra
Kestra
Kestra is an open-source, event-driven orchestrator that simplifies data operations and improves collaboration between engineers and business users. By bringing Infrastructure as Code best practices to data pipelines, Kestra allows you to build reliable workflows and manage them with confidence. Thanks to the declarative YAML interface for defining orchestration logic, everyone who benefits from analytics can participate in the data pipeline creation process. The UI automatically adjusts the YAML definition any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is defined declaratively in code, even if some workflow components are modified in other ways. -
2
Determined AI
Determined AI
Distributed training without changing your model code, determined takes care of provisioning machines, networking, data loading, and fault tolerance. Our open source deep learning platform enables you to train models in hours and minutes, not days and weeks. Instead of arduous tasks like manual hyperparameter tuning, re-running faulty jobs, and worrying about hardware resources. Our distributed training implementation outperforms the industry standard, requires no code changes, and is fully integrated with our state-of-the-art training platform. With built-in experiment tracking and visualization, Determined records metrics automatically, makes your ML projects reproducible and allows your team to collaborate more easily. Your researchers will be able to build on the progress of their team and innovate in their domain, instead of fretting over errors and infrastructure. -
3
VeloDB
VeloDB
Powered by Apache Doris, VeloDB is a modern data warehouse for lightning-fast analytics on real-time data at scale. Push-based micro-batch and pull-based streaming data ingestion within seconds. Storage engine with real-time upsert、append and pre-aggregation. Unparalleled performance in both real-time data serving and interactive ad-hoc queries. Not just structured but also semi-structured data. Not just real-time analytics but also batch processing. Not just run queries against internal data but also work as a federate query engine to access external data lakes and databases. Distributed design to support linear scalability. Whether on-premise deployment or cloud service, separation or integration of storage and compute, resource usage can be flexibly and efficiently adjusted according to workload requirements. Built on and fully compatible with open source Apache Doris. Support MySQL protocol, functions, and SQL for easy integration with other data tools. -
4
Qlik Staige
QlikTech
Harness the power of Qlik® Staige™ to make AI real by delivering a trusted data foundation, automation, actionable predictions, and company-wide impact. AI isn’t just experiments and initiatives — it’s an entire ecosystem of files, scripts, and results. Wherever your investments, we’ve partnered with top sources to bring you integrations that save time, enable management, and validate quality. Automate the delivery of real-time data into AWS data warehouses or data lakes, and make it easily accessible through a governed catalog. Through our new integration with Amazon Bedrock, you can easily connect to foundational large language models (LLMs) including A21 Labs, Amazon Titan, Anthropic, Cohere, and Meta. Seamless integration with Amazon Bedrock makes it easier for AWS customers to leverage large language models with analytics for AI-driven insights. -
5
Baidu Palo
Baidu AI Cloud
Palo helps enterprises to create the PB-level MPP architecture data warehouse service within several minutes and import the massive data from RDS, BOS, and BMR. Thus, Palo can perform the multi-dimensional analytics of big data. Palo is compatible with mainstream BI tools. Data analysts can analyze and display the data visually and gain insights quickly to assist decision-making. It has the industry-leading MPP query engine, with column storage, intelligent index,and vector execution functions. It can also provide in-library analytics, window functions, and other advanced analytics functions. You can create a materialized view and change the table structure without the suspension of service. It supports flexible and efficient data recovery. -
6
Baidu AI Cloud Stream Computing
Baidu AI Cloud
Baidu Stream Computing (BSC) provides real-time streaming data processing capacity with low delay, high throughput and high accuracy. It is fully compatible with Spark SQL; and can realize the logic data processing of complicated businesses through SQL statement, which is easy to use; provides users with full life cycle management for the streaming-oriented computing jobs. Integrate deeply with multiple storage products of Baidu AI Cloud as the upstream and downstream of stream computing, including Baidu Kafka, RDS, BOS, IOT Hub, Baidu ElasticSearch, TSDB, SCS and others. Provide a comprehensive job monitoring indicator, and the user can view the monitoring indicators of the job and set the alarm rules to protect the job. -
7
definity
definity
Monitor and control everything your data pipelines do with zero code changes. Monitor data and pipelines in motion to proactively prevent downtime and quickly root cause issues. Optimize pipeline runs and job performance to save costs and keep SLAs. Accelerate code deployments and platform upgrades while maintaining reliability and performance. Data & performance checks in line with pipeline runs. Checks on input data, before pipelines even run. Automatic preemption of runs. definity takes away the effort to build deep end-to-end coverage, so you are protected at every step, across every dimension. definity shifts observability to post-production to achieve ubiquity, increase coverage, and reduce manual effort. definity agents automatically run with every pipeline, with zero footprints. Unified view of data, pipelines, infra, lineage, and code for every data asset. Detect in run-time and avoid async checks. Auto-preempt runs, even on inputs. -
8
ModelOp
ModelOp
ModelOp is the leading AI governance software that helps enterprises safeguard all AI initiatives, including generative AI, Large Language Models (LLMs), in-house, third-party vendors, embedded systems, etc., without stifling innovation. Corporate boards and C‑suites are demanding the rapid adoption of generative AI but face financial, regulatory, security, privacy, ethical, and brand risks. Global, federal, state, and local-level governments are moving quickly to implement AI regulations and oversight, forcing enterprises to urgently prepare for and comply with rules designed to prevent AI from going wrong. Connect with AI Governance experts to stay informed about market trends, regulations, news, research, opinions, and insights to help you balance the risks and rewards of enterprise AI. ModelOp Center keeps organizations safe and gives peace of mind to all stakeholders. Streamline reporting, monitoring, and compliance adherence across the enterprise. -
9
Timbr.ai
Timbr.ai
The smart semantic layer integrates data with business meaning and relationships, unifies metrics, and accelerates the delivery of data products with 90% shorter SQL queries. Easily model data using business terms to give it common meaning and align business metrics. Define semantic relationships that substitute JOINs so queries become much simpler. Use hierarchies and classifications to better understand data. Automatically map data to the semantic model. Join multiple data sources with a powerful distributed SQL engine to query data at scale. Consume data as a connected semantic graph. Boost performance and save compute costs with an intelligent cache engine and materialized views. Benefit from advanced query optimizations. Connect to most clouds, datalakes, data warehouses, databases, and any file format. Timbr empowers you to work with your data sources seamlessly. When a query is run, Timbr optimizes the query and pushes it down to the backend. -
10
Gable
Gable
Data contracts facilitate communication between data teams and developers. Don’t just detect problematic changes, prevent them at the application level. Detect every change, from every data source using AI-based asset registration. Drive the adoption of data initiatives with upstream visibility and impact analysis. Shift left both data ownership and management through data governance as code and data contracts. Build data trust through the timely communication of data quality expectations and changes. Eliminate data issues at the source by seamlessly integrating our AI-driven technology. Everything you need to make your data initiative a success. Gable is a B2B data infrastructure SaaS that provides a collaboration platform to author and enforce data contracts. ‘Data contracts’, refer to API-based agreements between the software engineers who own upstream data sources and data engineers/analysts that consume data to build machine learning models and analytics. -
11
Azure Marketplace
Microsoft
Azure Marketplace is a comprehensive online store that provides access to thousands of certified, ready-to-use software applications, services, and solutions from Microsoft and third-party vendors. It enables businesses to discover, purchase, and deploy software directly within the Azure cloud environment. The marketplace offers a wide range of products, including virtual machine images, AI and machine learning models, developer tools, security solutions, and industry-specific applications. With flexible pricing options like pay-as-you-go, free trials, and subscription models, Azure Marketplace simplifies the procurement process and centralizes billing through a single Azure invoice. It supports seamless integration with Azure services, enabling organizations to enhance their cloud infrastructure, streamline workflows, and accelerate digital transformation initiatives. -
12
Unity Catalog
Databricks
Databricks Unity Catalog is the industry’s only unified and open governance solution for data and AI, built into the Databricks Data Intelligence Platform. With Unity Catalog, organizations can seamlessly govern both structured and unstructured data in any format, as well as machine learning models, notebooks, dashboards, and files across any cloud or platform. Data scientists, analysts, and engineers can securely discover, access, and collaborate on trusted data and AI assets across platforms, leveraging AI to boost productivity and unlock the full potential of the lakehouse environment. This unified and open approach to governance promotes interoperability and accelerates data and AI initiatives while simplifying regulatory compliance. Easily discover and classify both structured and unstructured data in any format, including machine learning models, notebooks, dashboards, and files across all cloud platforms. -
13
MLlib
Apache Software Foundation
Apache Spark's MLlib is a scalable machine learning library that integrates seamlessly with Spark's APIs, supporting Java, Scala, Python, and R. It offers a comprehensive suite of algorithms and utilities, including classification, regression, clustering, collaborative filtering, and tools for constructing machine learning pipelines. MLlib's high-quality algorithms leverage Spark's iterative computation capabilities, delivering performance up to 100 times faster than traditional MapReduce implementations. It is designed to operate across diverse environments, running on Hadoop, Apache Mesos, Kubernetes, standalone clusters, or in the cloud, and accessing various data sources such as HDFS, HBase, and local files. This flexibility makes MLlib a robust solution for scalable and efficient machine learning tasks within the Apache Spark ecosystem. -
14
Botify.cloud
Botify.cloud
Botify.cloud is an innovative platform designed to streamline and simplify cryptocurrency automation through a certified, all-in-one AI agent marketplace. With Botify.cloud, users can explore a diverse range of agent categories, including trading, volume management, social media, and utility agents. Our instant agent creation tool allows users to customize agents to their needs quickly and easily. It offers features such as agent creation, selling agents on the marketplace, Botify certification for every agent, diverse agent categories, and easy editing of agents' names and profiles. Users can also save their favorite agents for later use. For every agent that is sold, a token is created, and basically, in any transaction on the platform, users earn rewards. Building an agent is straightforward: simply choose a category, fill in the required fields, choose a large language model, and decide the temperature of your agent. -
15
NVIDIA Magnum IO
NVIDIA
NVIDIA Magnum IO is the architecture for parallel, intelligent data center I/O. It maximizes storage, network, and multi-node, multi-GPU communications for the world’s most important applications, using large language models, recommender systems, imaging, simulation, and scientific research. Magnum IO utilizes storage I/O, network I/O, in-network compute, and I/O management to simplify and speed up data movement, access, and management for multi-GPU, multi-node systems. It supports NVIDIA CUDA-X libraries and makes the best use of a range of NVIDIA GPU and networking hardware topologies to achieve optimal throughput and low latency. In multi-GPU, multi-node systems, slow CPU, single-thread performance is in the critical path of data access from local or remote storage devices. With storage I/O acceleration, the GPU bypasses the CPU and system memory, and accesses remote storage via 8x 200 Gb/s NICs, achieving up to 1.6 TB/s of raw storage bandwidth. -
16
Unravel
Unravel Data
Unravel makes data work anywhere: on Azure, AWS, GCP or in your own data center– Optimizing performance, automating troubleshooting and keeping costs in check. Unravel helps you monitor, manage, and improve your data pipelines in the cloud and on-premises – to drive more reliable performance in the applications that power your business. Get a unified view of your entire data stack. Unravel collects performance data from every platform, system, and application on any cloud then uses agentless technologies and machine learning to model your data pipelines from end to end. Explore, correlate, and analyze everything in your modern data and cloud environment. Unravel’s data model reveals dependencies, issues, and opportunities, how apps and resources are being used, what’s working and what’s not. Don’t just monitor performance – quickly troubleshoot and rapidly remediate issues. Leverage AI-powered recommendations to automate performance improvements, lower costs, and prepare. -
17
matchit
360Science
The foundation of our matching software, matchit® is designed specifically to deliver results that mirror human-like perception, at scale and without preprocessing. Using Artificial Intelligence, a proprietary phonetic algorithm, lexicons, and a contextual scoring engine, matchit defeats the errors, inconsistencies, and challenges commonly found in contact and business data. Conventional matching solutions require a user to define matching logic, which is a combination of functions and off-the-shelf fuzzy algorithms, used to produce an alphanumeric value. This alphanumeric value, or ‘match key’, forms the basis for comparing two records together and ultimately finding matches. Unlike conventional matching solutions, matchit doesn’t rely on a single comparison between match keys to find a match. Instead, matchit evaluates records contextually, running a variety of comparisons and scoring them individually to grade similarity between all the relevant elements that make up your data. -
18
OctoData
SoyHuCe
OctoData is deployed at a lower cost, in Cloud hosting and includes personalized support from the definition of your needs to the use of the solution. OctoData is based on innovative open-source technologies and knows how to adapt to open up to future possibilities. Its Supervisor offers a management interface that allows you to quickly capture, store and exploit a growing quantity and variety of data. With OctoData, prototype and industrialize your massive data recovery solutions in the same environment, including in real time. Thanks to the exploitation of your data, obtain precise reports, explore new possibilities, increase your productivity and gain in profitability. -
19
IBM SPSS Modeler
IBM
IBM SPSS Modeler is a leading visual data science and machine learning (ML) solution designed to help enterprises accelerate time to value by speeding up operational tasks for data scientists. Organizations worldwide use it for data preparation and discovery, predictive analytics, model management and deployment, and ML to monetize data assets. IBM SPSS Modeler automatically transforms data into the best format for the most accurate predictive modeling. It now only takes a few clicks for you to analyze data, identify fixes, screen out fields and derive new attributes. Leverage IBM SPSS Modeler’s powerful graphics engine to bring your insights to life. The smart chart recommender finds the perfect chart for your data from among dozens of options, so you can share your insights quickly and easily using compelling visualizations. -
20
Daft
Daft
Daft is a framework for ETL, analytics and ML/AI at scale. Its familiar Python dataframe API is built to outperform Spark in performance and ease of use. Daft plugs directly into your ML/AI stack through efficient zero-copy integrations with essential Python libraries such as Pytorch and Ray. It also allows requesting GPUs as a resource for running models. Daft runs locally with a lightweight multithreaded backend. When your local machine is no longer sufficient, it scales seamlessly to run out-of-core on a distributed cluster. Daft can handle User-Defined Functions (UDFs) in columns, allowing you to apply complex expressions and operations to Python objects with the full flexibility required for ML/AI. Daft runs locally with a lightweight multithreaded backend. When your local machine is no longer sufficient, it scales seamlessly to run out-of-core on a distributed cluster. -
21
Mage Platform
Mage Data
Mage Data™ is the leading solutions provider of data security and data privacy software for global enterprises. Built upon a patented and award-winning solution, the Mage platform enables organizations to stay on top of privacy regulations while ensuring security and privacy of data. Top Swiss Banks, Fortune 10 organizations, Ivy League Universities, and Industry Leaders in the financial and healthcare businesses protect their sensitive data with the Mage platform for Data Privacy and Security. Deploying state-of-the-art privacy enhancing technologies for securing data, Mage Data™ delivers robust data security while ensuring privacy of individuals. Visit the website to explore the company’s solutions. -
22
DataNimbus
DataNimbus
DataNimbus is an AI-powered platform that streamlines payments and accelerates AI adoption through innovative, cost-efficient solutions. By seamlessly integrating with Databricks components like Spark, Unity Catalog, and ML Ops, DataNimbus enhances scalability, governance, and runtime operations. Its offerings include a visual designer, a marketplace for reusable connectors and machine learning blocks, and agile APIs, all designed to simplify workflows and drive data-driven innovation. -
23
Precisely Connect
Precisely
Integrate data seamlessly from legacy systems into next-gen cloud and data platforms with one solution. Connect helps you take control of your data from mainframe to cloud. Integrate data through batch and real-time ingestion for advanced analytics, comprehensive machine learning and seamless data migration. Connect leverages the expertise Precisely has built over decades as a leader in mainframe sort and IBM i data availability and security to lead the industry in accessing and integrating complex data. Access to all your enterprise data for the most critical business projects is ensured by support for a wide range of sources and targets for all your ELT and CDC needs.