Spark Streaming Integrations

The solution provided by Activeeon is suited to fit modern challenges such as the growth of data, new infrastructures, cloud strategy evolving, new application architecture, etc. It provides orchestration and scheduling to automate and build a solid base for future growth. ProActive Workflows & Scheduling is a java-based cross-platform workflow scheduler and resource manager that is able to run workflow tasks in multiple languages and multiple environments (Windows, Linux, Mac, Unix, etc). ProActive Resource Manager makes compute resources available for task execution. It handles on-premises and cloud compute resources in an elastic, on-demand and distributed fashion. ProActive AI Orchestration from Activeeon empowers data engineers and data scientists with a simple, portable and scalable solution for machine learning pipelines. It provides pre-built and customizable tasks that enable automation within the machine learning lifecycle, which helps data scientists and IT Operations work.

Solace PubSub+ Platform helps enterprises design, deploy and manage event-driven systems across hybrid and multi-cloud and IoT environments so they can be more event-driven and operate in real-time. The PubSub+ Platform includes the powerful PubSub+ Event Brokers, event management capabilities with PubSub+ Event Portal, as well as monitoring and integration capabilities all available via a single cloud console. PubSub+ allows easy creation of an event mesh, an interconnected network of event brokers, allowing for seamless and dynamic data movement across highly distributed network environments. PubSub+ Event Brokers can be deployed as fully managed cloud services, self-managed software in private cloud or on-premises environments, or as turnkey hardware appliances for unparalleled performance and low TCO. PubSub+ Event Portal is a complimentary toolset for design and governance of event-driven systems including both Solace and Kafka-based event broker environments.

Apache Spark™ is a unified analytics engine for large-scale data processing. Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. Spark offers over 80 high-level operators that make it easy to build parallel apps. And you can use it interactively from the Scala, Python, R, and SQL shells. Spark powers a stack of libraries including SQL and DataFrames, MLlib for machine learning, GraphX, and Spark Streaming. You can combine these libraries seamlessly in the same application. Spark runs on Hadoop, Apache Mesos, Kubernetes, standalone, or in the cloud. It can access diverse data sources. You can run Spark using its standalone cluster mode, on EC2, on Hadoop YARN, on Mesos, or on Kubernetes. Access data in HDFS, Alluxio, Apache Cassandra, Apache HBase, Apache Hive, and hundreds of other data sources.