Apache Spark
Apache Spark™ is a unified analytics engine for large-scale data processing. Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. Spark offers over 80 high-level operators that make it easy to build parallel apps. And you can use it interactively from the Scala, Python, R, and SQL shells. Spark powers a stack of libraries including SQL and DataFrames, MLlib for machine learning, GraphX, and Spark Streaming. You can combine these libraries seamlessly in the same application. Spark runs on Hadoop, Apache Mesos, Kubernetes, standalone, or in the cloud. It can access diverse data sources. You can run Spark using its standalone cluster mode, on EC2, on Hadoop YARN, on Mesos, or on Kubernetes. Access data in HDFS, Alluxio, Apache Cassandra, Apache HBase, Apache Hive, and hundreds of other data sources.
Learn more
Lumio
Lumio is a web-based learning platform that offers more ways to effortlessly make learning fun and engaging on student devices.
Whether educators have PDFs, Google Slides, PowerPoint, or Notebook files, Lumio effortlessly transforms them into dynamic, engaging learning experiences on any device. Take students on an interactive journey filled with activities, games, group workspaces, formative assessments, and more, all from a single place.
Lumio is specifically designed to improve student outcomes – it’s not just fun, it works!
For schools and districts looking for instructional consistency and more efficient workflows, Lumio’s Spark plan offers a library to house approved curriculum content, integration with LMS platforms, and the ability for teachers to collaborate on content for increased productivity with minimal effort.
Note: Lumio was formerly known as SMART Learning Suite Online (SLSO)
Learn more
PySpark
PySpark is an interface for Apache Spark in Python. It not only allows you to write Spark applications using Python APIs, but also provides the PySpark shell for interactively analyzing your data in a distributed environment. PySpark supports most of Spark’s features such as Spark SQL, DataFrame, Streaming, MLlib (Machine Learning) and Spark Core. Spark SQL is a Spark module for structured data processing. It provides a programming abstraction called DataFrame and can also act as distributed SQL query engine. Running on top of Spark, the streaming feature in Apache Spark enables powerful interactive and analytical applications across both streaming and historical data, while inheriting Spark’s ease of use and fault tolerance characteristics.
Learn more
marimo
A reactive notebook for Python — run reproducible experiments, execute as a script, deploy as an app, and version with git.
🚀 batteries-included: replaces jupyter, streamlit, jupytext, ipywidgets, papermill, and more
⚡️ reactive: run a cell, and marimo reactively runs all dependent cells or marks them as stale
🖐️ interactive: bind sliders, tables, plots, and more to Python — no callbacks required
🔬 reproducible: no hidden state, deterministic execution, built-in package management
🏃 executable: execute as a Python script, parametrized by CLI args
🛜 shareable: deploy as an interactive web app or slides, run in the browser via WASM
🛢️ designed for data: query dataframes and databases with SQL, filter and search dataframes
🐍 git-friendly: notebooks are stored as .py files
⌨️ a modern editor: GitHub Copilot, AI assistants, vim keybindings, variable explorer, and more
Learn more