Open Source Scala Machine Learning Software

SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines. SynapseML builds on Apache Spark and SparkML to enable new kinds of machine learning, analytics, and model deployment workflows. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with the Open Neural Network Exchange (ONNX), LightGBM, The Cognitive Services, Vowpal Wabbit, and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of data sources. SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. For production-grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library written in Scala that runs on top of Apache Spark. It was developed with a focus on accelerating machine learning developer productivity through machine learning automation, and an API that enforces compile-time type-safety, modularity, and reuse. Through automation, it achieves accuracies close to hand-tuned models with almost 100x reduction in time.

Byzer (former MLSQL) is a low-code, open-sourced, and distributed programming language for data pipeline, analytics, and AI in a cloud-native way. Design protocol: Everything is a table. Byzer is a SQL-like language, to simplify data pipeline, analytics, and AI, combined with built-in algorithms and extensions. We believe that everything is a table, a simple and powerful SQL-like language can significantly reduce human efforts of data development without switching different tools.

Metarank is a service that can personalize any type of content: product listings, articles, recommendations and search results in 3 easy steps with a few lines of code. It’s often considered "too risky" to spend 6+ months on an in-house moonshot project to reinvent the wheel without an experienced team and no existing open-source tools. Metarank makes it easy not only for Amazon to do personalization but for everyone else. Ingest historical item listings, clicks and item metadata so Metarank can find hidden dependencies in the data using our simple JSON format.No Machine Learning experience is required, run our CLI tool with a set of features in a YAML configuration. Run Metarank API service, feed it with real-time events and receive a personalized ranking for your items that will boost conversion, click-through rate or any other business-critical metric you define.

Aerosolve is an open-source machine learning library developed by Airbnb, designed for interpretable and human-friendly modeling. Built around sparse, human-intuitive features (like geography, pricing), it supports feature quantization, interaction specification, and rule-based priors—enabling domain experts to contribute directly to model behavior.

Apache PredictionIO® is an open source Machine Learning Server built on top of a state-of-the-art open source stack for developers and data scientists to create predictive engines for any machine learning task. Quickly build and deploy an engine as a web service on production with customizable templates; respond to dynamic queries in real-time once deployed as a web service; evaluate and tune multiple engine variants systematically; unify data from multiple platforms in batch or in real-time for comprehensive predictive analytics; speed up machine learning modeling with systematic processes and pre-built evaluation measures; support machine learning and data processing libraries such as Spark MLLib and OpenNLP; implement your own machine learning models and seamlessly incorporate them into your engine; simplify data infrastructure management.

An artificial neural network, currently specialized to save a specific bit pattern, mainly by changing the signal propagation delays in links. More features, variables and algorithms will be added in time.

PhiWeave is a machine learning library for structured prediction via factor graphs. It is part of an ongoing effort to implement and improve on the current state-of-the-art in inference and parameter estimation for graphical models.

Apache PredictionIO is an open-source machine learning server designed to simplify the process of building and deploying predictive engines. It offers a scalable infrastructure with support for multiple ML algorithms, event data collection, and deployment workflows. Developers can use templates or build custom engines, making it a flexible solution for integrating machine learning into applications.