Search Results for "spark" - Page 4
Sort By:
Showing 141 open source projects for "spark"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
CoolplaySpark
Spark Cool Play: Spark source code analysis, Spark class library, etc.
CoolplaySpark is a learning and practice repository designed to help users understand and work with Apache Spark. It serves as a companion resource for the book 深入理解Spark核心思想与源码分析 (In-Depth Understanding of Spark’s Core Concepts and Source Code Analysis). The project contains annotated examples, explanations, and exercises that guide learners through Spark’s architecture, execution model, and source code internals. It is particularly valuable for developers who want to strengthen their understanding of Spark by not only using it as a data processing engine but also exploring how its internals function. ... -
2
TensorFlowOnSpark
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters
By combining salient features from the TensorFlow deep learning framework with Apache Spark and Apache Hadoop, TensorFlowOnSpark enables distributed deep learning on a cluster of GPU and CPU servers. It enables both distributed TensorFlow training and inferencing on Spark clusters, with a goal to minimize the amount of code changes required to run existing TensorFlow programs on a shared grid. -
3
-
4
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
5
Snowplow Analytics
Enterprise-strength marketing and product analytics platform
Snowplow is ideal for data teams who want to manage the collection and warehousing of data across all their platforms and products. -
6
-
7
Spark OS Codename : r5x Device: Realme 5 Maintainer: HardBricked_69 BuildType: UnOfficial
-
8
-
9
SparrowRecSys
A Deep Learning Recommender System
...SparrowRecSys supports a wide range of state-of-the-art recommendation algorithms, including models for click-through rate prediction and user behavior modeling that are widely used in advertising and content recommendation systems. The system is designed as a modular platform combining technologies such as Spark, TensorFlow, and web server components to represent the full lifecycle of recommendation pipelines. -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
10
Spark JobServer
REST job server for Apache Spark
Spark Job Server offers a RESTful interface for submitting, managing, and running jobs or contexts on Apache Spark. Rather than requiring every application to embed Spark or manage Spark contexts manually, this server abstracts a long-lived service where clients can upload JARs, start and stop contexts, submit jobs synchronously or asynchronously, and manage named objects (RDDs / DataFrames) across job executions. -
11
CueLake
Use SQL to build ELT pipelines on a data lakehouse
...CueLake uses Celery as the executor and celery-beat as the scheduler. Celery jobs trigger Zeppelin notebooks. Zeppelin auto-starts and stops the Spark cluster for every scheduled run of notebooks. -
12
-
13
-
14
Amadeus
Harmonious distributed data analysis in Rust
Amadeus is a high-performance, distributed data processing framework written in Rust, designed to offer an ergonomic and safe alternative to tools like Apache Spark. It provides both streaming and batch capabilities, allowing users to work with real-time and historical data at scale. Thanks to Rust’s memory safety and zero-cost abstractions, Amadeus delivers performance gains while reducing the complexity and bugs common in large-scale data pipelines. It emphasizes developer productivity through a fluent, expressive API and makes it easier to build composable and reliable data transformation pipelines without sacrificing speed or safety. -
15
Cisco TMS PhoneBook - PHP Version
Emulates Cisco TMS PhoneBook for CMM or Endpoints with PHP
Add the possibility to search and call endpoints from Cisco Meeting Management or from Cisco Endpoints (SX10/SX20/Codec Plus/Spark Room/Etc) without having a Cisco TMS (TelePresence Management Suite) server. Cisco didn't include a phone-book or the possibility to create a new video-conference from CMM. Those functions are still in the old TMS server where you need to buy a license for every endpoint you add... Anyway, this tool emulates the TMS PhoneBook and allows you to search and add new participants to a existing conference. ... -
16
SZT-bigdata
SZT‑bigdata is an open source project
SZT‑bigdata is an open-source project analyzing real Shenzhen metro (subway) card usage data using big‑data frameworks like Spark, Hadoop, Hive, Kafka, Flink, ClickHouse, HBase, and Elasticsearch. Aimed at exploring transit passenger flow patterns and system optimization using a variety of Scala-based technologies. -
17
Open Source Data Quality and Profiling
World's first open source data quality & data preparation project
...This project is also known as "Aggregate Profiler" Resful API for this project is getting built as (Beta Version) https://sourceforge.net/projects/restful-api-for-osdq/ apache spark based data quality is getting built at https://sourceforge.net/projects/apache-spark-osdq/ -
18
WhyLogs Java Library
Profile and monitor your ML data pipeline end-to-end
This is a Java implementation of WhyLogs, with support for Apache Spark integration for large scale datasets. Understanding the properties of data as it moves through applications is essential to keeping your ML/AI pipeline stable and improving your user experience, whether your pipeline is built for production or experimentation. WhyLogs is an open source statistical logging library that allows data science and ML teams to effortlessly profile ML/AI pipelines and applications, producing log files that can be used for monitoring, alerts, analytics, and error analysis. ... -
19
Albedo
A recommender system for discovering GitHub repos
Albedo is an open-source recommender system aimed at helping developers discover GitHub repositories by learning from activity signals. It treats repositories and developers as a graph of interactions and applies large-scale matrix factorization to model affinities, with Apache Spark providing the distributed data processing. The project focuses on implicit feedback—stars, watches, and other engagement metrics—so it can build useful recommendations without explicit ratings. A reproducible setup and Makefile-driven workflow streamline tasks like spinning up services, loading datasets, training models, and generating candidate lists. ... -
20
TransmogrifAI
TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library
TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library written in Scala that runs on top of Apache Spark. It was developed with a focus on accelerating machine learning developer productivity through machine learning automation, and an API that enforces compile-time type-safety, modularity, and reuse. Through automation, it achieves accuracies close to hand-tuned models with almost 100x reduction in time. -
21
Weld
High-performance runtime for data analytics applications
...This approach reduces data movement between libraries and enables the system to generate highly optimized machine code for parallel execution. Weld is particularly useful for workloads involving large-scale data processing in frameworks such as NumPy, Spark, and TensorFlow. The language includes built-in constructs for expressing data-parallel operations, enabling efficient execution on modern hardware architectures. By combining operations from multiple libraries into a single optimized execution plan, Weld can significantly improve performance in analytics and machine learning pipelines. -
22
Custom Apache Big data Distribution
A Custom Apache Distribution including Spark and Hadoop, for Windows.
This Distribution has been customized to work out of the box. So, just download it, and unzip it. Set the Path variables for bin folders, HADOOP_HOME, SPARK_HOME, and JAVA_HOME. That's it..! use Hadoop and Spark natively on Windows. -
23
PixieDust
Python Helper library for Jupyter Notebooks
PixieDust is an open source Python helper library that works as an add-on to Jupyter notebooks to improve the user experience of working with data. It also fills a gap for users who have no access to configuration files when a notebook is hosted on the cloud. -
24
benchm-ml
A benchmark of commonly used open source implementations
...The benchmarks cover algorithms like logistic regression, random forest, gradient boosting, and deep neural networks, and they compare across toolkits such as scikit-learn, R packages, xgboost, H2O, Spark MLlib, etc. The repository is structured in logical folders, each corresponding to algorithm categories. -
25
spark-ml-source-analysis
Spark ml algorithm principle analysis and specific source code
spark-ml-source-analysis is a technical repository that analyzes the internal implementation of machine learning algorithms within Apache Spark’s MLlib library. The project aims to help developers and data scientists understand how distributed machine learning algorithms are implemented and optimized inside the Spark ecosystem. Instead of providing a runnable software system, the repository focuses on explaining algorithm principles and examining the underlying source code used in Spark’s machine learning package. ...
