Search Results for "spark" - Page 3
Sort By:
Showing 141 open source projects for "spark"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
Apache Bigtop
Bigtop is an Apache Foundation project for Infrastructure Engineers
Apache Bigtop is a project focused on building and packaging the Hadoop ecosystem and related big data components. It provides a consistent framework for testing, packaging, and deploying Hadoop distributions, including tools like HDFS, YARN, Spark, Hive, HBase, and more. By maintaining cross-platform builds (RPMs, DEBs, Docker images, and Kubernetes support), Bigtop makes it easier for organizations to deploy big data stacks in different environments. It also includes a set of integration tests and smoke tests to ensure compatibility and stability between ecosystem components. ... -
2
Kedro
A Python framework for creating reproducible, maintainable code
Kedro is an open sourced Python framework for creating maintainable and modular data science code. Provides the scaffolding to build more complex data and machine-learning pipelines. In addition, there's a focus on spending less time on the tedious "plumbing" required to maintain data science code; this means that you have more time to solve new problems. Standardises team workflows; the modular structure of Kedro facilitates a higher level of collaboration when teams solve problems... -
3
Cassandra Spark Connector
Apache Spark to Apache Cassandra connector
The Apache Cassandra Spark Connector allows Spark jobs (RDDs or DataFrames/Datasets) to read from and write to Cassandra tables. Compatible with Apache Cassandra (v2.1+), Spark 1.0–3.5, and Scala 2.11–2.13, it supports mapping Cassandra rows to Scala case classes, saving results back to Cassandra, and executing arbitrary CQL within Spark applications. -
4
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
...The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. Users write Python-like code (via DataFrame APIs or SQL strings) to express their transformations; behind the scenes, tasks are scheduled (often via Ray) and pushed into DuckDB instances operating on partitioned data. Because the storage layer (3FS) is optimized for random access and high throughput, smallpond can shuffle data, repartition, and manage intermediate results across nodes. -
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
5
HugeGraph
A graph database that supports more than 100+ billion data
...HugeGraph supports fast import performance in the case of more than 10 billion Vertices and Edges Graph, millisecond-level OLTP query capability, and can be integrated into big data platforms like Hadoop or Spark for OLAP analysis. The main scenarios of HugeGraph include correlation search, fraud detection, and knowledge graph. Not only supports Gremlin graph query language and RESTful API but also provides commonly used graph algorithm APIs. To help users easily implement various queries and analyses, HugeGraph has a full range of accessory tools, such as supporting distributed storage, data replication, scaling horizontally, and supports many built-in backends of storage engines. -
6
go-chart
go chart is a basic charting library in go
Package chart is a very simple golang native charting library that supports time-series and continuous line charts. Master should now be on the v3.x codebase, which overhauls the api significantly. Per usual, see examples for more information. Actual chart configurations and examples can be found in the ./examples/ directory. They are simple CLI programs that write to output.png (they are also updated with go generate. Everything on the chart.Chart object has defaults that can be overridden.... -
7
Everything for VU+, Formuler, Wetek, GigaBlue, Amiko, Venton, Dreambox & Zgemma.
-
8
SageMaker Spark
A Spark library for Amazon SageMaker
SageMaker Spark depends on hadoop-aws-2.8.1. To run Spark applications that depend on SageMaker Spark, you need to build Spark with Hadoop 2.8. However, if you are running Spark applications on EMR, you can use Spark built with Hadoop 2.7. -
9
json-scada
A portable SCADA/IoT platform centered on the MongoDB database server.
Standard IT tools applied to SCADA/IoT (MongoDB, PostgreSQL/TimescaleDB,Node.js, C#, Golang, Grafana, etc.). MongoDB as the real-time core database, persistence layer, config store, SOE historian. Portability and interoperability over Linux, Windows, x86/64, ARM. Horizontal scalability, from a single computer to big clusters (MongoDB-sharding), Bare Metal, Docker containers, VM, cloud, or hybrid deployments. Unlimited tags, servers, and users. HTML5 Web interface. UTF-8/I18N. Protocols:... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
-
11
-
12
Spark
Web-based, cross-platform and full-featured Remote Administration Tool
✨Spark is a web-based, cross-platform and full-featured Remote Administration Tool (RAT) written in Go that allows you control all your devices anywhere. Spark是一个Go编写的，网页UI、跨平台以及多功能的远程控制和监控工具，你可以随时随地监控和控制所有设备。 -
13
Horovod
Distributed training framework for TensorFlow, Keras, PyTorch, etc.
...With Horovod, an existing training script can be scaled up to run on hundreds of GPUs in just a few lines of Python code. Horovod can be installed on-premise or run out-of-the-box in cloud platforms, including AWS, Azure, and Databricks. Horovod can additionally run on top of Apache Spark, making it possible to unify data processing and model training into a single pipeline. Once Horovod has been configured, the same infrastructure can be used to train models with any framework, making it easy to switch between TensorFlow, PyTorch, MXNet, and future frameworks as machine learning tech stacks continue to evolve. Start scaling your model training with just a few lines of Python code. ... -
14
Alphadroid-Avicii
OnePlus Nord (Avicii) Builds of Alphadroid OS
This is an unofficial repository for OnePlus Nord (Avicii) to host Alphadroid OS builds. AlphaDroid is a custom ROM for Android, based on crDroid, with a new look and some extra features and optimizations. Memory optimizations and performance tweaks come mostly from RisingTechOSS and ProtonAOSP. Other extra features are picked from other custom ROMs and adapted to our needs or implemented by us. -
15
aqueduct LLM
Aqueduct allows you to run LLM and ML workloads on any infrastructure
...You can connect Aqueduct to your existing cloud infrastructure (docs), and Aqueduct will seamlessly move your code from your laptop to the cloud or between different cloud infrastructure layers. Aqueduct provides a single interface to running machine learning tasks on your existing cloud infrastructure — Kubernetes, Spark, Lambda, etc. From the same Python API, you can run code across any or all of these systems seamlessly and gain visibility into how your code is performing. -
16
-
17
spark
A performance profiler for Minecraft clients, servers, and proxies
spark is a performance profiling and diagnostic tool designed for Minecraft servers, clients, and proxy software. It helps developers and server administrators analyze performance issues such as lag, high CPU usage, and memory bottlenecks. The plugin collects statistical information about the server’s execution and generates detailed profiling data that can be used to identify inefficient code or problematic plugins. spark includes tools for CPU profiling, memory inspection, and monitoring server health metrics over time. ... -
18
Feathr
A scalable, unified data and AI engineering platform for enterprise
Feathr is a data and AI engineering platform that is widely used in production at LinkedIn for many years and was open sourced in 2022. It is currently a project under LF AI & Data Foundation. Define data and feature transformations based on raw data sources (batch and streaming) using Pythonic APIs. Register transformations by names and get transformed data(features) for various use cases including AI modeling, compliance, go-to-market and more. Share transformations and data(features)... -
19
data-science-on-gcp
Source code accompanying book: Data Science on the GCP
...The repository is organized into multiple directories that reflect real-world pipelines, such as ingesting data, running SQL-based analytics, streaming data processing, using Spark and Dataproc, applying BigQuery ML, and deploying models with Vertex AI. It emphasizes practical, production-oriented workflows rather than isolated examples, showing how different Google Cloud services interact to form cohesive pipelines. -
20
SnappyData
Memory optimized analytics database, based on Apache Spark
SnappyData (aka TIBCO ComputeDB) is a distributed, in-memory optimized analytics database. SnappyData delivers high throughput, low latency, and high concurrency for a unified analytics workload. By fusing an in-memory hybrid database inside Apache Spark, it provides analytic query processing, mutability/transactions, access to virtually all big data sources and stream processing all in one unified cluster. One common use case for SnappyData is to provide analytics at interactive speeds over large volumes of data with minimal or no pre-processing of the dataset. For instance, there is no need to often pre-aggregate/reduce or generate cubes over your large data sets for ad-hoc visual analytics. ... -
21
Spark.jl
Julia binding for Apache Spark
A Julia interface to Apache Spark. Spark.jl provides an interface to Apache Spark™ platform, including SQL / DataFrame and Structured Streaming. It closely follows the PySpark API, making it easy to translate existing Python code to Julia. Spark.jl supports multiple cluster types (in client mode), and can be considered as an analog to PySpark or RSpark within the Julia ecosystem. -
22
Elephas
Distributed Deep learning with Keras & Spark
Elephas is an extension of Keras, which allows you to run distributed deep learning models at scale with Spark. Elephas currently supports a number of applications. Elephas brings deep learning with Keras to Spark. Elephas intends to keep the simplicity and high usability of Keras, thereby allowing for fast prototyping of distributed models, which can be run on massive data sets. Elephas implements a class of data-parallel algorithms on top of Keras, using Spark's RDDs and data frames. ... -
23
WTFJS
A list of funny and tricky JavaScript examples
...It’s designed as both a fun read and a serious learning aid, helping developers build an intuition for how JavaScript evaluates expressions. By highlighting common misconceptions, it encourages safer coding patterns and more reliable mental models. Teachers, interviewers, and learners use it to spark discussion and deepen understanding of JavaScript’s semantics. -
24
-
25
osm4scala
Reading OpenStreetMap Pbf files.
Scala and polyglot Spark library (Scala, PySpark, SparkSQL, ... ) focused on reading OpenStreetMap Pbf files.
