Search Results for "mapreduce"

MIT-6.824 is a study repository containing source code and supporting material for MIT's distributed systems course. Its materials are based on the 2021 version of the class and organize resources around lectures and practical exercises. Topics include MapReduce, remote procedure calls, threads, the Google File System, primary-backup replication, and Raft consensus. The repository includes Go-based exercises such as crawler, key-value, and MapReduce labs. It also collects lecture questions intended to reinforce distributed systems concepts and paper discussions. Links to course schedules, lecture videos, and foundational papers provide additional material for independent study.

Scalding is a Scala DSL built on Cascading that simplifies writing Hadoop MapReduce jobs. It lets users describe data transformations using Scala’s functional abstractions, while abstracting away low-level Hadoop boilerplate. It enables expressive and testable pipeline definitions and integrates with various input/output formats.

Summingbird is a streaming + batch hybrid computation framework developed by Twitter. Its aim is to let developers express data aggregation pipelines in a unified way, where the same logic can run either in real time (stream) or in batch mode, and the results can be merged or reconciled. In effect, Summingbird abstracts over multiple execution engines (such as Storm, Scalding, etc.) to provide one high-level program that composes transformations and aggregations, and then executes them in...

Framework for development of simple evolutionary algorithms / island models programs in distributed environment using MapReduce programming model based on hadoop.