Search Results for "mapreduce"

At The New York Times, our development teams have been adopting the Go programming language over the last three years to build better back-end services. In the past I’ve written about using Go for Elastic MapReduce streaming. I’ve also talked about using Go at GothamGo for news analysis and to improve our email and alert systems at the Golang NYC Meetup. We use Go for a wide variety of tasks, but the most common use throughout the company is for building JSON APIs. When we first began building APIs with Go, we didn’t use any frameworks or shared interfaces. ...

Scalding is a Scala DSL built on Cascading that simplifies writing Hadoop MapReduce jobs. It lets users describe data transformations using Scala’s functional abstractions, while abstracting away low-level Hadoop boilerplate. It enables expressive and testable pipeline definitions and integrates with various input/output formats.

Summingbird is a streaming + batch hybrid computation framework developed by Twitter. Its aim is to let developers express data aggregation pipelines in a unified way, where the same logic can run either in real time (stream) or in batch mode, and the results can be merged or reconciled. In effect, Summingbird abstracts over multiple execution engines (such as Storm, Scalding, etc.) to provide one high-level program that composes transformations and aggregations, and then executes them in...

MapReduce++ is a project for implementation of parallel algorithms. It has currently two C++ implementations of the MapReduce abstraction: the MapMP library (multiprocessors) and the MaPI framework (multicomputers).

OceanSync is an Hadoop Management System that allows users to control a variety of aspects of Hadoop. This includes a Graphical User Interface that allows a user to perform HDFS maintenance tasks and submit new jobs to the cluster. The OceanSync product sits on top of any Hadoop Architecture.

Framework for development of simple evolutionary algorithms / island models programs in distributed environment using MapReduce programming model based on hadoop.