Mapreduce chaining in Pydoop

Help
jaganadhg
2014-03-04
2014-03-05
  • jaganadhg
    jaganadhg
    2014-03-04

    Hi Team,

    Is there any example available for MapReduce chaining in Pydoop.

    Best regards

    Jaggu

     
  • Simone Leo
    Simone Leo
    2014-03-05

    Hello,

    Pydoop has no specific built-in feature for chaining (i.e., there's no Python equivalent of ChainMapper). You have to either implement the full logic in a single mapper, which prevents you from reusing existing components, or run separate MR jobs, which will likely be less efficient.

    Simone

     
    Last edit: Simone Leo 2014-03-05
  • Luca Pireddu
    Luca Pireddu
    2014-03-05

    Or implement a ChainMapper for Pydoop, which we'd be happy to integrate into the project ;-)

    Luca