MarDRe

MarDRe is a de novo MapReduce-based parallel tool to remove duplicate and near-duplicate DNA reads through the clustering of single-end and paired-end sequences from FASTQ/FASTA datasets (plain or compressed with Gzip/BZip2). This tool allows bioinformaticians to avoid the analysis of not necessary reads, reducing the time of subsequent procedures with the dataset. (e.g., assemblies, mappings, etc.). Among each group of duplicate reads, MarDRe keeps the one with the highest average quality. MarDRe can also remove only the optical duplicates, that is, duplicate reads of the same tile that are within a given distance of each other according to the X:Y coordinates included in their names.

MarDRe is the Big Data counterpart of ParDRe (link above), which employs HPC technologies (i.e., hybrid MPI/multithreading) to reduce runtime on multicore systems. Instead, MarDRe takes advantage of the MapReduce programming model to significantly improve ParDRe performance on distributed systems.