dupehasher scans the given directory for files that are identical except by filename.
dupeshasher will generate a report plaintext file named 'dupelist.txt' in the working directory (which contains, per line, tab-delimited names of files whose contents are identical).


Depending upon the arguments supplied by the user upon invoking dupeshasher, it will also delete all duplicates, keeping either the oldest or newest of all sets of duplicates. Additionally it can batch-rename all remaining (post-deletion) files in a chronologically-sequential manner; or according to the md5sum or sha256 sum of every file.

File extensions (.jpg for example) are preserved, and there is a user option to ensure that the extension is correct (for example, a .jpg file is indeed a jpeg and not, for example, a png).

Finally, an interactive mode exists for users who are not comfortable with the command line.

Features

  • Detect multiple copies of a file that are identical (based on md5sum)
  • Bulk-delete all but oldest copy for each set of identical files
  • Automatically fix file extension to match file type
  • Batch rename all files in directory sequentially
  • Batch rename all files in directory according to md5sum
  • Batch rename all files in directory according to sha256sum

Project Activity

See All Activity >

License

GNU General Public License version 2.0 (GPLv2)

Follow dupehasher

dupehasher Web Site

Other Useful Business Software
Full-stack observability with actually useful AI | Grafana Cloud Icon
Full-stack observability with actually useful AI | Grafana Cloud

Our generous forever free tier includes the full platform, including the AI Assistant, for 3 users with 10k metrics, 50GB logs, and 50GB traces.

Built on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
Create free account
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of dupehasher!

Additional Project Details

Operating Systems

BSD, Linux

Intended Audience

Advanced End Users, End Users/Desktop, Information Technology, Other Audience, System Administrators

User Interface

Console/Terminal

Programming Language

Unix Shell

Related Categories

Unix Shell File Managers, Unix Shell Archiving Software, Unix Shell Cataloguing Software

Registered

2014-03-20