Open Source C Duplicate File Finders

FDUPES is a lightweight command-line utility that helps users find and optionally delete duplicate files within specified directories by comparing file contents, which can be extremely useful for cleaning up storage clutter or organizing large collections of files. It works by scanning directories and subdirectories, identifying sets of files with identical content through size and hash comparisons, and then listing them together so users can examine duplicates. Once duplicates are identified, the tool offers interactive deletion options where users can choose which copy to keep and which to remove, or apply flags to automate deletion while preserving a single instance. Because it operates directly on file content rather than just filenames, fdupes can accurately detect true copies and guide cleaning operations in data cleanup or migration tasks. It’s a simple, efficient, and widely used utility on Unix-like systems, appreciated by administrators, developers, and power users.

The littleutils include duplicate file finders (repeats, repeats.pl, repeats.py), image optimizers (opt-jpg, opt-png, opt-gif, recomp-jpg), file rename tools (lowercase, uppercase, pren), archive recompressors (to-gzip, to-bzip2, to-bzip3, to-7zip, to-lzma, to-lzip, to-xz), a tempfile utility (tempname), file property tools (filedate, filemode, filenode, fileown, filesize, and lrealpath), and others. See the README file for more details.

It compares files and folders, even inside compressed archives. When done, you can send the duplicate files or folders to the recycle bin, another folder or erase them, or save the list to work later.

Searches a drive (possibly with additional mounted drives) for duplicate files, outputs a potential script for deleting them after editing.

When you've been using your computer for quite a long time, often you end up duplicating your files and you don't even know about them. Duplicate File Finder is a Linux/Unix utility than scans your hard drive for finding duplicates of a given file.