Search Results for "duplicate"
Sort By:
Showing 91 open source projects for "duplicate"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI Studio. Switch between models without switching platforms.
-
Cut Cloud Costs with Google Compute EngineSave on compute costs with Compute Engine. Reduce your batch jobs and workload bill 60-91% with Spot VMs. Compute Engine's committed use offers customers up to 70% savings through sustained use discounts. Plus, you get one free e2-micro VM monthly and $300 credit to start.
-
1
Czkawka
Multi functional app to find duplicates, empty folders, similar images
Czkawka (Polish for “hiccup”) is a lightning‑fast, multi‑purpose file cleaning tool written in Rust. It helps users declutter storage by finding duplicate files, similar images or audio, empty folders, and unusually large files through CPU‑efficient multithreading. Available with both GUI (GTK‑based) and CLI versions for flexible usage. -
2
FDUPES
FDUPES is a program for identifying or deleting duplicate files
FDUPES is a lightweight command-line utility that helps users find and optionally delete duplicate files within specified directories by comparing file contents, which can be extremely useful for cleaning up storage clutter or organizing large collections of files. It works by scanning directories and subdirectories, identifying sets of files with identical content through size and hash comparisons, and then listing them together so users can examine duplicates. -
3
Duplicate Agent
Duplicate Files Finder and Cleaner
This program was created to detect and delete unnecessary files on your computer that you've unknowingly created, copied, or backed up in some way. You can find the release notes on my Github address: https://github.com/shampuan/Duplicate-Agent/issues -
4
duf
Disk usage and free utility, a better 'df' alternative
...You can simply start duf without any command-line arguments. If you supply arguments, duf will only list specific devices & mount points. If you want to list everything (including pseudo, duplicate, inaccessible file systems), you can. You can show and hide specific tables. You can also show and hide specific filesystems. List inode information instead of block usage. If duf doesn't detect your terminal's colors correctly, you can set a theme. Users of oh-my-zsh should be aware that it already defines an alias called duf, which you will have to remove in order to use duf. -
$300 in Free Credit Across 150+ Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale with Google Cloud.
-
5
Text Line Duplicate Remover
Remove duplicate lines from your text
This standalone offline web browser tool helps you remove duplicate lines from your text, with additional text processing options. Simply open it in your browser by double-clicking the html file. It also includes the source code too. I made this when I was working with long lists of entries and needed something to automatically clean them up. As a bonus you can also change the Sentence Case of the text, make it lowercase, UPPERCASE or Sentence case. -
6
littleutils
Various small and useful command-line utilities
The littleutils include duplicate file finders (repeats, repeats.pl, repeats.py), image optimizers (opt-jpg, opt-png, opt-gif, recomp-jpg), file rename tools (lowercase, uppercase, pren), archive recompressors (to-gzip, to-bzip2, to-bzip3, to-7zip, to-lzma, to-lzip, to-xz), a tempfile utility (tempname), file property tools (filedate, filemode, filenode, fileown, filesize, and lrealpath), and others. -
7
Swiss File Knife
One hundred command line tools in a small and portable binary.
Create zip files, extract zip files, replace text in files, search in files using expressions, stream text editor, instant command line ftp and http server, send folder via network, copy folder excluding sub folders and files, find duplicate files, run a command on all files of a folder, split and join large files, make md5 checksum lists of files, remove tab characters, convert CR/LF, list newest or biggest files of a folder, compare folders, treesize, show first or last lines of a file, find filenames fast using index files, rename many files using expressions, copy part of a file, change times of a file, set file time from filename, print colored text to terminal, convert csv to tab separated, download files from web, send http or udp requests, print tcp or udp traffic, create hexdump of files, join many text files into one, list nested .zip .tar .tar.gz .tar.bz2 archive contents. ... -
8
PeaZip
Free Zip software and Rar extractor
...Create: 7Z, ARC, Brotl, BZip, GZip, PEA, TAR, WIM, XZ, ZPAQ, ZIP, Zstandard files and more Open and extract 200+ file types: ACE, CAB, DEB, ISO, RAR, ZIPX and more Features of PeaZip includes extract, create and convert multiple archives at once, create self-extracting archives (sfx), split files, strong encryption with two factor authentication, encrypted password manager, secure deletion, find duplicate files, calculate hashes, export task definition as command line script. -
9
Photo Agent
Duplicate Photo Finder and Cleaner for Linux
The program is very easy to use. Simply point to the path and click scan. Mark any unwanted photos and click "Send to Fake Trash." Then, switch to the Fake Trash tab, make sure you've selected the correct files, and select and delete them all (they can no longer be recovered). If you want to return any, simply mark them and recover them (Restore Selected). If the directory you're pointing to is very large or complex, the program may freeze for a moment when you press the scan button. This is... -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
local_search
Load filenames to a local database for an easy search
Load filenames to a local database for an easy search. This program is made with my Swedish Compiler. That is focusing on a simple Gui and support many different databases (PostgreSQL, MariaDB, SqLite, Odbc). It’s working under Linux (Wine). -
11
Blazefox
Blazefox: Fast, safe file deduplication and management tool for Linux
Blazefox is a powerful and efficient file deduplication and management tool designed for both everyday users and system administrators. It scans directories recursively, identifies duplicate files using advanced content hashing, and provides safe options to remove, move, or rename duplicates. Blazefox supports flexible filtering with regex/glob patterns, interactive and dry-run modes, and advanced conflict resolution strategies, ensuring no data is lost accidentally. Its undo and redo functionality allows users to safely reverse operations, giving complete control over file management tasks. ... -
12
File-Studio
A tool that automates complex file operations.
File studio is a tool that assists in handling complex file operations such as bulk renaming, organizing folders and more. -
13
garysfm
An advanced file manager with qss themes and iso and folder previews
garysfm which stands for Gary's File Manager is a file manager with some advanced features. Those features include bulk renaming and folder image previews. I has rather advanced search functions, tab browsing with persistence between launches. It remembers your folder sorting and view options in icon view. It also remembers your active tabs between sessions. It has progress dialog while doing large operations like copying large files, and folders with many files. python version works on... -
14
MediaFileDuplicateFinder
MediaFileDuplicateFinder finds duplicated video, image, & audio files
Media File Duplicate Finder finds duplicated video and image files based on similarity. Finds duplicates which have different resolution, frame rate, watermark, and video file tags. It's a cross-platform program that supports Windows, Linux and Mac-OS. Please use below link to report bugs, feature-request, or questions. https://github.com/David-Maisonave/MediaFileDuplicateFinder/issues/new/choose For more information, visit the following Wiki page: https://github.com/David-Maisonave/MediaFileDuplicateFinder/wiki -
15
CSI-EXPLOREFS
Tools Explore the List of Files based Mount
... - The tools was created to make one list of entries inside one filesystem Unix or Windows Volumes. - The list can be used to map the state of usage, inventory, extract ranges of lists based dates, propose a list of duplicate files for analysis or removal. - The options and results can be mapped looking the online help. See "ExploreFS --help" and "ExploreDiff --help" * Support: The Support Service is FREE. Do you need support? Open a ticket and I will get back to you as soon as possible. * Professional Services: Do you need any FREE professional services to use? ... -
16
CDC File Transfer
Tools for synching and streaming files from Windows to Linux
...Unlike fixed-size blocks, content-defined chunks are cut at data-dependent boundaries, which means shifts and insertions don’t invalidate the rest of the file’s delta map. The tool maintains chunk fingerprints and a manifest so it can de-duplicate across versions and even across different files that share regions of content. This approach makes it well suited for large logs, VM images, and binary assets where small edits would otherwise force full re-uploads. The implementation focuses on being robust over unreliable networks with resumable transfers and integrity checks that verify every chunk. ... -
17
dupeGuru
Find duplicate files
dupeGuru is a cross-platform GUI application written in Python (with Qt/Cocoa UI) that quickly detects duplicate files on your computer using flexible scanning modes—including filename fuzzy matching, content comparison, and specialized Music/Picture modes. On some linux systems pyrcc5 is not put on the path when installing python3-pyqt5, this will cause some issues with the resource files (and icons). These systems should have a respective pyqt5-dev-tools package, which should also be installed. ... -
18
link-dups
Generate hard links for duplicate files
Searches for files that are duplicates of each other and creates a script that will cause hard links to replace files that are duplicated so saving disk space. Download executable and type 'link-dups -h' for command line help. -
19
SortPhotos
SortPhotos is a Python script that organizes photos and videos
...SortPhotos includes options for copying versus moving files, recursive searches, silent or test modes, and customizable start times for when a “day” begins. It also prevents duplicate files by comparing content, with an option to keep duplicates if needed. With support for automation through launch agents or cron jobs, SortPhotos is well-suited for photographers, archivists, and anyone looking to streamline large personal or professional media collections. -
20
hk-dup-finder
hadi kiamarsi duplicate file finder
hk-dup-finder is a free tool for find duplicate file in your computers . this software is written by Hadi Kiamarsi. -
21
sfk
Swiss File Knife - A Command Line Tools Collection
SFK combines many functions in a single, portable executable that belongs onto every USB stick. Search and convert text files, instant simple FTP/HTTP server, find duplicate files, compare folders, treesize, run own commands on all files of a folder - it's all within a single tool. -
22
CTREE Graphic Tree Generator for C
An analysis tool showing the structure of a C program
CTREE is a command line utility providing analysis tools useful to software engineers tasked with tracking down problems or verifying a program's soundness. It is also an aid to understanding the structure of unfamiliar programs. As it works with standard C it finds most of its utility in embedded systems and older software. CTREE comes with a comprehensive manual. CTREE is is released by Cleveland Engineering Design, LLC - developer of the CoRTOS cooperative... -
23
fdf
FDF: Find Duplicate Files
Searches a drive (possibly with additional mounted drives) for duplicate files, outputs a potential script for deleting them after editing. -
24
Scrollout F1
An easy-to-use anti-spam email gateway
• Designed for Linux and Windows email system administrators, Scrollout F1 is an easy to use, already adjusted email firewall (gateway) offering free anti-spam and anti-virus protection aiming to secure existing email servers, old or new, such as Microsoft Exchange, Lotus Domino, Postfix, Exim, Sendmail, Qmail and others. • Built-in multilayer security levels make configuration effort equal to a car radio. • It combines simplicity with effective protection using powerful open source... -
25
h3d
Heavy Duty Directory Deduplicator
A tool to detect duplicate or similar directories on locally accessible drives. The tool is currently linux-specific, but can be extended easily to run anywhere Java runs.
