Search Results for "remove duplicate files"
Sort By:
Showing 740 open source projects for "remove duplicate files"
View related business solutions-
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
Earn up to 16% annual interest with Nexo.Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
-
1
FDUPES
FDUPES is a program for identifying or deleting duplicate files
FDUPES is a lightweight command-line utility that helps users find and optionally delete duplicate files within specified directories by comparing file contents, which can be extremely useful for cleaning up storage clutter or organizing large collections of files. It works by scanning directories and subdirectories, identifying sets of files with identical content through size and hash comparisons, and then listing them together so users can examine duplicates. ... -
2
Files for Meteor.js
Upload files via DDP or HTTP to Meteor server FS
Stable, fast, robust, and well-maintained Meteor.js package for file management using MongoDB Collection API. What does exactly this means? Calling .insert() method would initiate a file upload and then insert a new record into the collection. Calling .remove() method would erase stored file and records from MongoDB Collection. And so on, no need to learn new APIs. It's flavored with extra low-level methods like .unlink() and .write() for complex integrations. -
3
Czkawka
Multi functional app to find duplicates, empty folders, similar images
Czkawka (Polish for “hiccup”) is a lightning‑fast, multi‑purpose file cleaning tool written in Rust. It helps users declutter storage by finding duplicate files, similar images or audio, empty folders, and unusually large files through CPU‑efficient multithreading. Available with both GUI (GTK‑based) and CLI versions for flexible usage. -
4
RemoveDuplicate
A software that can helps you remove duplicate files
RemoveDuplicate is a Windows application designed to help users find and remove duplicate files within folders. It uses MD5 hashing to identify identical files and provides a user-friendly interface to manage and delete duplicates while keeping one copy of each file. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
RemoveDuplicateLinesFromTxt
Remove Duplicate lines From Files in TXT
This program remove duplicate lines in txt files. -
6
Video-subtitle-extractor
A GUI tool for extracting hard-coded subtitle (hardsub) from videos
...Filter the text in the non-subtitle area and remove the watermark (station logo) text. -
7
Duplicate Agent
Duplicate Files Finder and Cleaner
This program was created to detect and delete unnecessary files on your computer that you've unknowingly created, copied, or backed up in some way. You can find the release notes on my Github address: https://github.com/shampuan/Duplicate-Agent/issues -
8
Ultimate Vocal Remover (UVR5)
GUI for a Vocal Remover that uses Deep Neural Networks
This application uses state-of-the-art source separation models to remove vocals from audio files. UVR's core developers trained all of the models provided in this package (except for the Demucs v3 and v4 4-stem models). -
9
Allows the removal of the com.apple.quarantine extended attribute from files and folders displayed in a Finder application window. This extend attribute is usually placed on downloads from the internet. In the case of folders, the removal action is recursive (so that every file and subfolder in the folder is acted upon).
-
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
SortMyShit
A tool designed to help you organize and manage your files
SortMyShit is an open-source Python project designed to help you organize and manage your files effortlessly. It provides customizable sorting rules to keep your directories clean and structured. -
11
Bunkr Downloader
Yet bunkrr album downloader
...The tool allows users to input album URLs and automatically download all associated media while maintaining a local metadata database for tracking content. It includes features for managing albums as a persistent library, enabling users to add, remove, or sync albums over time without re-downloading existing files. bonkrr also categorizes downloaded content into different media types such as images, videos, and archives, improving organization and usability. The use of SQLite for local storage ensures efficient tracking of downloads and metadata synchronization. Overall, bonkrr is a practical utility for users who want to maintain organized offline collections of media from online album sources. -
12
Video-subtitle-remover (VSR)
AI tool that removes hardcoded subtitles and text from videos locally
Video Subtitle Remover is an AI-based application designed to remove hardcoded subtitles from videos and generate new files without the embedded text. Video Subtitle Remover analyzes video frames and detects subtitle regions, then replaces the removed areas using an AI algorithm that fills the space with reconstructed visual content. This process aims to maintain the original resolution and visual continuity of the video after subtitle removal. -
13
Mole
🐹 Deep clean and optimize your Mac.
...It aims to consolidate the functionality of multiple popular Mac utilities—cleaners, uninstallers, disk analyzers, and system tuners—into a single open-source binary. From the command line, you can scan and remove large caches, logs, temporary files, browser remnants, and other junk that quietly accumulates over time. Mole’s smart uninstall mode walks the filesystem to remove not just an application bundle, but also associated caches, preferences, launch agents, logs, and other leftovers in more than twenty locations. It also includes optimization routines that rebuild caches, reset network services, trim swap files, and refresh indexes to keep the system feeling responsive. ... -
14
SortPhotos
SortPhotos is a Python script that organizes photos and videos
...SortPhotos includes options for copying versus moving files, recursive searches, silent or test modes, and customizable start times for when a “day” begins. It also prevents duplicate files by comparing content, with an option to keep duplicates if needed. With support for automation through launch agents or cron jobs, SortPhotos is well-suited for photographers, archivists, and anyone looking to streamline large personal or professional media collections. -
15
BFG Repo-Cleaner
Remove large or troublesome blobs
The BFG is a simpler, faster alternative to git-filter-branch for cleansing bad data out of your Git repository history. You can use it for removing crazy big files, and for removing passwords, credentials and other private data. The git-filter-branch command is enormously powerful and can do things that the BFG can't, but the BFG is much better for the tasks above, because is faster and simpler. The BFG isn't particularily clever, but is focused on making the above tasks easy. If you need... -
16
Grabber
Very customizable imageboard/booru downloader with powerful filenaming
Very customizable imageboard/booru browser and mass downloader, with powerful file naming features. Thanks to its powerful naming features, you just have to set your filename and save the directory using all the tokens available, and the program will generate a filename using the image's information. With this, you can store and manage your pictures in advanced directory structures, and save images with custom filenames. Imageboard/booru downloader which can download thousands of images from... -
17
S5CMD
Parallel S3 and local filesystem execution tool
...Upload, download or delete objects. Move, copy or rename objects. Set Server Side Encryption using AWS Key Management Service (KMS). Set Access Control List (ACL) for objects/files on the upload, copy, move. Print object contents to stdout. Select JSON records from objects using SQL expressions. Create or remove buckets. Summarize objects sizes, grouping by storage class. Wildcard support for all operations. -
18
supabase-py
Python Client for Supabase. Query Postgres from Flask, Django
Python Client for Supabase. Query Postgres from Flask, Django, FastAPI. Python user authentication, security policies, edge functions, file storage, and realtime data streaming. Good first issue. -
19
Meta Package Manager
Wraps all package managers with a unifying CLI
Meta Package Manager wraps all package managers with a unifying CLI, and provides the MPM CLI, a wrapper around all package managers. MPM is like yt-dlp, but for package managers instead of videos. MPM solves XKCD #1654 - Universal Install Script. List installed packages. List duplicate installed packages. Search for packages. Install a package, remove a package, and list outdated packages. Sync local package infos. Upgrade all outdated packages. Backup list of installed packages to TOML file. Restore/install list of packages from TOML files. Pin-point commands to a subset of package managers (include/exclude selectors). ... -
20
Mac Cleaner CLI
Scan and remove junk files, caches, logs, and more
Mac Cleaner CLI is a free and open-source terminal-based utility that helps users scan, identify, and remove unnecessary files from their macOS systems to reclaim storage space and keep systems tidy. Through a simple command-line interface, the tool performs deep scans to find caches, temporary files, logs, browser data, and other clutter, presenting results in an organized interactive menu where users can choose exactly what to clean. -
21
Duplicate Photo Finder
Free duplicate & similar photo finder & cleaner for Windows
Duplicate Photo Finder is a free Windows app that finds and deletes duplicate photos and similar images in seconds. It detects exact duplicate photos (identical copies) and visually similar pictures - burst shots, resized and messenger copies, screenshots and edited versions - then automatically keeps the best-quality copy and moves the rest to the Recycle Bin. Free up disk space and clean your photo library without losing a single original. No ads, no sign-up, no watermark - every... -
22
Swiss File Knife
One hundred command line tools in a small and portable binary.
Create zip files, extract zip files, replace text in files, search in files using expressions, stream text editor, instant command line ftp and http server, send folder via network, copy folder excluding sub folders and files, find duplicate files, run a command on all files of a folder, split and join large files, make md5 checksum lists of files, remove tab characters, convert CR/LF, list newest or biggest files of a folder, compare folders, treesize, show first or last lines of a file, find filenames fast using index files, rename many files using expressions, copy part of a file, change times of a file, set file time from filename, print colored text to terminal, convert csv to tab separated, download files from web, send http or udp requests, print tcp or udp traffic, create hexdump of files, join many text files into one, list nested .zip .tar .tar.gz .tar.bz2 archive contents. ... -
23
diskover-community
Open source file indexing & storage analytics powered by Elasticsearch
...Diskover also helps identify outdated or unused files, duplicate data, and inefficient storage usage that can waste resources or increase operational costs. A Python-based indexing engine performs the scanning and indexing tasks. -
24
claude-real-video
Let Claude (or any LLM) actually watch a video
claude-real-video is a local video processing tool that prepares videos for analysis by Claude or other LLMs. It accepts public video URLs or local files and extracts the visual and audio evidence an AI model needs. Instead of sampling at a fixed interval, it detects scene changes and removes near-duplicate frames to reduce unnecessary token usage. It can transcribe audio, generate a clean output folder, and create a local viewer with the video, keyframe grid, and transcript. The tool can also be installed as a Claude Code skill so an agent can process videos more directly. ... -
25
Text Line Duplicate Remover
Remove duplicate lines from your text
This standalone offline web browser tool helps you remove duplicate lines from your text, with additional text processing options. Simply open it in your browser by double-clicking the html file. It also includes the source code too. I made this when I was working with long lists of entries and needed something to automatically clean them up. As a bonus you can also change the Sentence Case of the text, make it lowercase, UPPERCASE or Sentence case.
