Search Results for "remove duplicate"
Sort By:
Showing 72 open source projects for "remove duplicate"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
FDUPES
FDUPES is a program for identifying or deleting duplicate files
FDUPES is a lightweight command-line utility that helps users find and optionally delete duplicate files within specified directories by comparing file contents, which can be extremely useful for cleaning up storage clutter or organizing large collections of files. It works by scanning directories and subdirectories, identifying sets of files with identical content through size and hash comparisons, and then listing them together so users can examine duplicates. -
2
Video-subtitle-extractor
A GUI tool for extracting hard-coded subtitle (hardsub) from videos
Video hard subtitle extraction, generate srt file. There is no need to apply for a third-party API, and text recognition can be implemented locally. A deep learning-based video subtitle extraction framework, including subtitle region detection and subtitle content extraction. A GUI tool for extracting hard-coded subtitles (hardsub) from videos and generating srt files. Use local OCR recognition, no need to set up and call any API, and do not need to access online OCR services such as Baidu... -
3
RemoveDuplicate
A software that can helps you remove duplicate files
RemoveDuplicate is a Windows application designed to help users find and remove duplicate files within folders. It uses MD5 hashing to identify identical files and provides a user-friendly interface to manage and delete duplicates while keeping one copy of each file. -
4
RemoveDuplicateLinesFromTxt
Remove Duplicate lines From Files in TXT
This program remove duplicate lines in txt files. -
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
5
Duplicate Photo Finder
Free duplicate & similar photo finder & cleaner for Windows
Duplicate Photo Finder is a free Windows app that finds and deletes duplicate photos and similar images in seconds. It detects exact duplicate photos (identical copies) and visually similar pictures - burst shots, resized and messenger copies, screenshots and edited versions - then automatically keeps the best-quality copy and moves the rest to the Recycle Bin. Free up disk space and clean your photo library without losing a single original. No ads, no sign-up, no watermark - every... -
6
Text Line Duplicate Remover
Remove duplicate lines from your text
This standalone offline web browser tool helps you remove duplicate lines from your text, with additional text processing options. Simply open it in your browser by double-clicking the html file. It also includes the source code too. I made this when I was working with long lists of entries and needed something to automatically clean them up. As a bonus you can also change the Sentence Case of the text, make it lowercase, UPPERCASE or Sentence case. -
7
Grabber
Very customizable imageboard/booru downloader with powerful filenaming
Very customizable imageboard/booru browser and mass downloader, with powerful file naming features. Thanks to its powerful naming features, you just have to set your filename and save the directory using all the tokens available, and the program will generate a filename using the image's information. With this, you can store and manage your pictures in advanced directory structures, and save images with custom filenames. Imageboard/booru downloader which can download thousands of images from... -
8
duf
Disk usage and free utility, a better 'df' alternative
...You can simply start duf without any command-line arguments. If you supply arguments, duf will only list specific devices & mount points. If you want to list everything (including pseudo, duplicate, inaccessible file systems), you can. You can show and hide specific tables. You can also show and hide specific filesystems. List inode information instead of block usage. If duf doesn't detect your terminal's colors correctly, you can set a theme. Users of oh-my-zsh should be aware that it already defines an alias called duf, which you will have to remove in order to use duf. -
9
Meta Package Manager
Wraps all package managers with a unifying CLI
Meta Package Manager wraps all package managers with a unifying CLI, and provides the MPM CLI, a wrapper around all package managers. MPM is like yt-dlp, but for package managers instead of videos. MPM solves XKCD #1654 - Universal Install Script. List installed packages. List duplicate installed packages. Search for packages. Install a package, remove a package, and list outdated packages. Sync local package infos. Upgrade all outdated packages. Backup list of installed packages to TOML file. Restore/install list of packages from TOML files. Pin-point commands to a subset of package managers (include/exclude selectors). ... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
SortMyShit
A tool designed to help you organize and manage your files
SortMyShit is an open-source Python project designed to help you organize and manage your files effortlessly. It provides customizable sorting rules to keep your directories clean and structured. -
11
NeatDisk
Disk cleaner and duplicate finder for Windows. Keep your drives clean.
...It gives you a clear picture of where your storage is going and the tools to take it back — without the bloat of traditional PC cleaners. Features: - Junk file cleaner — remove temporary files, browser caches, and Windows leftovers - Duplicate finder — find and delete exact duplicate files wasting space across your drives - Large file finder — quickly surface the biggest space hogs on any drive or folder - Disk usage analyzer — visual breakdown of what's eating your storage - Driver store cleaner — safely prune outdated driver packages from the Windows Driver Store - Stale file detector — identify files you haven't touched in months or years - Startup manager — control which programs launch at boot to keep Windows snappy - App uninstaller — cleanly remove installed programs - Restore point manager — review and trim old system restore points - Disk health monitor — check drive S.M.A.R.T. sta -
12
ParDRe
Parallel tool to remove duplicate DNA reads
ParDRe is a parallel tool to remove duplicate reads. Duplicate reads can be seen as identical or nearly identical sequences with some mismatches. This tool will let the users to avoid the analysis of not necessary reads, reducing the time of subsequent procedures with the dataset (e.g., assemblies, mappings, etc.). The tool is implemented with MPI in order to exploit the parallel capabilities of multicore clusters. -
13
Swiss File Knife
One hundred command line tools in a small and portable binary.
Create zip files, extract zip files, replace text in files, search in files using expressions, stream text editor, instant command line ftp and http server, send folder via network, copy folder excluding sub folders and files, find duplicate files, run a command on all files of a folder, split and join large files, make md5 checksum lists of files, remove tab characters, convert CR/LF, list newest or biggest files of a folder, compare folders, treesize, show first or last lines of a file, find filenames fast using index files, rename many files using expressions, copy part of a file, change times of a file, set file time from filename, print colored text to terminal, convert csv to tab separated, download files from web, send http or udp requests, print tcp or udp traffic, create hexdump of files, join many text files into one, list nested .zip .tar .tar.gz .tar.bz2 archive contents. ... -
14
DupRem
Simple application to remove duplicate and empty lines on text files.
DupRem is a simple easy-to-use cross-platform application to remove duplicate and empty lines from any text file. It is also possible to keep or ignore case sensitive. Works also from command-line interface, e.g. "java -jar duprem.jar -r input_file.txt >output_file.txt", to create output file, or "java -jar duprem.jar -r input_file.txt >>output_file.txt", to create or append to output file. DupRem is portable, does not need installation and is developed in Java, so needs the Java Virtual Machine (version 6 or higher) that, in the case, is very easy to download and install. -
15
ImageMerge
A GUI tool to merge images/videos from multiple folders.
A GUI tool to merge images/videos from multiple folders, remove duplicates with SHA-256, and rename files into a clean sequential format. -
16
Blaiz Tools
Tools for working with Gossamer codebase and app source code
Tools for packing files, converting images, manipulating text, checking code and working with the Gossamer codebase and app source code. -
17
Image Comparator
Find and remove duplicate or similar images.
Image Comparator is a robust Windows desktop application designed to identify and manage duplicate or visually similar images using advanced Discrete Cosine Transform (DCT) algorithms. Unlike standard comparison tools, it effectively detects images that have been resized, compressed, or slightly modified. -
18
Blazefox
Blazefox: Fast, safe file deduplication and management tool for Linux
Blazefox is a powerful and efficient file deduplication and management tool designed for both everyday users and system administrators. It scans directories recursively, identifies duplicate files using advanced content hashing, and provides safe options to remove, move, or rename duplicates. Blazefox supports flexible filtering with regex/glob patterns, interactive and dry-run modes, and advanced conflict resolution strategies, ensuring no data is lost accidentally. Its undo and redo functionality allows users to safely reverse operations, giving complete control over file management tasks. ... -
19
Virtualdub Batch Video DeShake v26.0204
Batch to compress [and deshake] all videos [or images] in folder
Installation: Execute "DeShakInst.BAT" VirtualDub2 44282; AviSynth+ 3.7.5 updated to C:\DVD DESHAK.BAT updated to C:\UT and added to PATH Usage: DESHAK task[s] [parameters] Tasks: tp1: deshake pass1 LOG generation for 2nd pass tp2: deshake pass2 and compress video and audio to MP3 tcomp: compress (no deshake) twav: extract WAV and/or uses external WAV audio Parameters (more in help): vEXT: video extension (ie: vmov), default: vAVI qN: h264... -
20
Introducing a lightweight duplicate file remover tool for Windows. With this software, you can efficiently identify duplicate files, determine the amount of storage occupied by these files, and seamlessly remove them. Free up valuable disk space by eliminating redundant copies and optimizing your storage usage. Some Antivers may show false virus alert ignore it , this tool delete files so i think thats why they think this is virus.
-
21
vConsole
A lightweight, extendable front-end developer tool for mobile web page
...After importing vConsole, logs will be printed to both front-end console and native console. Some useful vConsole properties and methods are available for plugin development. You can destroy an vConsole instance object and remove vConsole panel from document. You can also add a new plugin to vConsole. Duplicate plugin will be ignored. Create an vConsole plugin object, bind plugin events to this object, and add this object to vConsole. While installing and running a plugin, vConsole will trigger some events to allow a plugin customizing it's functions. -
22
Remove Duplicate Lines
Remove duplicate lines in text file
A handy tool with graphical interface that remove/delete duplicate lines in a text file. GitHub source: https://github.com/ahmed-fathy/remove-duplicates/ -
23
Delete Duplicate Folder Levels
A spreadsheet to collapse redundant folders
This project offers an Excel spreadsheet, available for download through the Files tab (above). This spreadsheet helps the user to identify and remove duplicate directory levels. For instance, it can reduce "D:\Folder Layer\Folder Layer\Folder Layer\Another Layer\Another Layer\Filename.doc" to "D:\Folder Layer\Another Layer\Filename.doc" Duplicate folder layers can form due to normal file renaming processes, and also when restoring from archive (e.g., ZIP, RAR) files that contain folder layers. ... -
24
Abdal Duplicate Remover
Remove duplicate content and more features
Remove duplicate content and more features for txt file -
25
Sucky Clipboard
A clipboard tool with a history function.
A useful tool when having to copy and paste several strings from one application to another. Imagine testing a payment gateway with set card number, name, and address details and having to switch between applications repeatedly to copy and paste each detail one-by-one. Instead, Sucky Clipboard allows you to copy each of the detail lines from one application, stores them on its clipboard interface, and then paste each line in turn into the destination fields.
