Search Results for "remove duplicates"
Sort By:
Showing 23 open source projects for "remove duplicates"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
FDUPES
FDUPES is a program for identifying or deleting duplicate files
...It works by scanning directories and subdirectories, identifying sets of files with identical content through size and hash comparisons, and then listing them together so users can examine duplicates. Once duplicates are identified, the tool offers interactive deletion options where users can choose which copy to keep and which to remove, or apply flags to automate deletion while preserving a single instance. Because it operates directly on file content rather than just filenames, fdupes can accurately detect true copies and guide cleaning operations in data cleanup or migration tasks. ... -
2
RemoveDuplicate
A software that can helps you remove duplicate files
RemoveDuplicate is a Windows application designed to help users find and remove duplicate files within folders. It uses MD5 hashing to identify identical files and provides a user-friendly interface to manage and delete duplicates while keeping one copy of each file. -
3
ParDRe
Parallel tool to remove duplicate DNA reads
...It is faster than multithreaded counterparts (end of 2015) for the same number of cores and, thanks to the message-passing technology, it can be executed on clusters. There also exists a MapReduce counterpart of ParDRe, called MarDRe (see the link above). UPDATE: From version 2.0.5 ParDRe also provides support to remove only optical duplicates (and leave biologically interesting duplicates) as well as to work with compressed input/output with .gz format. -
4
MangoDisk
Open-source disk cleaner for Windows, macOS and Linux
...The resource panel shows CPU, memory, network and disk activity. Scans run locally and don't delete files. Choose what to remove and confirm before cleanup. No signup or login is required. Unsure about a file or an app? Optional online AI explanations cover cleanup items, startup entries, large files, duplicates and app uninstall. They explain purpose, impact and cautions; they don't delete files or replace your judgment. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Tartube
Download videos/channels/playlists from YouTube and many other sites
Tartube is a GUI front-end for youtube-dl, yt-dlp and other compatible video downloaders. It is written in Python 3 / Gtk 3 and runs on MS Windows, Linux, BSD and MacOS. -
6
Blazefox
Blazefox: Fast, safe file deduplication and management tool for Linux
Blazefox is a powerful and efficient file deduplication and management tool designed for both everyday users and system administrators. It scans directories recursively, identifies duplicate files using advanced content hashing, and provides safe options to remove, move, or rename duplicates. Blazefox supports flexible filtering with regex/glob patterns, interactive and dry-run modes, and advanced conflict resolution strategies, ensuring no data is lost accidentally. Its undo and redo functionality allows users to safely reverse operations, giving complete control over file management tasks. ... -
7
SafeUtils
110+ developer tools as native MacOS, Linux & Windows desktop apps.
Tools: https://safeutils.com/barcode-generator https://safeutils.com/color-picker https://safeutils.com/qr-code-generator https://safeutils.com/qr-code-scanner https://safeutils.com/word-counter https://safeutils.com/base-64-decoder https://safeutils.com/diff-checker https://safeutils.com/hex-to-ascii https://safeutils.com/json-formatter https://safeutils.com/lorem-ipsum-generator https://safeutils.com/random-generator https://safeutils.com/time-converter https://safeutils.com/... -
8
Stock Photo Tagger
Simple tool for tagging photos for stock photo agencies
Based on tool "Image Tagger", adapted for stock photo agencies - handles multiple keywords as a long string with separator, Title and Description which can be automatically recognized by stock photo agencies. Keywords can be sorted and duplicates removed. Metadata can be applied on multiple selected images. All metadata can be copied and pasted from the image from different directory. If there is metadata missing, the image name is shown in red. Whats new in version 1.1.0: Added context... -
9
Angular-filter
Bunch of useful filters for AngularJS with no external dependencies!
Bunch of useful filters for AngularJS (with no external dependencies!). You can install angular-filter using 4 different methods. Include angular-filter.js (or angular-filter.min.js) in your index.html, after including Angular itself. Add 'angular.filter' to your main module's list of dependencies. If a string is provided, it will filter out duplicates using the provided expression. -
PRTG Catches Network Issues Before They Cause DowntimeReactive troubleshooting usually means hearing about a problem from frustrated users, not your monitoring tool. PRTG sets threshold-based alerts across devices, servers and applications, notifying your team by email, SMS or push the moment a metric crosses a set limit. That means catching a failing disk or overloaded server before it becomes an outage and getting time back from firefighting. Start a free trial and set your first alerts today.
-
10
Uniq Copy
Java Desktop application which helps de-duplicate files during copy.
UniqCopy scans a source folder to get a list of all the files and then guesses duplicated files based on file name and file size. Using this information, it copies all files to the target folder, maintaining the relative structure of the files within the target folder. Files considered to have been duplicated in the source folder are copied only once to one of the matching locations in the target folder, and the other locations are skipped. A summary of these "skipped duplicates" is... -
11
ClinQC
ClinQC: A tool for quality control of Sanger and NGS data in clinic
...It can analyze raw sequencing data and produces unified output as FASTQ files per sample/patient with Sanger quality encoding. First, ClinQC convert input read files from their native formats to a common FASTQ format and remove adapters, and PCR primers. Next, it split barcoded samples, filter duplicates, contamination and low quality sequences and generates a QC report. -
12
toastr
Simple javascript toast notifications
toastr is a Javascript library for Gnome / Growl type non-blocking notifications. jQuery is required. The goal is to create a simple core library that can be customized and extended. Toastr will supply default animations, so you do not have to provide any of these settings. However you have the option to override the animations if you like. Optionally override the animation easing to show or hide the toasts. Default is swing. swing and linear are built into jQuery. Use the jQuery show/hide... -
13
duplicate
Remove duplicate files using binary compression
Utility that checks all files for given directory and removed the duplicates by binary compression comparison. -
14
GDuplicateFinder
GDuplicateFinder - A FREE Groovy way to find file duplicates!
Unlike other duplicate finders such as Easy Duplicate File Finder, Auslogics Duplicate File Finder, MindGems Fast Duplicate File Finder, Duplicate Cleaner, DupFiles and others, GDuplicateFinder is a FREE cross-platform application, with the ability to search among not just local files, but files on the network, such as a Windows or a Linux share using VFS library. Taking advantage of Groovy facilities and GPars power to process in parallel, GDuplicateFinder will help you get rid of those... -
15
Toolsverse ETL Framework
Open source Extract Transform Load engine written in Java
ETL Framework is a standalone Extract Transform Load engine written in Java. It includes executables for all major platforms and can be easily integrated into other applications. Key Features: * embeddable, open source and free * fast and scalable * uses target database features to do transformations and loads * manual and automatic data mapping * data streaming * bulk data loads * data quality features using SQL, JavaScript? and regex * data transformations Requirements *... -
16
n way Duplicate File / Directory Finder
please use open source- htp://code.google.com/p/off-dup-finder/
...This project Manages duplicate file / directories in offline/online mode . 'ls-laR' output(offline) from 'ubuntu' linux can be given to this program to generate copy/diff/automated file/dir remove scripts for one way or two way comparison. disk gain, incremental comparison, patterns partial match/timer, copy script generator- are supported. Can be setup in a central server to clean up network duplicates. This is free software and partly open source. Has good speed And Low Memory usage!! License is GPL V3. -
17
Outlook LookForDuplicates
Find and remove duplicate items in Outlook such as contacts and mails.
Find&remove duplicates easily! This VBA-Script with a form searches for duplicate entries of any type (calendar, contact, e-mail, notes, task, etc.) across folders in Outlook. Suspected doublets are listed and can be moved to the folder for Deleted Items. -
18
This is a simple Python script which allows you to remove duplicate lines from a text file. This application was designed specifically for creating word-list dictionaries and ensuring no duplicates exist. Use files with one entry per line.
-
19
Doppioni searches for duplicates in your disks. I wrote Doppioni as exercise, after one year without coding in Java. At the moment is just an alpha version. Help would be very apreciated.... want to join this project?
-
20
Java Graphical app that easily locate and remove duplicates. StopDuplicates find real duplicates using MD5 anywhere in a directory and all its subdirectories.
-
21
Searches for font files, sorts them into folders by name and look and with human readable names, stores gathered font information in a database, can generate previews and posters, can find and remove duplicates and can reunite PostScript font families.
-
22
PyBookmark manipulates bookmark files. It can sync files (no server required), merge, sort, remove duplicates, and check links. Its library pybookmarklib provides access to these operations, data structures, and parser for further extensibility.
-
23
Video DeDup
Find duplicate videos by content
...Then a lot of options and cache mechanism enable to get correct performance. At the end all duplicates are copied in an analysed folder. . To finalise you'll have to look at analysed folder to make your decisions : remove duplicate, flag some images as irrelevent or flag pair of videos as NOT dupes.
- Previous
- You're on page 1
- Next
