Search Results for "duplicate" - Page 2
Sort By:
Showing 58 open source projects for "duplicate"
View related business solutions-
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
Bean Network Tester
Simulate bad network conditions on Windows - latency, packet loss
Bean Network Tester is a tool for testers and developers: check how your application behaves on a poor connection. Like Clumsy or NetLimiter, it lets you deliberately degrade the network - add ping, drop packets, cap the speed, tear connections down, and more. It works by intercepting traffic with the WinDivert driver (via PyDivert), and offers both a clear windowed interface with tooltips and a command-line mode for CI. https://beannetworktester.donislawdev.com/ -
2
TextureAtlas Toolbox
A powerful, free and open-source tool for TextureAtlases/Spritesheets
TextureAtlas Toolbox is an all-in-one solution for working with texture atlases and sprite sheets. Extract sprites into organized frame collections and GIF/WebP/APNG animations, generate optimized atlases from individual frames, or convert between 15+ atlas formats. Perfect for game developers, modders, and anyone creating showcases of game sprites. Formerly known as TextureAtlas to GIFs and Frames Licensed under AGPL-3.0 Third-party licenses: See... -
3
garysfm
An advanced file manager with qss themes and iso and folder previews
garysfm which stands for Gary's File Manager is a file manager with some advanced features. Those features include bulk renaming and folder image previews. I has rather advanced search functions, tab browsing with persistence between launches. It remembers your folder sorting and view options in icon view. It also remembers your active tabs between sessions. It has progress dialog while doing large operations like copying large files, and folders with many files. python version works on... -
4
text-dedup
All-in-one text de-duplication
text-dedup is a Python library that enables efficient deduplication of large text corpora by using MinHash and other probabilistic techniques to detect near-duplicate content. This is especially useful for NLP tasks where duplicated training data can skew model performance. text-dedup scales to billions of documents and offers tools for chunking, hashing, and comparing text efficiently with low memory usage. It supports Jaccard similarity thresholding, parallel execution, and flexible deduplication strategies, making it ideal for cleaning web-scraped data, language model training datasets, or document archives. -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
5
Introducing a lightweight duplicate file remover tool for Windows. With this software, you can efficiently identify duplicate files, determine the amount of storage occupied by these files, and seamlessly remove them. Free up valuable disk space by eliminating redundant copies and optimizing your storage usage. Some Antivers may show false virus alert ignore it , this tool delete files so i think thats why they think this is virus.
-
6
CleanRL
High-quality single file implementation of Deep Reinforcement Learning
...The implementation is clean and simple, yet we can scale it to run thousands of experiments using AWS Batch. CleanRL is not a modular library and therefore it is not meant to be imported. At the cost of duplicate code, we make all implementation details of a DRL algorithm variant easy to understand, so CleanRL comes with its own pros and cons. You should consider using CleanRL if you want to 1) understand all implementation details of an algorithm's variant or 2) prototype advanced features that other modular DRL libraries do not support (CleanRL has minimal lines of code so it gives you great debugging experience and you don't have to do a lot of subclassing like sometimes in modular DRL libraries). -
7
dupeGuru
Find duplicate files
dupeGuru is a cross-platform GUI application written in Python (with Qt/Cocoa UI) that quickly detects duplicate files on your computer using flexible scanning modes—including filename fuzzy matching, content comparison, and specialized Music/Picture modes. On some linux systems pyrcc5 is not put on the path when installing python3-pyqt5, this will cause some issues with the resource files (and icons). These systems should have a respective pyqt5-dev-tools package, which should also be installed. ... -
8
grab-site
Web crawler for archiving and backing up sites into WARC archives
...Users can dynamically apply ignore patterns during an active crawl, allowing them to skip problematic or unnecessary URLs that could slow down or block the archiving process. grab-site also provides predefined ignore sets for common site structures such as forums and other complex web platforms. Additional mechanisms like duplicate page detection help avoid re-crawling identical content. -
9
CC-Net
Tools to download and cleanup Common Crawl data
cc_net provides tools to download, segment, clean, and filter Common Crawl to build large-scale text corpora, including monolingual datasets and the multilingual CC-100 collection introduced in the associated paper. It includes pipelines to fetch snapshots, extract text, de-duplicate, identify language, and apply quality filtering based on heuristics and language models. The outputs are intended for pretraining language models and for creating standardized corpora that can be reproduced or updated with new crawls. The repository documents practical concerns like HTTP failures, snapshot differences, and stats JSONs, reflecting community use across many languages. ... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
ProxyBroker
Asynchronous tool for finding and checking public proxy servers
ProxyBroker is an open source Python tool designed to automatically discover and verify public proxy servers from many online sources. It operates asynchronously, allowing it to gather and test large numbers of proxies efficiently while performing multiple checks concurrently. It collects proxy addresses from dozens of providers and evaluates whether they are functional and suitable for use. It supports several proxy protocols, including HTTP, HTTPS, SOCKS4, and SOCKS5, making it flexible... -
11
MULTITAS
File management software for 64 bits windows 7/8/10 os
This program provides these features: 1) Remove duplicate files 2) Move a file from one folder to another. Download the only setup.exe file to windows os laptop then install this program. -
12
ProfileTool is a Windows 7 command line utility, for various operations on Windows 7 profiles. Its main purpose is to duplicate a profile as "default profile". It can also copy, delete, check size and delete temporary files on all profiles.
-
13
With this song management application you are able to keep an eye on your UltraStar collection. Quickly find songs with missing covers, backgrounds or videos.
-
14
IL Music Library Deduplicator
Easily find and remove duplicates from a music collection
If you want to easily find and remove audio duplicate from a music collection IL Music Library Deduplicator is for you. It can distinguish between similar songs offering you a list with best matches highlighted in green. Audio files with lower similarity are listed as well in case there are duplicates with lower bitrate. These are highlighted in yellow and red. You can select them by color if you wish to delete them or you can add them to a list with excluded files in case you wish to keep duplicates or not to be scanned next time. ... -
15
Music Merge Manager
Identify duplicate songs based on track metadata (ID3, etc)
Given two trees of audio files, examine the metadata for each file to try and determine the differences in the content of the trees. This helps find duplicate songs (or non-duplicated songs) even if the music is stored in different formats. Toggles let you determine how specific the matching criteria must be: Track number and song name? Artist, album, and track length? All of the above? Ultimately this program will let you delete duplicates from either the left or right trees, or copy non-duplicated files from one tree to the other. ... -
16
sitecheck
Modular web site spider for web developers.
More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation errors and accessibility errors. Sitecheck can also spot some common causes of PCI compliance failure such as insecure content on secure pages, SQL injection/cross-site scripting (XSS) vulnerabilities, insecure encryption ciphers and open mail relays. Sources of information leakage such as email addresses and IP addresses in the headers or the page will be logged. ... -
17
QUASR
Cross-platform NGS processing and analysis pipeline in Python
QUASR is a lightweight pipeline written to process and analyse next-generation sequencing (NGS) data from Illumina, 454, and Ion Torrent platforms. Although originally written for viral data, it is generic enough to work on any NGS dataset. Functions include: duplicate removal demultiplexing primer-removal quality-assurance (QA) graphing quality control (QC) consensus-generation minority-variant determination minority-variant graphing The main current version is 6.X, which is written in Python3. 7.X is my rewrite in Java, but is still work in progress. Both are written to be as lightweight as possible so they can run with minimal memory-requirements on a desktop or laptop as well as on a compute cluster. ... -
18
PyPedal is a Python module that provides tools for the manipulation of pedigrees, simple visualization of pedigrees, and the calculation of measures of genetic diversity from pedigrees.
-
19
JustOneFile help you to find duplicate files and managed them. It's a Free and Open Source software under GNU GPL licence.
-
20
Clone Digger is a duplicate code detection tool, which supports Python and Java languages. Discovered clones can differ in small subexpressions; comments and whitespaces are ignored. Clone digger is platform-independent and is written in Python language
-
21
This Inkscape extension provides table support for Inkscape. This is one of the features that are present in Corel Draw but are not present in Inkscape. So, just install this extension and you will have table support in Inkscape
-
22
-
23
A command line tool that allows you to (live) analyze netfilter (iptables) log files. It provides a nice output and has features like displaying hostnames, duplicate detection... More info (+ screenshots) can be found in the pdf inside the tarball.
-
24
FileUniq identifies duplicate files, and optionally removes them or replaces them with hard or soft links. It is designed to work well with heavily hard-linked cp -al / rsync backups. For more information please see http://fileuniq.sourceforge.net
-
25
Cozy is a backup solution for Linux. It aims to be similar to Apple's Time Machine, with the difference that Cozy recognizes duplicate files and incremental backups are realized through binary diffs to save space.
