Search Results for "duplicate linux" - Page 2
Sort By:
Showing 46 open source projects for "duplicate linux"
View related business solutions-
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
grab-site
Web crawler for archiving and backing up sites into WARC archives
grab-site is an open source web crawling tool designed to archive and back up websites by recursively downloading their content. It works by taking a starting URL and systematically following links across the site, capturing pages and resources and saving them into WARC archive files for long-term preservation. Internally, the crawler uses a fork of the wpull engine to fetch and process web pages efficiently during large-scale crawls. grab-site includes a built-in dashboard that displays... -
2
CC-Net
Tools to download and cleanup Common Crawl data
cc_net provides tools to download, segment, clean, and filter Common Crawl to build large-scale text corpora, including monolingual datasets and the multilingual CC-100 collection introduced in the associated paper. It includes pipelines to fetch snapshots, extract text, de-duplicate, identify language, and apply quality filtering based on heuristics and language models. The outputs are intended for pretraining language models and for creating standardized corpora that can be reproduced or... -
3
ProxyBroker
Asynchronous tool for finding and checking public proxy servers
ProxyBroker is an open source Python tool designed to automatically discover and verify public proxy servers from many online sources. It operates asynchronously, allowing it to gather and test large numbers of proxies efficiently while performing multiple checks concurrently. It collects proxy addresses from dozens of providers and evaluates whether they are functional and suitable for use. It supports several proxy protocols, including HTTP, HTTPS, SOCKS4, and SOCKS5, making it flexible... -
4
With this song management application you are able to keep an eye on your UltraStar collection. Quickly find songs with missing covers, backgrounds or videos.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Music Merge Manager
Identify duplicate songs based on track metadata (ID3, etc)
Given two trees of audio files, examine the metadata for each file to try and determine the differences in the content of the trees. This helps find duplicate songs (or non-duplicated songs) even if the music is stored in different formats. Toggles let you determine how specific the matching criteria must be: Track number and song name? Artist, album, and track length? All of the above? Ultimately this program will let you delete duplicates from either the left or right trees, or copy... -
6
sitecheck
Modular web site spider for web developers.
More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation... -
7
QUASR
Cross-platform NGS processing and analysis pipeline in Python
QUASR is a lightweight pipeline written to process and analyse next-generation sequencing (NGS) data from Illumina, 454, and Ion Torrent platforms. Although originally written for viral data, it is generic enough to work on any NGS dataset. Functions include: duplicate removal demultiplexing primer-removal quality-assurance (QA) graphing quality control (QC) consensus-generation minority-variant determination minority-variant graphing The main current version is 6.X, which is... -
8
PyPedal is a Python module that provides tools for the manipulation of pedigrees, simple visualization of pedigrees, and the calculation of measures of genetic diversity from pedigrees.
-
9
JustOneFile help you to find duplicate files and managed them. It's a Free and Open Source software under GNU GPL licence.
-
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
This Inkscape extension provides table support for Inkscape. This is one of the features that are present in Corel Draw but are not present in Inkscape. So, just install this extension and you will have table support in Inkscape
-
11
-
12
A command line tool that allows you to (live) analyze netfilter (iptables) log files. It provides a nice output and has features like displaying hostnames, duplicate detection... More info (+ screenshots) can be found in the pdf inside the tarball.
-
13
FileUniq identifies duplicate files, and optionally removes them or replaces them with hard or soft links. It is designed to work well with heavily hard-linked cp -al / rsync backups. For more information please see http://fileuniq.sourceforge.net
-
14
Cozy is a backup solution for Linux. It aims to be similar to Apple's Time Machine, with the difference that Cozy recognizes duplicate files and incremental backups are realized through binary diffs to save space.
-
15
-
16
MediaCMN provides tools to create a consistent music library. In detail these tools, index all audio files of certain directories, find duplicate files, tag the files using musicbrainz and organize the audio files in a filesystem.
-
17
Analyzes Cpp code for unused public functions, public functions which are candidates to be private functions and duplicate implementation of public functions.
-
18
ReplicateChecker is a small application allowing developpers using Hammer (Valve Map Editor) to search for duplicate entities or brushes in VMF files
-
19
IMAP4 server working on top of SQL database (currently PostgreSQL but Oracle may work too). It has some unique features like elimination of duplicate messages throughout the whole system, possibility of clustering or load balancing etc.
-
20
Lists or removes old duplicate versions of packages installed by apt-get. With this tool it's easy to keep only the newest version of each package in APT cache.
-
21
Video DeDup
Find duplicate videos by content
VIDEO DeDup Find duplicate videos by content. Parse a video directory to create one image every n seconds, then identify duplicate images and show possible video duplicates for manual analysis. . 1f_parse scan video source folder, replicate with ffmpeg an image folder, then for each image create a fingerprint. . CompareV2 perform the fingerprints comparison and works in memory with constant memory usage. It is folder agnostic and parse fingerprints ordered by source file name. Since...
