Search Results for "remove duplicate lines"
Sort By:
Showing 116 open source projects for "remove duplicate lines"
View related business solutions-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
RemoveDuplicateLinesFromTxt
Remove Duplicate lines From Files in TXT
This program remove duplicate lines in txt files. -
2
Video-subtitle-extractor
A GUI tool for extracting hard-coded subtitle (hardsub) from videos
Video hard subtitle extraction, generate srt file. There is no need to apply for a third-party API, and text recognition can be implemented locally. A deep learning-based video subtitle extraction framework, including subtitle region detection and subtitle content extraction. A GUI tool for extracting hard-coded subtitles (hardsub) from videos and generating srt files. Use local OCR recognition, no need to set up and call any API, and do not need to access online OCR services such as Baidu... -
3
Text Line Duplicate Remover
Remove duplicate lines from your text
This standalone offline web browser tool helps you remove duplicate lines from your text, with additional text processing options. Simply open it in your browser by double-clicking the html file. It also includes the source code too. I made this when I was working with long lists of entries and needed something to automatically clean them up. As a bonus you can also change the Sentence Case of the text, make it lowercase, UPPERCASE or Sentence case. -
4
FDUPES
FDUPES is a program for identifying or deleting duplicate files
FDUPES is a lightweight command-line utility that helps users find and optionally delete duplicate files within specified directories by comparing file contents, which can be extremely useful for cleaning up storage clutter or organizing large collections of files. It works by scanning directories and subdirectories, identifying sets of files with identical content through size and hash comparisons, and then listing them together so users can examine duplicates. -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
RemoveDuplicate
A software that can helps you remove duplicate files
RemoveDuplicate is a Windows application designed to help users find and remove duplicate files within folders. It uses MD5 hashing to identify identical files and provides a user-friendly interface to manage and delete duplicates while keeping one copy of each file. -
6
Grabber
Very customizable imageboard/booru downloader with powerful filenaming
Very customizable imageboard/booru browser and mass downloader, with powerful file naming features. Thanks to its powerful naming features, you just have to set your filename and save the directory using all the tokens available, and the program will generate a filename using the image's information. With this, you can store and manage your pictures in advanced directory structures, and save images with custom filenames. Imageboard/booru downloader which can download thousands of images from... -
7
Discordeno
Discord API library for Node and Deno
...The functional API eliminates the challenges of extending built-in classes and inheritance while ensuring overall simple but performant code. Discordeno offers the option to have practically any component of a bot as a separate piece, including standalone REST, gateways, custom caches, and more. Remove any properties, if your bot doesn't need them. For instance, remove Channel.topic if your bot doesn't require it. You may save GBs of RAM in this way. A few lines of code are all that are needed to accomplish this for any property on any object. -
8
DupRem
Simple application to remove duplicate and empty lines on text files.
DupRem is a simple easy-to-use cross-platform application to remove duplicate and empty lines from any text file. It is also possible to keep or ignore case sensitive. Works also from command-line interface, e.g. "java -jar duprem.jar -r input_file.txt >output_file.txt", to create output file, or "java -jar duprem.jar -r input_file.txt >>output_file.txt", to create or append to output file. DupRem is portable, does not need installation and is developed in Java, so needs the Java Virtual Machine (version 6 or higher) that, in the case, is very easy to download and install. -
9
Swiss File Knife
One hundred command line tools in a small and portable binary.
Create zip files, extract zip files, replace text in files, search in files using expressions, stream text editor, instant command line ftp and http server, send folder via network, copy folder excluding sub folders and files, find duplicate files, run a command on all files of a folder, split and join large files, make md5 checksum lists of files, remove tab characters, convert CR/LF, list newest or biggest files of a folder, compare folders, treesize, show first or last lines of a file, find filenames fast using index files, rename many files using expressions, copy part of a file, change times of a file, set file time from filename, print colored text to terminal, convert csv to tab separated, download files from web, send http or udp requests, print tcp or udp traffic, create hexdump of files, join many text files into one, list nested .zip .tar .tar.gz .tar.bz2 archive contents. ... -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
Meta Package Manager
Wraps all package managers with a unifying CLI
Meta Package Manager wraps all package managers with a unifying CLI, and provides the MPM CLI, a wrapper around all package managers. MPM is like yt-dlp, but for package managers instead of videos. MPM solves XKCD #1654 - Universal Install Script. List installed packages. List duplicate installed packages. Search for packages. Install a package, remove a package, and list outdated packages. Sync local package infos. Upgrade all outdated packages. Backup list of installed packages to TOML file. Restore/install list of packages from TOML files. Pin-point commands to a subset of package managers (include/exclude selectors). ... -
11
autopep8
A tool that automatically formats Python code to conform to the PEP 8
...Correct deprecated or non-idiomatic Python code (via lib2to3). Use this for making Python 2.7 code more compatible with Python 3. Put a blank line between a class docstring and its first method declaration. Remove blank lines between a function declaration and its docstring. -
12
duf
Disk usage and free utility, a better 'df' alternative
...You can simply start duf without any command-line arguments. If you supply arguments, duf will only list specific devices & mount points. If you want to list everything (including pseudo, duplicate, inaccessible file systems), you can. You can show and hide specific tables. You can also show and hide specific filesystems. List inode information instead of block usage. If duf doesn't detect your terminal's colors correctly, you can set a theme. Users of oh-my-zsh should be aware that it already defines an alias called duf, which you will have to remove in order to use duf. -
13
Cirrus
The SCSS framework for the modern web
...Forget bundling tons of JavaScript with each component. All styles require no JS for interactions/functionality (see Modals, Dropdowns, etc.). Add additional components, remove utility classes, disable features, etc. Cirrus takes a generative approach to building your stylesheets. A large number of features with a minimal footprint. Fully responsive by design. Getting started with Cirrus is a few commands or lines of HTML away (depending on how you want to import it). The framework is designed to be pluggable into any new or existing projects. -
14
ngx-quill
Angular components for the Quill Rich Text Editor
...In general, QuillJS recommends using a QuillJS instance to present your content. Just create a quill editor without a toolbar and in read-only mode. With some simple CSS lines you can remove the default border around the content. -
15
Amazon CodeGuru Profiler Python Agent
Amazon CodeGuru Profiler Python Agent
Amazon CodeGuru Profiler collects runtime performance data from your live applications and provides recommendations that can help you fine-tune your application performance. Using machine learning algorithms, CodeGuru Profiler can help you find your most expensive lines of code and suggest ways you can improve efficiency and remove CPU bottlenecks. CodeGuru Profiler provides different visualizations of profiling data to help you identify what code is running on the CPU, see how much time is consumed, and suggest ways to reduce CPU utilization. Use CodeGuru Profiler to help profile your applications in the cloud from a single, centralized dashboard. ... -
16
ParDRe
Parallel tool to remove duplicate DNA reads
ParDRe is a parallel tool to remove duplicate reads. Duplicate reads can be seen as identical or nearly identical sequences with some mismatches. This tool will let the users to avoid the analysis of not necessary reads, reducing the time of subsequent procedures with the dataset (e.g., assemblies, mappings, etc.). The tool is implemented with MPI in order to exploit the parallel capabilities of multicore clusters. -
17
NeatDisk
Disk cleaner and duplicate finder for Windows. Keep your drives clean.
...It gives you a clear picture of where your storage is going and the tools to take it back — without the bloat of traditional PC cleaners. Features: - Junk file cleaner — remove temporary files, browser caches, and Windows leftovers - Duplicate finder — find and delete exact duplicate files wasting space across your drives - Large file finder — quickly surface the biggest space hogs on any drive or folder - Disk usage analyzer — visual breakdown of what's eating your storage - Driver store cleaner — safely prune outdated driver packages from the Windows Driver Store - Stale file detector — identify files you haven't touched in months or years - Startup manager — control which programs launch at boot to keep Windows snappy - App uninstaller — cleanly remove installed programs - Restore point manager — review and trim old system restore points - Disk health monitor — check drive S.M.A.R.T. sta -
18
SortMyShit
A tool designed to help you organize and manage your files
SortMyShit is an open-source Python project designed to help you organize and manage your files effortlessly. It provides customizable sorting rules to keep your directories clean and structured. -
19
Blaiz Tools
Tools for working with Gossamer codebase and app source code
Tools for packing files, converting images, manipulating text, checking code and working with the Gossamer codebase and app source code. -
20
Subtitle Editor
Subtitle Editor for SRT, SUB, SBV, TXT, ASS, SSA subs
Srt Editor is a small subtitle editing tool for Linux, OSX and Win64. You can edit the following file formats in various ways : SRT, SUB, TXT, SBV, ASS, SSA. Last version : 4.0.3 (MacOS & Linux) / 3.2.1 (Windows) -
21
logwitch
a simple log file scanner for linux written in shell and lua
logwitch checks single or rotated log files, either plain text or gzipped. It can work with log4j and gnu/linux system logs. You configure it to watch for lines in log files that interest you in by creating a simple text file in its config directory. It emails a daily report to you and timestamps logs so that you do not receive duplicate information. logwitch is GPL3 licensed -
22
UniFormat
This project process code format to a unified style.
This project is used to process code format to a unified style. First we can process C/C++ code, we will process other languages later. -
23
ImageMerge
A GUI tool to merge images/videos from multiple folders.
A GUI tool to merge images/videos from multiple folders, remove duplicates with SHA-256, and rename files into a clean sequential format. -
24
PyPingMon
A simple ping server monitor
PyPingMon is a server monitoring program written in Python. At the specified interval it uses a single ping command to check if a server is available. This is my first Python program. I chose this particular type of program because I wanted to write my own server monitor and this was a great way to learn how to code in Python. -
25
Image Comparator
Find and remove duplicate or similar images.
Image Comparator is a robust Windows desktop application designed to identify and manage duplicate or visually similar images using advanced Discrete Cosine Transform (DCT) algorithms. Unlike standard comparison tools, it effectively detects images that have been resized, compressed, or slightly modified.