Search Results for "deduplication"
Sort By:
Showing 41 open source projects for "deduplication"
View related business solutions-
ConnectWise SIEM (formerly Perch) offers threat detection and response backed by an in-house Security Operations Center (SOC). Defend against business email compromise, account takeovers, and see beyond your network traffic. Our team of threat analysts does all the tedium for you, eliminating the noise and sending only identified and verified treats to action on. Built with multi-tenancy, ConnectWise SIEM helps you keep clients safe with the best threat intel on the market.
-
Out-of-the-box scripts. Around-the-clock monitoring. Unmatched automation capabilities. Start doing more with less and exceed service delivery expectations.
-
1
wagmi
React Hooks for Ethereum
wagmi is a collection of React Hooks containing everything you need to start working with Ethereum. wagmi makes it easy to "Connect Wallet," display ENS and balance information, sign messages, interact with contracts, and much more, all with caching, request deduplication, and persistence. We create a wagmi Client and pass it to the WagmiConfig React Context. The client is set up to use the ethers Default Provider and automatically connect to previously connected wallets. Next, we use... -
2
ipwb
A distributed and persistent archive replay system using IPFS
InterPlanetary Wayback (ipwb) facilitates permanence and collaboration in web archives by disseminating the contents of WARC files into the IPFS network. IPFS is a peer-to-peer content-addressable file system that inherently allows deduplication and facilitates opt-in replication. ipwb splits the header and payload of WARC response records before disseminating into IPFS to leverage the deduplication, builds a CDXJ index with references to the IPFS hashes returned, and combines the header... -
3
ClusterFuzz
Scalable fuzzing infrastructure
ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software. Google uses ClusterFuzz to fuzz all Google products and as the fuzzing backend for OSS-Fuzz. ClusterFuzz provides many features which help seamlessly integrate fuzzing into a software project's development process. Can run on any size cluster (e.g. OSS-Fuzz instance runs on 100,000 VMs). Fully automatic bug filing, triage and closing for various issue trackers (e.g. Monorail, Jira).... -
4
NeMo Curator
Scalable data pre processing and curation toolkit for LLMs
NeMo Curator is a Python library specifically designed for fast and scalable dataset preparation and curation for large language model (LLM) use-cases such as foundation model pretraining, domain-adaptive pretraining (DAPT), supervised fine-tuning (SFT) and paramter-efficient fine-tuning (PEFT). It greatly accelerates data curation by leveraging GPUs with Dask and RAPIDS, resulting in significant time savings. The library provides a customizable and modular interface, simplifying pipeline... -
SysAid is an ITSM, Service Desk and Help Desk software solution that integrates all of the essential IT tools into one product. Its rich set of features include a powerful Help Desk, IT Asset Management, and other easy-to-use tools for analyzing and optimizing IT performance.
-
5
Pachyderm
Data-Centric Pipelines and Data Versioning
Data-driven pipelines automatically trigger based on detecting data changes. Automatic immutable data lineage and data versioning of all data types. Autoscaling and parallel processing built on Kubernetes for resource orchestration. Uses standard object stores for data storage with automatic deduplication. Runs across all major cloud providers and on-premises installations. Automatic and intelligent versioning of even the largest data sets of unstructured and structured data. Git-like... -
6
Nuke
Image loading system
Nuke ILS provides an efficient way to download and display images in your app. It's easy to learn and use thanks to a clear and concise API. Its architecture enables many powerful features while offering virtually unlimited possibilities for customization. Despite the number of features, the framework is lean and compiles in just under 3 seconds¹. Nuke has an automated test suite 2x the size of the codebase itself, ensuring excellent reliability. Every feature is carefully designed and... -
7
SWR
React Hooks library for remote data fetching
The name “SWR” is derived from stale-while-revalidate, a HTTP cache invalidation strategy popularized by HTTP RFC 5861. SWR is a strategy to first return the data from cache (stale), then send the fetch request (revalidate), and finally come with the up-to-date data. With SWR, components will get a stream of data updates constantly and automatically. And the UI will be always fast and reactive. With just one single line of code, you can simplify the logic of data fetching in your project,... -
8
zpaqfranz
Zpaq compatible archiver for Win, Linux, Free/OpenBSD, Solaris, MacOS
Advanced archiver (zpaq fork) with deduplication and versioning. Never prune your backups: keeps a forever-to-ever copy (even thousands of versions) of your files, conceptually similar to Mac time machines, or zfs' snapshots, but much more efficiently. Ideal for virtual machine backup and "freezing" zfs snapshots forever. Easily handles millions of files and TBs of data. Allows rsync backups to the cloud (or NAS, or USB) with minimal data transfer, encryption and very limited bandwidth... -
9
Underscore Backup
Private, secure backups in the cloud.
Private, secure backups in the cloud of any size with minimal resource usage exactly the way you want them. Your data is encrypted by default using encryption keys only available to you and before leaving your network. Backed up data is signed to ensure protection from tampering. Data can be stored with any amount of redundancy and even multiple backup locations simultaneously. No information about the contents of your data is saved in any backup location. Only you can download and... -
Define and Deliver Comprehensive Cybersecurity Services. Security threats continue to grow, and your clients are most likely at risk. Small- to medium-sized businesses (SMBs) are targeted by 64% of all cyberattacks, and 62% of them admit lacking in-house expertise to deal with security issues. Now technology solution providers (TSPs) are a prime target. Enter ConnectWise Cybersecurity Management (formerly ConnectWise Fortify) — the advanced cybersecurity solution you need to deliver the managed detection and response protection your clients require. Whether you’re talking to prospects or clients, we provide you with the right insights and data to support your cybersecurity conversation. From client-facing reports to technical guidance, we reduce the noise by guiding you through what’s really needed to demonstrate the value of enhanced strategy.
-
10
XSIBackup-App
Backup and replicate Linux files, databases and ©VMWare ©ESXi VMs
Free virtual appliance to backup and replicate Linux servers and ©VMWare ©ESXi virtual machines from version 5.1 to 8.0. https://33hops.com/xsibackup-app-detailed-installation-instructions.html ©XSIBackup-App connects to multiple Linux or ©ESXi servers and backs VMs & files up to, local disk, NFS, iSCSI, Samba, etc... or to any Linux or ©ESXi server over IP, it just needs the SSH port open. This appliance is based in CentOS 7 and has root access. You can install any available... -
11
XSIBackup-DC
VMWare ESXi Virtual Machine Backup, runs in free ESXi
©XSIBackup-DC is a C binary that runs in the ©ESXi shell and behaves as a backup and replica service. It follows the same principles as our previous ©XSIBackup editions, that is: a utility that runs in the hypervisor command line and accepts arguments and values stating which VMs to backup and where to copy them. It accepts local and remote SSH paths, thus you can backup and replicate VMs to local datastores or to remote repositories in any additional ©ESXi or Linux server. Download... -
12
XSIBackup-Classic
Local and over IP backups for ©VMWare ©ESXi 5.1 to 6.7
VMWare backup and replication for ESXi vSphere virtual machines to datastore or differential over IP. No agents, self capable, runs in the hypervisor host, ESXi 5.1, ESXi 5.5 & ESXi <=6.7 compatible, hot backups. Consists in a cron programmable service running in the hypervisor. Hot backups, fully functional, standalone, ideal for dedicated servers. Full version supports quiescing, differential backup, deduplication and much more... Download link: https://33hops.com/xsibackup-vmware-esxi... -
13
wadmerge
Merges WAD files for Doom engine games.
Merges .WAD files from Doom engine games, such as Doom, Doom2, Hexen and Heretic. Takes any number of input wads, and produces a sorted single output .WAD file. Runs on Linux and Windows, does not duplicate wad entries and allows space saving with an optional 'deduplication' method, where different entries which have the same data, only have the data stored once. -
14
zsfx
zpaqfranz/zpaq SFX module for Windows 32 and 64 bit
Convert zpaq archive (full features) to self extraction (SFX) executable for Windows 32/64 bit Now embedded in zpaqfranz (for Windows) https://sourceforge.net/projects/zpaqfranz -
15
This is a simple BASH script which will backup source directories to a destination target, such as a mounted NFS share. It will create a full directory structure matching the source, but only use disk space for new/changed files. This is done by linking existing files. Essentially, it backs up like an incremental backup, and consumes space like an incremental backup, but restores like a full backup.
-
16
Open Source Data Quality and Profiling
World's first open source data quality & data preparation project
This project is dedicated to open source data quality and data preparation solutions. Data Quality includes profiling, filtering, governance, similarity check, data enrichment alteration, real time alerting, basket analysis, bubble chart Warehouse validation, single customer view etc. defined by Strategy. This tool is developing high performance integrated data management platform which will seamlessly do Data Integration, Data Profiling, Data Quality, Data Preparation, Dummy Data... -
17
UrBackup
Open Source Client/Server Network Backup System
Efficient Client-Server Backup system for Linux, macOS and Windows. A client for Windows, FreeBSD and Linux lets you backup open files and complete partition images (Windows). Backups are stored to disks in a efficient way (deduplication) on either Windows or Linux servers. -
18
KonDB
Organizer for your Kontakt files
- Easily Organize, Sort Through, and Load your Kontakt NKI/NKM Files in Windows - This project was built to satisfy a need to have a separate app to manage a large Kontakt files collection. - This app does not delete/move/rename/etc. your files, but it will make management of your files a lot easier. Getting Started: Drag-and-drop files/folders into this app from your computer at any time to add to your index. If you drag in the same file(s) twice, it won't be re-added. Drag-and-drop... -
19
PRIMAL_RAD
DICOM tag modification and routing tool
PRIMAL can be used to route, modify and store DICOM Radiology image files. Typically this tool is put between a image acquisition station (modality) and a PACS to route images and modify the DICOM tags in ways that are not easy to do on the acquisition station but needs to be done prior to the study arriving at the PACS. PRIMAL is designed to turn a Fedora or Redhat Linux server into a DICOM routing appliance. Most information can be viewed through the web interface. Commercial... -
20
tbackup2
Backup tool
Backup files using rsync and deduplication with hardlinks. Written in ruby. -
21
Offnet
Program that saves complete web pages retaining multiple timestamps
Offnet is an open source tool for mirroring web pages. It lets you manage several snapshots of web pages in order to retain the newest as well as some elder versions of a web page. It has also file based deduplication features to store content efficiently. The application comes with an integrated web server to navigate directly through your web browser. Project goals: - Web page downloads for less experienced users, including easy setup - Project based page maintanance - Not too... -
22
BinDiff
Helper app for VMware .vmdk files differential backup
This application is intended to make a diff files for the backup purposes. Imagine that you have a VM with 10GB vmdk file that you backup every day. So, you need to keep 100GB (let say 60GB with compression) for 10 days. However from those 10GB file only ~500MB is daily changed and you can shrink your backup storage to 15GB, if you keep only the daily difference. The BinDiff makes this job for you. Initially you make a full copy of vmfk file (let's say every month or two). That time you... -
23
GraphQL multipart request specification
A spec for GraphQL multipart form requests (file uploads).
An interoperable multipart form field structure for GraphQL requests, used by various file upload client/server implementations. An “operations object” is an Apollo GraphQL POST request (or array of requests if batching). An “operations path” is an object-path string to locate a file within an operations object. -
24
DataCleaner
Data quality analysis, profiling, cleansing, duplicate detection +more
DataCleaner is a data quality analysis application and a solution platform for DQ solutions. It's core is a strong data profiling engine, which is extensible and thereby adds data cleansing, transformations, enrichment, deduplication, matching and merging. Website: http://datacleaner.github.io -
25
ZipSnapNG
Portable multi-platform backup tool with advanced features
ZipSnapNG is a portable multi-platform backup tool which, despite its simplicity (200Kb and no installation required), provides advanced features such as incremental backup, data deduplication, encryption and configurable retention. It may run through the command line or within a graphical interface.