Search Results for "deduplication"

InterPlanetary Wayback (ipwb) facilitates permanence and collaboration in web archives by disseminating the contents of WARC files into the IPFS network. IPFS is a peer-to-peer content-addressable file system that inherently allows deduplication and facilitates opt-in replication. ipwb splits the header and payload of WARC response records before disseminating into IPFS to leverage the deduplication, builds a CDXJ index with references to the IPFS hashes returned, and combines the header and payload from IPFS at the time of replay. An important aspect of archival replay systems is rewriting various resource references for proper memento reconstruction so that they are dereferenced properly from the archive from around the same datetime as of the root memento and not from the live site (in which case the resource might have changed or gone missing). ...

ByteSync is a powerful open-source tool for remote file comparison and synchronization, designed to transfer only what truly matters: the differences. Whether you're syncing large files, backing up remote folders, or handling extensive sets of small files, ByteSync delivers speed, security, and efficiency—without unnecessary data transfers. Compatible with Windows, macOS, and Linux, ByteSync leverages cloud-based orchestration with end-to-end encryption, eliminating the need for VPNs or...

PRIMAL can be used to route, modify and store DICOM Radiology image files. Typically this tool is put between a image acquisition station (modality) and a PACS to route images and modify the DICOM tags in ways that are not easy to do on the acquisition station but needs to be done prior to the study arriving at the PACS. PRIMAL is designed to turn a Fedora or Redhat Linux server into a DICOM routing appliance. Most information can be viewed through the web interface. Commercial...

A lightweight Redis-backed message queue for handling simple distributed messaging and job processing.