Search Results for "parallel compression"

pgBackRest aims to be a reliable, easy-to-use backup and restore solution that can seamlessly scale up to the largest databases and workloads by utilizing algorithms that are optimized for database-specific requirements. Compression is usually the bottleneck during backup operations but, even with now-ubiquitous multi-core servers, most database backup solutions are still single-process. pgBackRest solves the compression bottleneck with parallel processing. Utilizing multiple cores for compression makes it possible to achieve 1TB/hr raw throughput even on a 1Gb/s link. ...

Based on source from http://zlib.net/pigz/

fpsc is a Fast Parallel Stream Compressor. It can use multiple CPUs to compress (gzip or bzip2) a data stream. (from stdin to stdout) It is built to achieve maximum throughput, everything else (especially memory usage) can be sacrified.

czip is a unix based utility written for the cell broadband engine. It allows its users to compress files in gzip format. It uses Jean-loup Gailly's deflate algorithm and is based in part on pigz, Mark Adlers parallel version of zlib.