Search Results for "high compression ratio"
Sort By:
Showing 160 open source projects for "high compression ratio"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
1
Zstandard
Zstandard - Fast real-time compression algorithm
Zstandard is a fast compression algorithm, providing high compression ratios. It also offers a special mode for small data, called dictionary compression. The reference library offers a very wide range of speed / compression trade-off, and is backed by an extremely fast decoder (see benchmarks below). Zstandard library is provided as open source software using a BSD license. -
2
XZ Utils
Open-source compression utility and library
xz is a widely used open-source compression utility and library that implements the high-ratio LZMA and LZMA2 compression algorithms. It provides both command-line tools and a reusable C library, enabling developers and system administrators to compress and decompress files efficiently across many environments. The project is known for delivering strong compression performance while maintaining reasonable memory usage, making it suitable for software distribution, backups, and archival workflows. xz is commonly included in Unix-like operating systems and is often used as a standard packaging format in Linux ecosystems. ... -
3
MCP Image Compression
A high-performance image compression microservice based on MCP
The MCP Image Compression server is a high-performance microservice based on the Model Context Protocol architecture. It focuses on providing fast and high-quality image compression capabilities to help developers optimize image resources for websites and applications, improving loading speed and user experience. -
4
Wan2.2
Wan2.2: Open and Advanced Large-Scale Video Generative Model
...It introduces a Mixture-of-Experts (MoE) architecture that splits the denoising process across specialized expert models, increasing total model capacity without raising computational costs. Wan2.2 integrates meticulously curated cinematic aesthetic data, enabling precise control over lighting, composition, color tone, and more, for high-quality, customizable video styles. The model is trained on significantly larger datasets than its predecessor, greatly enhancing motion complexity, semantic understanding, and aesthetic diversity. Wan2.2 also open-sources a 5-billion parameter high-compression VAE-based hybrid text-image-to-video (TI2V) model that supports 720P video generation at 24fps on consumer-grade GPUs like the RTX 4090. ... -
Earn up to 16% annual interest with Nexo.Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
Brotli
Brotli compression format
...This happens when "one-shot" decoding API is used (or input chunk for streaming API is not limited), input size (chunk size) is larger than 2GiB, and input contains uncompressed blocks. After the overflow happens, memcpy is invoked with a gigantic num value, that will likely cause the crash. Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression. The specification of the Brotli Compressed Data Format is defined in RFC 7932. ... -
6
WAL-G
Archival and Restoration for databases in the Cloud
...To configure the compression method used for backups. Possible options are lz4, lzma, zstd, brotli. The default method is lz4. LZ4 is the fastest method, but the compression ratio is bad. LZMA is way much slower. However, it compresses backups about 6 times better than LZ4. Brotli and zstd are a good trade-off between speed and compression ratio, which is about 3 times better than LZ4. -
7
DwarFS
A fast high compression read-only file system for Linux, Windows
The Deduplicating Warp-speed Advanced Read-only File System. A fast high compression read-only file system for Linux and Windows. DwarFS is a read-only file system with a focus on achieving very high compression ratios in particular for very redundant data. This probably doesn't sound very exciting, because if it's redundant, it should compress well. However, I found that other read-only, compressed file systems don't do a very good job at making use of this redundancy. ... -
8
Snappy
A fast compressor/decompressor
Snappy is a compression/decompression library. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. -
9
compress
Optimized Go Compression Packages
This package provides various compression algorithms. zstandard compression and decompression in pure Go. S2 is a high-performance replacement for Snappy. Optimized deflate packages which can be used as a dropin replacement for gzip, zip and zlib. snappy is a drop-in replacement for github.com/golang/snappy offering better compression and concurrent streams. huff0 and FSE implementations for raw entropy encoding. gzhttp Provides client and server wrappers for handling gzipped requests efficiently. pgzip is a separate package that provides a very fast parallel gzip implementation. ... -
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
FramePack
Lets make video diffusion practical
FramePack explores compact representations for sequences of image frames, targeting tasks where many near-duplicate frames carry redundant information. The idea is to “pack” frames by detecting shared structure and storing differences efficiently, which can accelerate training or inference on video-like data. By reducing I/O and memory bandwidth, datasets become lighter to load while models still see the essential temporal variation. The repository demonstrates both packing and unpacking... -
11
OrioleDB
Building a modern cloud-native storage engine
OrioleDB is a next-generation storage engine for PostgreSQL, aiming to enhance scalability, performance, and modern data management. Designed as a table access method (TAM) extension, it introduces features like MVCC improvements, compression, and efficient concurrency handling. OrioleDB seeks to future-proof PostgreSQL by enabling cloud-native, high-throughput workloads without altering the core PostgreSQL behavior. -
12
nghttp2
HTTP/2 C Library and tools
...Since then we have updated nghttp2 library constantly to the latest specification and nghttp2 is now one of the most mature HTTP/2 implementations. HTTP/2 utilizes header compression method called HPACK. We offer HPACK encoder and decoder are available as public API. nghttp2 library itself is a bit low-level. The experimental high-level C++ API is also available. We have Python binding of this library, but we have not covered everything yet. -
13
DeepSpeed
Deep learning optimization library: makes distributed training easy
...Achieve excellent system throughput and efficiently scale to thousands of GPUs 3. Train/Inference on resource constrained GPU systems 4. Achieve unprecedented low latency and high throughput for inference 5. Achieve extreme compression for an unparalleled inference latency and model size reduction with low costs DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. ... -
14
IoTDB
Apache IoTDB
Apache IoTDB (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud. Due to its light-weight architecture, high performance and rich feature set together with its deep integration with Apache Hadoop, Spark and Flink, Apache IoTDB can meet the requirements of massive data storage, high-speed data ingestion and complex data analysis in the IoT industrial fields. In the scene of factories, there... -
15
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
16
PaddleOCR
Awesome multilingual OCR toolkits based on PaddlePaddle
...Inspired by PaddlePaddle, PaddleOCR is an ultra lightweight OCR system, with multilingual recognition, digit recognition, vertical text recognition, as well as long text recognition. It features a PPOCR series of high-quality pre-trained models, which includes: ultra lightweight ppocr_mobile series models, general ppocr_server series models, and ultra lightweight compression ppocr_mobile_slim series models. PaddleOCR is easy to install and easy to use on Windows, Linux, MacOS and other systems. -
17
WavTokenizer
SOTA discrete acoustic codec models with 40/75 tokens per second
WavTokenizer is a state-of-the-art discrete acoustic codec designed specifically for audio language modeling, capable of compressing 24 kHz audio into just 40 or 75 tokens per second while preserving high perceptual quality. It is built to represent speech, music, and general audio with extremely low bitrate, making it ideal as a front-end for large audio language models like GPT-4o and similar architectures. The model uses a single-quantizer design together with temporal compression to achieve extreme compression without sacrificing reconstruction fidelity. ... -
18
FlexLLMGen
Running large language models on a single GPU
FlexLLMGen is an open-source inference engine designed to run large language models efficiently on limited hardware resources such as a single GPU. The system focuses on high-throughput generation workloads where large batches of text must be processed quickly, such as large-scale data extraction or document analysis tasks. Instead of requiring expensive multi-GPU systems, the framework uses techniques such as memory offloading, compression, and optimized batching to run large models on commodity hardware. ... -
19
Google Highway
Performance-portable, length-agnostic SIMD with runtime dispatch
Google Highway is a high-performance C++ library designed to provide portable SIMD (Single Instruction, Multiple Data) vectorization across multiple CPU architectures while maintaining predictable and efficient behavior. It abstracts low-level vector intrinsics into a consistent API that maps closely to hardware instructions, allowing developers to write high-performance code without relying heavily on compiler auto-vectorization. -
20
Vortex
The LLVM of columnar file formats
Vortex is a high-performance toolkit designed for working with compressed Apache Arrow arrays, providing functionality for in-memory, on-disk, and over-the-wire data handling. It aims to be an advanced successor to Apache Parquet, offering dramatically faster random access reads and scans, while maintaining similar compression ratios. Vortex's modular design allows for extensibility, enabling developers to implement custom encodings for efficient data management, particularly for large-scale columnar datasets. -
21
Audiogen Codec
48khz stereo neural audio codec for general audio
AGC (Audiogen Codec) is a convolutional autoencoder based on the DAC architecture, which holds SOTA. We found that training with EMA and adding a perceptual loss term with CLAP features improved performance. These codecs, being low compression, outperform Meta's EnCodec and DAC on general audio as validated from internal blind ELO games. We trained (relatively) very low compression codecs in the pursuit of solving a core issue regarding general music and audio generation, low acoustic quality, and audible artifacts, which hinder industry use for these models. Our hope is to encourage researchers to build hierarchical generative audio models that can efficiently use high sequence length representations without sacrificing semantic abilities. -
22
libavif
Library for encoding and decoding .avif files
...The API is straightforward, allowing applications to integrate AVIF support natively — creating, decoding, compressing, or decompressing images programmatically. Recent developments have added experimental support for HDR gain maps, improved color-space handling (including wide color gamut and high bit depth), and better quality tuning modes for encoding, making libavif a cutting-edge tool for high-quality image compression. -
23
LinDB
LinDB is a scalable, high performance, high availability database
LinDB is a scalable, high-performance, high-availability distributed time series database. A single server could easily support more than one million write TPS; With fundamental techniques like efficient compression storage and parallel computing, LinDB delivers highly optimized query performance. The multi-channel replication protocol supports any amount of nodes, and ensures the system's availability. -
24
catpaq
zpaq GUI
ZPAQ is one of the best compression formats around — great compression ratio, incremental backups, full version history. The catch: it has no official GUI. catpaq fixes that. Drop it next to zpaqfranz, open an archive, and you're done. -
25
IOWOW
A C utility library and persistent key/value storage engine
IOWOW is a lightweight, high-performance key-value store and data storage library written in C, designed for use in embedded systems and high-speed applications that require efficient file-based storage.
