Search Results for "compression"
Sort By:
Showing 35 open source projects for "compression"
View related business solutions-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
rust-rocksdb
Rust wrapper for rocksdb
Binding is statically linked with a specific version of RocksDB. If you want to build it yourself, make sure you've also cloned the RocksDB and compression submodules. By default, support for the Snappy, LZ4, Zstd, Zlib, and Bzip2 compression is enabled through crate features. If support for all of these compression algorithms is not needed, default features can be disabled and specific compression algorithms can be enabled. The underlying RocksDB does allow column families to be created and dropped from multiple threads concurrently. ... -
2
WAL-G
Archival and Restoration for databases in the Cloud
...To configure the compression method used for backups. Possible options are lz4, lzma, zstd, brotli. The default method is lz4. LZ4 is the fastest method, but the compression ratio is bad. LZMA is way much slower. However, it compresses backups about 6 times better than LZ4. Brotli and zstd are a good trade-off between speed and compression ratio, which is about 3 times better than LZ4. -
3
pg_probackup
Backup and recovery manager for PostgreSQL
pg_probackup is an advanced backup and recovery tool for PostgreSQL developed by PostgresPro. It supports full, incremental, and differential backups with compression and encryption, and provides fine-grained control over backup and restore processes. It’s designed for enterprise-grade backup strategies in PostgreSQL environments. -
4
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
5
ClickHouse Go
Golang driver for ClickHouse
...Bulk write support (for database/sql use begin->prepare->(in loop exec)->commit). AsyncInsert, named and numeric placeholders support. LZ4/ZSTD compression support. External data. -
6
TimescaleDB
An open-source time-series SQL database optimized for fast ingest
...Horizontally scale to petabytes. Don’t worry about cardinality. Simplify your stack, ask more complex questions, and build more powerful applications. Don’t break the bank. 94-97% compression rates from best-in-class algorithms. Downsample + retention policies. Decouple compute and storage. You’re in trusted hands. Fully-managed TimescaleDB on AWS, Azure, or GCP in just one click. Top-rated support and ops available 24/7 worldwide. A modern, cloud-native relational database platform for time-series data based on TimescaleDB and PostgreSQL. -
7
GreptimeDB
An open-source, cloud-native, unified time series database for metrics
GreptimeDB treats all time series as contextual events with timestamps, and thus unifies the processing of metrics, logs, and events. It supports analyzing metrics, logs, and events with SQL, PromQL, and streaming with continuous aggregation. GreptimeDB is a time-series database optimized for storing and querying large amounts of time-series data, commonly used in monitoring and IoT applications. -
8
ArcticDB
ArcticDB is a high performance, serverless DataFrame database
...Giving you the keys to protect your most valuable asset. Supports large concurrent writes to many tables ensuring datasets can be onboarded fast and in the most convenient format. Scale-out architecture and built-in compression ensure that networks and storage are utilized efficiently. Support for DataFrames with thousands of columns enables large trading universes. -
9
DB Dumper
Dump the contents of a database
DB Dumper is a simple PHP library for creating database dumps of MySQL, PostgreSQL, and SQLite databases. It provides a fluent, expressive API that lets developers configure backup paths, credentials, and additional options. DB Dumper is ideal for integrating automated backups into Laravel or other PHP-based applications. -
Atera - an All-in-one platform for IT managementYour IT essentials, integrated & elevated. Take your IT management from automated to autonomous, download Atera's agent to start your free trial!
-
10
LinDB
LinDB is a scalable, high performance, high availability database
LinDB is a scalable, high-performance, high-availability distributed time series database. A single server could easily support more than one million write TPS; With fundamental techniques like efficient compression storage and parallel computing, LinDB delivers highly optimized query performance. The multi-channel replication protocol supports any amount of nodes, and ensures the system's availability. Schema-free multi-dimensional data model with Metric, Tags, and Fields; The LinQL is flexible yet handy for real-time data analytics. Horizontal scalable is made simple by adding more new broker and storage nodes without too much thinking and manual operations. ... -
11
Carmine
Redis client and message queue for Clojure
...Flexible, high-performance binary-safe serialization using Nippy. Simple, high-performance message queue (v2+, Redis 2.6+). Simple, high-performance distributed lock (v2+, Redis 2.6+). Pluggable compression and encryption support (v2+). Includes Tundra, an API for replicating data to an additional datastore (v2+, Redis 2.6+). You'll usually want to define a single connection pool, and one connection spec for each of your Redis servers. Note that executing multiple Redis commands in a single wcar request uses efficient Redis pipelining under the hood, and returns a pipeline reply (vector) for easy destructuring, etc. -
12
TDengine
Open-source time-series database with high-performance and scalability
...TDengine can be widely applied to IoT, Industrial Internet, Connected Vehicles, DevOps, Energy , Finance and many other use-cases. TDengine’s innovative design and purpose-built storage engine outperforms other time-series databases for data ingestion, querying and data compression while significantly reducing storage and computing costs. TDengine’s native distributed architecture powers out-of-the-box scalability and high availability. Nodes can be added through simple configuration to achieve greater data processing power. In addition, this feature is open source. TDengine uses SQL as the query language, thereby reducing learning and migration costs, while adding SQL extensions to handle time-series data better, and supporting convenient and flexible schemaless data ingestion. -
13
IoTDB
Apache IoTDB
Apache IoTDB (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud. Due to its light-weight architecture, high performance and rich feature set together with its deep integration with Apache Hadoop, Spark and Flink, Apache IoTDB can meet the requirements of massive data storage, high-speed data ingestion and complex data analysis in the IoT industrial fields. In the scene of factories, there... -
14
IOWOW
A C utility library and persistent key/value storage engine
IOWOW is a lightweight, high-performance key-value store and data storage library written in C, designed for use in embedded systems and high-speed applications that require efficient file-based storage. -
15
Hydra Columnar
Postgres-native columnar storage extension
Hydra Columnar is an open-source columnar storage extension for PostgreSQL designed to deliver analytics performance on par with modern data warehouses. It integrates seamlessly with the PostgreSQL ecosystem, allowing users to benefit from columnar compression, vectorized execution, and late materialization without leaving their existing database setup. Hydra enables hybrid row-column storage, making it ideal for OLAP workloads on Postgres. -
16
SymmetricDS
Database Replication, Filtered Synchronization, and Transformation
SymmetricDS is database replication and file synchronization software. It is a platform-independent, web-enabled, and database-agnostic synchronization tool. With initial data load, bidirectional change data capture, conflict resolution, and data transformation, it is a comprehensive data replication solution. -
17
Arctic TimeSeries and Tick store
High performance datastore for time series and tick data
...Serializes a number of data types eg. Pandas DataFrames, Numpy arrays, Python objects via pickling etc. so you don't have to handle different datatypes manually. Uses LZ4 compression by default on the client side to get big savings on network / disk. Allows you to version different stages of an object and snapshot the state (In some ways similar to git), and allows you to freely experiment and then just revert back the snapshot. [VersionStore only] Does the chunking (breaking a Dataframe to smaller part for you. ... -
18
CursusDB
CursusDB is an open-source distributed in-memory database
CursusDB is a time-series database built for high-performance analytics and data processing, optimized for handling large volumes of sequential data efficiently. -
19
Cetus
Cetus is a high performance middleware that provides routing
Cetus is a high-performance middleware that provides transparent routing between your application and any backend MySQL Servers. Cetus is the middleware of relational database MySQL developed by C language, which mainly provides a comprehensive database access proxy function. The connection mode of Cetus is basically compatible with MySQL, and the application program can access the database through Cetus without modification, which realizes the horizontal expansion and high availability of... -
20
Sqlite Index Blaster
Create huge Sqlite indexes at breakneck speeds
SQLite Blaster is an advanced SQLite extension that enhances database performance by enabling multi-threading, data compression, and memory optimizations. It is designed for applications that require fast local storage with improved query efficiency. -
21
MySQLDump - PHP
PHP version of mysqldump cli that comes with MySQL
This is a php version of mysqldump cli that comes with MySQL, without dependencies, output compression and sane defaults. Out of the box, MySQLDump-PHP supports backing up table structures, the data itself, views, triggers and events. MySQLDump-PHP is the only library that supports output binary blobs as hex. Resolves view dependencies (using Stand-In tables). Output compared against original mysqldump. Linked to travis-ci testing system (testing from php 5.3 to 7.3 & hhvm). -
22
Replibyte
Seed your development database with real data
...Database Subsetting: Scale down a production database to a more reasonable size. Start a local database with the prod data in a single command. On-the-fly data (de)compression (Zlib). On-the-fly data de/encryption (AES-256). Fully stateless (no server, no daemon) and lightweight binary. Use custom transformers. Auto-detect and version database schema change. Auto-detect sensitive fields. Auto-clean backed up data. At Qovery (the company behind Replibyte), developers can clone their applications and databases just with one click. ... -
23
sled
The champagne of beta embedded databases
...Fully atomic single-key operations, including compare and swap. Forward and reverse iterators over ranges of items. A crash-safe monotonic ID generator capable of generating 75-125 million unique ID's per second. Zstd compression (use the compression build feature, disabled by default). Cpu-scalable lock-free implementation. Flash-optimized log-structured storage. Uses modern b-tree techniques such as prefix encoding and suffix truncation for reducing the storage costs of long keys with shared prefixes. If keys are the same length and sequential then the system can avoid storing 99%+ of the key data in most cases, essentially acting like a learned index. sled performs prefix encoding on long keys with similar prefixes that are grouped together in a range, as well as suffix truncation to further reduce the indexing costs of long keys. -
24
localResizeIMG
The front-end local client compresses pictures, compatible with IOS
...I have stepped on many pitfalls, after several version iterations, as well as the feedback and help of many netizens, and model testing. The picture is distorted, some devices do not automatically rotate the direction of the picture, and there is no jpeg compression algorithm. New Blob is not supported, and the size of the file constructed by formData is 0. There are also inexplicable bugs in some models and browsers (such as QQX5 browser). On-demand loading (JS files will be automatically loaded asynchronously according to the corresponding device, saving unnecessary bandwidth) -
25
CStore FDW
Columnar storage extension for Postgres
cstore_fdw is a columnar store extension for PostgreSQL built by Citus Data using the Foreign Data Wrapper (FDW) interface. It stores data in a compressed, columnar format that significantly reduces disk I/O for analytical queries. Ideal for read-heavy workloads and time-series data, cstore_fdw enhances Postgres’ ability to serve as a lightweight data warehouse.
