Search Results for "compression"
Sort By:
Showing 55 open source projects for "compression"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
rust-rocksdb
Rust wrapper for rocksdb
Binding is statically linked with a specific version of RocksDB. If you want to build it yourself, make sure you've also cloned the RocksDB and compression submodules. By default, support for the Snappy, LZ4, Zstd, Zlib, and Bzip2 compression is enabled through crate features. If support for all of these compression algorithms is not needed, default features can be disabled and specific compression algorithms can be enabled. The underlying RocksDB does allow column families to be created and dropped from multiple threads concurrently. ... -
2
Reliable PostgreSQL Backup & Restore
Reliable PostgreSQL backup and restore
pgBackRest aims to be a reliable, easy-to-use backup and restore solution that can seamlessly scale up to the largest databases and workloads by utilizing algorithms that are optimized for database-specific requirements. Compression is usually the bottleneck during backup operations but, even with now-ubiquitous multi-core servers, most database backup solutions are still single-process. pgBackRest solves the compression bottleneck with parallel processing. Utilizing multiple cores for compression makes it possible to achieve 1TB/hr raw throughput even on a 1Gb/s link. ... -
3
WAL-G
Archival and Restoration for databases in the Cloud
...To configure the compression method used for backups. Possible options are lz4, lzma, zstd, brotli. The default method is lz4. LZ4 is the fastest method, but the compression ratio is bad. LZMA is way much slower. However, it compresses backups about 6 times better than LZ4. Brotli and zstd are a good trade-off between speed and compression ratio, which is about 3 times better than LZ4. -
4
pg_probackup
Backup and recovery manager for PostgreSQL
pg_probackup is an advanced backup and recovery tool for PostgreSQL developed by PostgresPro. It supports full, incremental, and differential backups with compression and encryption, and provides fine-grained control over backup and restore processes. It’s designed for enterprise-grade backup strategies in PostgreSQL environments. -
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
5
ClickHouse Go
Golang driver for ClickHouse
...Bulk write support (for database/sql use begin->prepare->(in loop exec)->commit). AsyncInsert, named and numeric placeholders support. LZ4/ZSTD compression support. External data. -
6
OrioleDB
Building a modern cloud-native storage engine
OrioleDB is a next-generation storage engine for PostgreSQL, aiming to enhance scalability, performance, and modern data management. Designed as a table access method (TAM) extension, it introduces features like MVCC improvements, compression, and efficient concurrency handling. OrioleDB seeks to future-proof PostgreSQL by enabling cloud-native, high-throughput workloads without altering the core PostgreSQL behavior. -
7
ClickHouse
A fast open-source OLAP database management system
ClickHouse® is a fast, open-source column-oriented database management system that can generate analytical data reports through SQL queries in real time. According to several independent benchmarks, it far exceeds other comparable column-oriented database management systems, working even up to 1000 times faster. It is able to process hundreds of millions to more than a billion rows and tens of gigabytes of data per single server per second. Apart from its blazing speed, ClickHouse is highly... -
8
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
9
TimescaleDB
An open-source time-series SQL database optimized for fast ingest
...Horizontally scale to petabytes. Don’t worry about cardinality. Simplify your stack, ask more complex questions, and build more powerful applications. Don’t break the bank. 94-97% compression rates from best-in-class algorithms. Downsample + retention policies. Decouple compute and storage. You’re in trusted hands. Fully-managed TimescaleDB on AWS, Azure, or GCP in just one click. Top-rated support and ops available 24/7 worldwide. A modern, cloud-native relational database platform for time-series data based on TimescaleDB and PostgreSQL. -
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
10
DB Dumper
Dump the contents of a database
DB Dumper is a simple PHP library for creating database dumps of MySQL, PostgreSQL, and SQLite databases. It provides a fluent, expressive API that lets developers configure backup paths, credentials, and additional options. DB Dumper is ideal for integrating automated backups into Laravel or other PHP-based applications. -
11
IOWOW
A C utility library and persistent key/value storage engine
IOWOW is a lightweight, high-performance key-value store and data storage library written in C, designed for use in embedded systems and high-speed applications that require efficient file-based storage. -
12
GreptimeDB
An open-source, cloud-native, unified time series database for metrics
GreptimeDB treats all time series as contextual events with timestamps, and thus unifies the processing of metrics, logs, and events. It supports analyzing metrics, logs, and events with SQL, PromQL, and streaming with continuous aggregation. GreptimeDB is a time-series database optimized for storing and querying large amounts of time-series data, commonly used in monitoring and IoT applications. -
13
MySQLDump - PHP
PHP version of mysqldump cli that comes with MySQL
This is a php version of mysqldump cli that comes with MySQL, without dependencies, output compression and sane defaults. Out of the box, MySQLDump-PHP supports backing up table structures, the data itself, views, triggers and events. MySQLDump-PHP is the only library that supports output binary blobs as hex. Resolves view dependencies (using Stand-In tables). Output compared against original mysqldump. Linked to travis-ci testing system (testing from php 5.3 to 7.3 & hhvm). -
14
TDengine
Open-source time-series database with high-performance and scalability
...TDengine can be widely applied to IoT, Industrial Internet, Connected Vehicles, DevOps, Energy , Finance and many other use-cases. TDengine’s innovative design and purpose-built storage engine outperforms other time-series databases for data ingestion, querying and data compression while significantly reducing storage and computing costs. TDengine’s native distributed architecture powers out-of-the-box scalability and high availability. Nodes can be added through simple configuration to achieve greater data processing power. In addition, this feature is open source. TDengine uses SQL as the query language, thereby reducing learning and migration costs, while adding SQL extensions to handle time-series data better, and supporting convenient and flexible schemaless data ingestion. -
15
IoTDB
Apache IoTDB
Apache IoTDB (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud. Due to its light-weight architecture, high performance and rich feature set together with its deep integration with Apache Hadoop, Spark and Flink, Apache IoTDB can meet the requirements of massive data storage, high-speed data ingestion and complex data analysis in the IoT industrial fields. In the scene of factories, there... -
16
ArcticDB
ArcticDB is a high performance, serverless DataFrame database
...Giving you the keys to protect your most valuable asset. Supports large concurrent writes to many tables ensuring datasets can be onboarded fast and in the most convenient format. Scale-out architecture and built-in compression ensure that networks and storage are utilized efficiently. Support for DataFrames with thousands of columns enables large trading universes. -
17
Carmine
Redis client and message queue for Clojure
...Flexible, high-performance binary-safe serialization using Nippy. Simple, high-performance message queue (v2+, Redis 2.6+). Simple, high-performance distributed lock (v2+, Redis 2.6+). Pluggable compression and encryption support (v2+). Includes Tundra, an API for replicating data to an additional datastore (v2+, Redis 2.6+). You'll usually want to define a single connection pool, and one connection spec for each of your Redis servers. Note that executing multiple Redis commands in a single wcar request uses efficient Redis pipelining under the hood, and returns a pipeline reply (vector) for easy destructuring, etc. -
18
LinDB
LinDB is a scalable, high performance, high availability database
LinDB is a scalable, high-performance, high-availability distributed time series database. A single server could easily support more than one million write TPS; With fundamental techniques like efficient compression storage and parallel computing, LinDB delivers highly optimized query performance. The multi-channel replication protocol supports any amount of nodes, and ensures the system's availability. Schema-free multi-dimensional data model with Metric, Tags, and Fields; The LinQL is flexible yet handy for real-time data analytics. Horizontal scalable is made simple by adding more new broker and storage nodes without too much thinking and manual operations. ... -
19
Barman for PostgreSQL
Backup and Recovery Manager for PostgreSQL
...It allows to perform remote backups of multiple servers in business critical environments and help DBAs during the recovery phase. Barman's most wanted features include backup catalogs, retention policies, remote recovery, archiving and compression of WAL files and backups. Barman is written and maintained by PostgreSQL professionals 2ndQuadrant. -
20
SymmetricDS
Database Replication, Filtered Synchronization, and Transformation
SymmetricDS is database replication and file synchronization software. It is a platform-independent, web-enabled, and database-agnostic synchronization tool. With initial data load, bidirectional change data capture, conflict resolution, and data transformation, it is a comprehensive data replication solution. -
21
GameLauncher
Frontend emulator launcher displaying box art image of associated ROMs
A free front-end emulator displaying associated game box art image for each game. GameLauncher can be controlled using standard Xbox controller. ROM associated image files can be downloaded using following link: http://axter.com/GameLauncher/Images.zip -
22
LocustDB
Massively parallel, high performance analytics database
An experimental analytics database aiming to set a new standard for query performance and storage efficiency on commodity hardware. See How to Analyze Billions of Records per Second on a Single Desktop PC and How to Read 100s of Millions of Records per Second from a Single Disk for an overview of current capabilities. Download the latest binary release, which can be run from the command line on most x64 Linux systems, including Windows Subsystem for Linux. When loading .csv or .csv.gz files... -
23
Hydra Columnar
Postgres-native columnar storage extension
Hydra Columnar is an open-source columnar storage extension for PostgreSQL designed to deliver analytics performance on par with modern data warehouses. It integrates seamlessly with the PostgreSQL ecosystem, allowing users to benefit from columnar compression, vectorized execution, and late materialization without leaving their existing database setup. Hydra enables hybrid row-column storage, making it ideal for OLAP workloads on Postgres. -
24
Arctic TimeSeries and Tick store
High performance datastore for time series and tick data
...Serializes a number of data types eg. Pandas DataFrames, Numpy arrays, Python objects via pickling etc. so you don't have to handle different datatypes manually. Uses LZ4 compression by default on the client side to get big savings on network / disk. Allows you to version different stages of an object and snapshot the state (In some ways similar to git), and allows you to freely experiment and then just revert back the snapshot. [VersionStore only] Does the chunking (breaking a Dataframe to smaller part for you. ... -
25
CursusDB
CursusDB is an open-source distributed in-memory database
CursusDB is a time-series database built for high-performance analytics and data processing, optimized for handling large volumes of sequential data efficiently.
