pgstream is an open source CDC command-line tool and library that offers Postgres replication support with DDL changes to any provided output. One of the main differentiators of pgstream is the fact that it tracks and replicates schema changes automatically. It relies on SQL triggers that will populate a Postgres table (pgstream.schema_log) containing a history log of all DDL changes for a given schema. Whenever a schema change occurs, this trigger creates a new row in the schema log table with the schema encoded as a JSON value. This table tracks all the schema changes, forming a linearised change log that is then parsed and used within the pgstream pipeline to identify modifications and push the relevant changes downstream.

Features

  • Schema change tracking and replication of DDL changes
  • Modular deployment configuration, only requires Postgres
  • Schema based message partitioning
  • Schema filtering
  • Elasticsearch/OpenSearch replication output plugin support
  • Automatic discovery of table primary key/unique not null columns for use as event identity
  • Highly customisable modules when used as library
  • Core metrics available via opentelemetry
  • Extendable support for custom replication output plugins
  • Continuous consumption of replication slot with configurable memory guards

Project Samples

Project Activity

See All Activity >

Categories

Database

License

Apache License V2.0

Follow pgstream

pgstream Web Site

Other Useful Business Software
Compliant and Reliable File Transfers Backed by Top Security Certifications Icon
Compliant and Reliable File Transfers Backed by Top Security Certifications

Cerberus FTP Server delivers SOC 2 Type II certified security and FIPS 140-2 validated encryption.

Stop relying on non-certified, legacy file transfer tools that creak under the weight of modern security demands. Get full audit trails, advanced access controls and more supported by an award-winning team of experts. Start your free 25-day trial today.
Start Free Trial
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of pgstream!

Additional Project Details

Operating Systems

Linux, Mac, Windows

Programming Language

Go

Related Categories

Go Database Software

Registered

2024-09-03