PeerDB Files

What's Changed

This release makes a lot of progress on the MongoDB connector and also adds support for ClickHouse's CoalescingMergeTree, along with some cleanup in the PeeDB UI side.

• mongo: add read preference to mongo config by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3200
• Timezone cleanup by @serprex in https://github.com/PeerDB-io/peerdb/pull/3202
• feat: upgrade docker-compose.yml stable image tags by @github-actions[bot] in https://github.com/PeerDB-io/peerdb/pull/3207
• [ui] update settings page layout to flex-col by @pratheekrebala in https://github.com/PeerDB-io/peerdb/pull/3214
• [mongo] try to save resumeToken even when idle by @heavycrystal in https://github.com/PeerDB-io/peerdb/pull/3203
• [mongo] fix rootCa handling by @heavycrystal in https://github.com/PeerDB-io/peerdb/pull/3208
• remove trimBoth, don't try parsing raw json in ch normalize by @serprex in https://github.com/PeerDB-io/peerdb/pull/3210
• mongo: make sure events from irrelevant tables are filtered out by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3196
• mongo: use enum for readPreference instead of string by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3217
• refactor(classifier): add new pattern for "not found in user directories" by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/3218
• mongo: fix bug in getPartitionForStartAndEnd by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3222
• further simplify mysql partitioning by @serprex in https://github.com/PeerDB-io/peerdb/pull/3223
• mongo: log a warning for unsupported events by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3224
• mongo: always return full table partition by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3227
• PEERDB_ORIGIN_METADATA_AS_DESTINATION_COLUMN by @serprex in https://github.com/PeerDB-io/peerdb/pull/3209
• use bitnami docker images by @serprex in https://github.com/PeerDB-io/peerdb/pull/3229
• Update README.md by @morsapaes in https://github.com/PeerDB-io/peerdb/pull/3230
• Fix SSH input component by @Amogh-Bharadwaj in https://github.com/PeerDB-io/peerdb/pull/3235
• [mongo] SSH tunnelling and metered connection by @heavycrystal in https://github.com/PeerDB-io/peerdb/pull/3231
• fix(classifier): add case of invalid snapshot by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/3238
• feat(classifier): rds binlog invalidation by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/3241
• feat(classifier): ch cannot read from socket as recoverable by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/3240
• mongo: test nested document at limit by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3242
• refactor custom order by ui by @serprex in https://github.com/PeerDB-io/peerdb/pull/3245
• CoalescingMergeTree by @serprex in https://github.com/PeerDB-io/peerdb/pull/3251
• ch: allow configuring clickhouse destination tables' PARTITION BY by @serprex in https://github.com/PeerDB-io/peerdb/pull/3239
• fix(classifier): normalization errors as mv by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/3243
• mongo: handle validation edge case by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3257
• remove custom sync by @serprex in https://github.com/PeerDB-io/peerdb/pull/3256
• s3/ch: PEERDB_S3_UUID_PREFIX by @serprex in https://github.com/PeerDB-io/peerdb/pull/3258
• add tables: adding tables to publication should also be part of selector for loop by @serprex in https://github.com/PeerDB-io/peerdb/pull/3254
• update workflow status to FAILED when completing with error by @serprex in https://github.com/PeerDB-io/peerdb/pull/3255
• mongo: handle resume token from table not in changestream by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3259
• mongo: remove manual computation of bytes ingested by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3267
• feat: upgrade docker-compose.yml stable image tags by @github-actions[bot] in https://github.com/PeerDB-io/peerdb/pull/3261
• refactor(metrics): lossy conversion warnings by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/3237
• uuid prefix: job must come after uuid by @serprex in https://github.com/PeerDB-io/peerdb/pull/3271
• refactor(classifier): check for pg conn errors earlier by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/3282
• scheduled tasks as one long running activity by @serprex in https://github.com/PeerDB-io/peerdb/pull/3268
• use uuid.NewString by @serprex in https://github.com/PeerDB-io/peerdb/pull/3284
• Add e2e test bucket to dev config by @ilidemi in https://github.com/PeerDB-io/peerdb/pull/3285
• failing to parse LSNs should not be an error by @heavycrystal in https://github.com/PeerDB-io/peerdb/pull/3286
• ch: redact error 117 INCORRECT_DATA by @serprex in https://github.com/PeerDB-io/peerdb/pull/3289
• pg: add e2e test for json types by @jgao54 in https://github.com/PeerDB-io/peerdb/pull/3292

New Contributors

• @pratheekrebala made their first contribution in https://github.com/PeerDB-io/peerdb/pull/3214
• @morsapaes made their first contribution in https://github.com/PeerDB-io/peerdb/pull/3230

Full Changelog: https://github.com/PeerDB-io/peerdb/compare/v0.30.7...v0.31.0