Benthos Files

For installation instructions check out the getting started guide.

Added

• mongodb_cdc: Added incremental snapshot mode with configurable signal collection for on-demand snapshot re-execution alongside streaming changes. (@squiidz, #4996)
• sftp: Added cache_mode configuration to enable re-consumption of files when their modification time changes in addition to initial consumption. (@rockdatasrl001, #5064)

Fixed

• aws_dynamodb_cdc: Fixed shutdown hang by settling undelivered CDC batches on close instead of waiting indefinitely for their acknowledgments. (@Leward, #4989)
• aws_dynamodb_cdc: Fixed shard completion tracking to prevent re-reading exhausted shards after cleanup and improved child shard ordering based on parent completion. (@squiidz, #5001)
• aws_dynamodb_cdc: Fixed shard completion marker handling to prevent late plain checkpoint writes from overwriting completion state. (@squiidz, #5014)
• aws_kinesis: Fixed shard handover by allowing takeover of shards with no committed sequence number in the claim. (@squiidz, #5028)
• kafka: Fixed concurrent map access in franz writer and restored dispatch signal for produced records to fix ordered partition input blocking. (@Jeffail, #5033)
• mongodb_cdc: Fixed snapshot signal behavior to work correctly in all snapshot modes and handle collections unknown to stored progress. (@squiidz, #5066)
• oracledb_cdc: Fixed timestamp precision handling in Oracle CDC to correctly preserve timestamp values. (@josephwoodward, #5022)
• postgres_cdc: Fixed publication recreation to handle switching between FOR TABLE and FOR ALL TABLES modes correctly. (@squiidz, #5029)
• postgres_cdc: Fixed PostgreSQL CDC to skip emitting empty records for Type, Truncate, and Origin WAL messages. (@nvanthao, #5084)
• redpanda_migrator: With schema_registry.translate_ids enabled, the output no longer rewrites the schema ID in the input message. A batch retried after a failed write was translated a second time, so records either failed forever with schema ID N not found in registry or were silently written with the wrong destination schema ID. (@josephwoodward, #5074)
• sap_hana: Fixed fractional timestamp precision, session loss handling, and credential refresh behavior in SAP HANA CDC with real hardware testing. (@squiidz, #5063)

Changed

• mssqlserver: Adopted the new ticket.Lock package for flush ticket management, replacing duplicate code with shared infrastructure for better maintainability. (@Leward, #4857)
• replication: Refactored flush ticket logic into a reusable ticket.Lock package with improved concurrency handling, cancellation support, and typed Ticket returns for safer API usage. (@Leward, #4856)

The full change log can be found here.