2.3.12 Release Note

Docs

• [Doc] Added deepwiki to README.md [#9354] (#9356)
• [Doc] Fix link error and DynamoDB parameter error (#9447)
• [Doc][Improve] update projection supported in Paimon (#9425)
• [Doc][Zeta] Add zeta tuning guide (#9539)
• [doc]The link in the document is 404 (#9561)
• [Docs] update jdbc related document (#9679)
• [Docs][Connector-Iceberg] Update Iceberg doc with support S3 Tables rest catalogs (#9686)
• [Fix][Doc] Fix rest api finished-jobs doc miss SAVEPOINT_DONE (#9676)
• [Fix][Doc] Fix transform-v2 TableFilter title (#9528)
• [Fix][Doc] Sink's plugin_input is incorrect configurations (#9492)
• [Fix][Doc] Update StarRocks doc change schema necessity to true (#9656)
• [Improve][Doc] Add GenericDialect support in jdbc doc (#9763)
• [Improve][Doc] Add multimodal support description to docs (#9652)
• [Improve][Doc] Add SeaTunnel tools into Readme (#9707)
• [Improve][Doc] Fix formatting of parameter types (#9753)
• [Improve][Doc] Update sql function result type ([#9703]) (#9711)

Bug Fix

• [Bug] [Connector-V2] Fix the issue of writing the ORC format Iceberg report "Illegal provider-class name" (#6754) (#9588)
• [Bug] [Connector-V2] Updates Iceberg version to 1.6.1 (#9387) (#9451)
• [Bug] [seatunnel-connector-http-base] An NPE (NullPointerException) will occur when the pageField is null (#9498)
• [Bug][connector-http] Fix paging request running infinitely (#9504)
• [Bug][Connector-V2] fix NPE when decimal type precision is incompatible for Paimon (#9452)
• [Bug][Connector-V2] NoSuchMethodError caused by Netty version conflict on Spark 3.3.0 (#9632)
• [bug][dist]fix the release packaging exception (#9436)
• [bugfix][Connector-V2] Fixed the load driver inaccurate situation (#9468)
• [Fix] [connector-jdbc] prevent precision loss in Float to BigDecimal conversion (#9670)
• [Fix] [Transform-V2] Fix embedding output columns vector dimension (#9646)
• [Fix][CI] Update iris docker version to fix ci (#9360)
• [Fix][Connecotr-kafka] Fix kafka IllegalArgumentException when offset is -1 (#9376)
• [Fix][Connector-clickhouse] Fix SeaTunnelRow tableId set error (#9585)
• [Fix][Connector-File] Fix parquet support user config schema (#9596)
• [Fix][Connector-HTTP] Add default content-type when user not set (#9497)
• [Fix][Connector-Http] fix Invalid mime type (#9363)
• [Fix][connector-http] fix parsing httpjson, the number of two fields is inconsistent with the import failure (#9103)
• [Fix][Connector-Iceberg] Fix Time Zone Issue for Iceberg Timestamp Type (#9460)
• [Fix][Connector-Jdbc] Supports reading and writing Postgres network dress types (#9618)
• [Fix][Connector-Jdbc]Fixed Vertica data source cannot upsert data. (#9607)
• [Fix][connector-rabbitmq] Set default value for durable, exclusive and auto-delete (#9631)
• [Fix][Connector-V2] Add Filter for Partitions to Prevent Blocking in KafkaConsumer StreamMode (#9598)
• [Fix][Connector-V2] ArrowToSeatunnelRowReader convertSeatunnelRowValue add handle Second TIMESTAMP type (#9393)
• [Fix][Connector-V2] Correct typo in batch-size-per-scan option key (#9434)
• [Fix][Connector-V2] Fix misleading parameter name in DorisStreamLoad (#9685)
• [Fix][Connector-V2] Fix OceanBase Oracle create unsupported data type (#9383)
• [Fix][Connector-V2] Fix prometheus check time can not parse double value (#9311)
• [Fix][Connector-V2] Fix starrocks decimal column definition generation(#9470) (#9471)
• [Fix][Connector-V2] Optimize Paimon DECIMAL type check to prevent precision loss (#9480)
• [Fix][Connector-V2] Optimize start mode of kafka recovery job (#9736)
• [Fix][Connector-V2] Oracle cdc not update transaction commit when LOB enabled (#9412)
• [Fix][Connector-V2] set default tableid in SeaTunnelRow for Debezium JSON deserialization (#9431)
• [Fix][Connector-V2] Update catalog table schema of debezium json (#9525)
• [Fix][Connector-V2] Update file filter pattern compilation to remove unnecessary quoting (#9658)
• [Fix][Connector-V2] Update waitCompaction value for batch mode and writeonly (#9479)
• [Fix][Connectors-Jdbc] Postgres supports streaming and batch reading and writing of the interval data type (#9590)
• [Fix][Connectors-v2] fix dynamic bucket for paimon sink (#9595)
• [fix][connectors-v2] repeated commit cause task exceptions (#9665)
• [Fix][Core]fix kotlin jar conflict (#9683)
• [Fix][E2E] Fixed e2e test cases of Kafka did not work as expected (#9359)
• [Fix][engine-server] Fix the issue of garbled characters in log output (#9594)
• [Fix][Mongo-CDC] Fix the issue where mongo isExactlyOnce defaults to true, causing room to malfunction (#9454)
• [Fix][paimon-e2e] e2e test error (#9467)
• [Fix][paimon-e2e] Optimize Paimon E2E Cases (#9612)
• [Fix][seatunnel-dist]fix duplicate dependency (#9458)
• [Fix][Spark] Fix source parallelism not working with Spark engine (#9319)
• [Fix][Transform-V2] correct handling of 'T' in date format strings (#9406)
• [Fix][Transform-v2] Fix custom udf throw exception (#9195)
• [Fix][Transform-V2] Reduce embedding precision from double to float (#9635)
• [Fix][Transform] Support integer input for from_unixtime function (#9738)
• [Fix][Zeta] Avoid redundant checkpoint reads when disabled checkpoint (#9552)
• [Fix][Zeta] change parallelStream to stream in SeaTunnelTask to avoid classloader mismatch (#9426)
• [Fix][Zeta] Enhance job state display by including pending jobs in status retrieval (#9489)
• [Fix][Zeta] Fix Imap resource leak (#9696)
• [Fix][Zeta] Fix local mode can not finish (#9549)
• [Fix][Zeta] Fix operation thread leak when master switch (#9464)
• [Fix][Zeta] Use dynamic port allocation for HTTP in RestApiHttpsForTruststoreTest and update Prometheus image (#9792)
• [Fix][POM] Fix transform udf dependency version (#9803)
• [Fix][Core] Fix the placeholders cannot pass parameters when using complex config (#9800)
• [Future][Connector-V2]Support the automatic creation of non-primary key table (#9219)
• [HotFix][CI] The Some CI subtasks are not executed (#9684)
• [Hotfix][Connector-Jdbc] Write MySQL to support set collection data type (#9553)
• [Hotfix][Transform-V2] Fix some model return number of dimensions (#9644)
• [Hotfix][Zeta] Fix custom https port configuration not working (#9705)

Feature

• [Chore] fix typos filed -> field (#9757)
• [Chore][Core] Remove useless parameter in ConfigBuilder (#9494)
• [Feature] [connector-file] Add configurable sheet_max_rows support for Excel sink connector (#9668)
• [Feature] Version from 2.3.11 to 2.3.12 (#9366)
• [Feature][Connector-Clickhouse] Support Clickhouse multi table source read (#9704)
• [Feature][Connector-File-Hadoop]Support multi table sink feature for HdfsFile (#9651)
• [Feature][connector-hive] hive sink connector support overwrite mode [#7843] (#7891)
• [Feature][connector-kudu] implement the filter (#9405)
• [Feature][connector-paimon] Paimon connector supports paimon privilege (#9722)
• [Feature][connector-tdengine] Support subtable and fieldNames in tdengine source (#9593)
• [Feature][connector-v2] Add Sensorsdata Connector Support [#9323] (#9432)
• [Feature][Connector-V2] Jdbc mysql support read tinyint(1) to byte(tinyint) (#9373)
• [Feature][Connector-V2] Support databend source/sink connector (#9331)
• [Feature][Connector-V2] Support like predicate pushdown in paimon (#9484)
• [Feature][Connector-v2] Support multi paimon source (#9759)
• [Feature][Connector-V2][JDBC] Add presto/trino dialect (#9388)
• [Feature][Connectors-v2] Optimize the size of CDC JAR Files (#9546)
• [Feature][Connectors-v2] Support auto-increment id for FakeSource (#9505)
• [feature][connectors-v2] Support in predicate pushdown in paimon (#9379)
• [Feature][Connectors-v2] Support Mysql8.4+ for mysql-cdc (#9720)
• [Feature][Connectors-V2][Elasticsearch] Support vector transformation sink (#9330)
• [Feature][Connectors-v2]Paimon version upgrade to 1.1.1 (#8074)
• [Feature][Core] Add plugin directory support for each connector (#9650)
• [Feature][Core] Support show cluster members information in seatunnel-cluster scripts (#9502)
• [Feature][elasticsearch-connector] Add API key authentication support (#9610)
• [Feature][Format] Improve maxwell_json,canal_json,debezium_json format add ts_ms and table (#9701)
• [Feature][Jdbc] Support read multiple tables by regular expressions (#9380)
• [Feature][MySQL CDC] MySQL cdc support start by time (#9735)
• [Feature][Redis] Add redis key into the result record (#9574)
• [Feature][Sink] File support new format: maxwell_json,canal_json,debezium_json (#9278) (#9336)
• [Feature][SQL function] Enhance SQL Transform COALESCE functions to support type cast (#9299)
• [Feature][SQL Transform] Check SQL Cast will fail or not during the SQL parsing stage (#9600)
• [Feature][Transform-V2] Add TRIM_SCALE function for sql transform (#9700)
• [Feature][transform-v2] Data Validator Transform support (#9445)
• [Feature][transform-v2] sql transform support multi_if function (#9154)
• [Feature][Transform-V2] Support jsonpath extracting multiple fields in a configuration block (#9712)
• [Feature][Transform-V2] Support vector series sql function (#9765)
• [Feature][Transform] Add Murmur64 function to obtain the lower 64 bit (#9748)
• [Feature][Transform] Support single/batch mode vectorization using Amazon Titan & cohere embedding model (#9120)
• [Feature][TransForms-V2] Support scala dynamic compile (#9625)
• [Feature][Transform V2] Add vector dimension reduction transform (#9783)
• [Feature][Zeta] Add metrics for task intermediate queue size (#9550)
• [Feature][Zeta] Add startTime to the job info (#9400)
• [Feature][Zeta] extended jobStateEvent event listening for zeta (#9689)
• [Feature][Zeta] Support SQL format in REST API (#9802)
• [improve] jdbc options (#9541)
• [Improve] JdbcInputFormat nextRecord Exception throw TableId (#9374)
• [improve] pulsar options (#9180)
• [improve] rocketmq options (#9251)
• [improve] selectdb options (#9252)
• [improve] socket options (#9517)
• [Improve] table_store options (#9515)
• [improve] tdengine options (#9399)
• [improve] typesense options (#9398)
• [Improve][API] Add metadata schema into catalog table (#9586)
• [Improve][API] Optimize the enumerator API semantics and reduce lock calls at the connector level (#9671)
• [Improve][CI] Disable close stable issue and JdbcVerticaIT (#9742)
• [Improve][CI] Update state action fix not all pr/issue be checked (#9557)
• [Improve][connector-clickhouse] Clickhouse support parallelism reading schema (#9446)
• [Improve][Connector-file] Add configurable binary chunk size support to BinaryReadStrategy (#9391)
• [Improve][Connector-Hudi] Add pre-combine field option for hudi sink (#9496)
• [improve][Connector-jdbc] add comments when schema not include all columns (#9559)
• [Improve][Connector-milvus]update milvus-sdk-java to 2.5.11 (#9710)
• [Improve][Connector-V2] Optimize Milvus doc and e2e test case (#9766)
• [Improve][Connector-V2] Add customizable row delimiter support for text file processing (#9608)
• [Improve][Connector-V2] Add tunnel_endpoint option to MaxCompute source for emulator test (#9548)
• [improve][Connector-V2] delete jdbc param support_upsert_by_query_primary_key_exist (#9408)
• [Improve][Connector-V2] File Source Support filtering files by last modified time. (#9526)
• [Improve][Connector-V2] Replace deprecated createDownloadSession by buildDownloadSession (#9555)
• [Improve][Connector-V2] Support like predicate pushdown in paimon (#9653)
• [Improve][Connector-V2] Support maxcompute sink writer upsert/delete action with upsert session mode (#9462)
• [Improve][Connector-V2] Support maxcompute sink writer with timestamp field type (#9234)
• [Improve][Connector-V2] Use key_field_name option when reading Redis hash data (#9642)
• [Improve][Core] Configure spotless to auto-replace imports with shaded packages. (#9655)
• [Improve][Core] Unify the aws-sdk-v2 version to 2.31.30 (#9698)
• [Improve][Core] Update apache common to apache common lang3 (#9694)
• [Improve][Csv] support configurable CSV delimiter in file connector (#9660)
• [Improve][E2E] Optimize Hudi E2E Cases (#9752)
• [Improve][E2E] Optimize iceberg E2E Cases (#9769)
• [Improve][Oracle-CDC] Remove duplicate load table names (#9357)
• [Improve][Transform-V2] Improve sql transform exception to locate error expression (#9227)
• [Improve][Transform-V2] Remove unnecessary unquoting method and streamline date parsing (#9478)
• [Improve][Transform-V2] SQL transform EXTRACT function support more fields (#9342)
• [Improve][Transform][SQL] Support cast to TINYINT/SMALLINT (#9347)
• [Improve][Zeta] Add verification for Env options (#9569)
• [Improve][Zeta] Improve local mode doc and default config (#9770)
• [Improve][Zeta] Make sure the pending job schedule always works (#9532)
• [Improve][Zeta] Optimize unstable CheckpointErrorRestoreEndTest cases (#9619)
• [Improve][Zeta] Remove distributed lock when store metrics imap (#9776)
• [Improve][Zeta] Set the default slot-num value to twice the number of CPU cores (#9601)
• [improve]improve FE node failover logging for better observability (#9657)

Full Changelog: https://github.com/apache/seatunnel/compare/2.3.11...2.3.12