DuckDB Files

This is a bug fix release for various issues discovered after we released 1.3.0 "Ossivalis". There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9 can be read by this version.

What's Changed

• MultiFileReader: Fix for handling nested list/map default values by @Mytherin in https://github.com/duckdb/duckdb/pull/17589
• Signed to Unsigned is not reversible by @Tmonster in https://github.com/duckdb/duckdb/pull/17571
• [Dev][CLI] Use an unused bit for DUCKDB_LATEST_STORAGE_VERSION by @Tishj in https://github.com/duckdb/duckdb/pull/17598
• minor restructure MAIN_BRANCH_VERSIONING by @c-herrewijn in https://github.com/duckdb/duckdb/pull/17601
• Main branch versioning set to false by @c-herrewijn in https://github.com/duckdb/duckdb/pull/17602
• Generate correct UUID v7 by @Mytherin in https://github.com/duckdb/duckdb/pull/17612
• Issue [#17606]: Disable TIMESTAMPTZ Casts by @hawkfish in https://github.com/duckdb/duckdb/pull/17614
• Bugfixes by @lnkuiper in https://github.com/duckdb/duckdb/pull/17543
• Improve Windows lock conflict error by @Mytherin in https://github.com/duckdb/duckdb/pull/17622
• Actually initialize in batch copy to file by @Mytherin in https://github.com/duckdb/duckdb/pull/17627
• Issue [#17621]: Streaming Window Reset by @hawkfish in https://github.com/duckdb/duckdb/pull/17649
• CLI: Print codename for '-version' by @szarnyasg in https://github.com/duckdb/duckdb/pull/17637
• [Python][Dev] Ignore DYNAMIC_FILTER TableFilters in filter pushdown by @Tishj in https://github.com/duckdb/duckdb/pull/17657
• [Dev] Throw if db is not available yet in setting certain configuration options by @Tishj in https://github.com/duckdb/duckdb/pull/17659
• chore: Fix initialization by @krlmlr in https://github.com/duckdb/duckdb/pull/17643
• chore: Fix initialization by @krlmlr in https://github.com/duckdb/duckdb/pull/17644
• DefaultSecretGenerator: require lock for modifying persistent_secrets by @Mytherin in https://github.com/duckdb/duckdb/pull/17650
• initialize the read with the OpenFile info and not just the path by @Tmonster in https://github.com/duckdb/duckdb/pull/17652
• Don't bail on TopN optimization if we don't have a cardinality by @lnkuiper in https://github.com/duckdb/duckdb/pull/17654
• Fixes for CSV fuzzer tests by @pdet in https://github.com/duckdb/duckdb/pull/17678
• chore: Fix strict aliasing warning on GCC by @krlmlr in https://github.com/duckdb/duckdb/pull/17641
• Partitioned copy: don't check if file exists for remote files by @Mytherin in https://github.com/duckdb/duckdb/pull/17689
• Fix version detection for sdist builds without git info by @evertlammerts in https://github.com/duckdb/duckdb/pull/17605
• Bugfixes by @lnkuiper in https://github.com/duckdb/duckdb/pull/17695
• Do not get file handle unnecessarily. by @Tmonster in https://github.com/duckdb/duckdb/pull/17698
• Allow table functions to disable statement caching by @Mytherin in https://github.com/duckdb/duckdb/pull/17702
• [SQLLogicTester] Replace keywords in <FILE>:pattern result for the query statement by @Tishj in https://github.com/duckdb/duckdb/pull/17710
• Parquet Reader: only read strings as fixed length strings if the type is FIXED_LEN_BYTE_ARRAY by @Mytherin in https://github.com/duckdb/duckdb/pull/17723
• Internal [#5022]: IN Pushdown Equalities by @hawkfish in https://github.com/duckdb/duckdb/pull/17731
• Internal [#4995]: Commutative INTERVAL Multiply by @hawkfish in https://github.com/duckdb/duckdb/pull/17730
• Issue [#17725]: Quantile NaN Compare by @hawkfish in https://github.com/duckdb/duckdb/pull/17761
• Backport CI fixes by @Mytherin in https://github.com/duckdb/duckdb/pull/17763
• Have the skip_rows option consider empty csv lines by @pdet in https://github.com/duckdb/duckdb/pull/17756
• Fix wrong assertion in Parquet DBP encoder by @lnkuiper in https://github.com/duckdb/duckdb/pull/17746
• CLI: make -f always bail on error by @Mytherin in https://github.com/duckdb/duckdb/pull/17768
• Don't Flatten() then Reference() by @hannes in https://github.com/duckdb/duckdb/pull/17769
• [Dev] Fix TRY expression crash on literals by @Tishj in https://github.com/duckdb/duckdb/pull/17753
• Pop up ICU errors to the csv sniffer by @pdet in https://github.com/duckdb/duckdb/pull/17719
• [Nested] Fix incorrect type casting in list_reduce lambda expressions by @maiadegraaf in https://github.com/duckdb/duckdb/pull/17581
• [chore] Avoid caching msys artifacts on PRs by @carlopi in https://github.com/duckdb/duckdb/pull/17777
• Skip encodings and spatial extensions on PRs by @carlopi in https://github.com/duckdb/duckdb/pull/17775
• Add FileBufferType::EXTERNAL_FILE and add to same queue as FileBufferType::BLOCK by @lnkuiper in https://github.com/duckdb/duckdb/pull/17771
• Storge the argument and value of arg_min_max in the state as a unique_ptr by @pdet in https://github.com/duckdb/duckdb/pull/17749
• Physical operator logging by @lnkuiper in https://github.com/duckdb/duckdb/pull/17752
• Take string size into account in GetRowSize in ParquetWriter by @lnkuiper in https://github.com/duckdb/duckdb/pull/17793
• [CSV Sniffer] Consider if null_padding is set to true when detecting midcomment line during sniffing by @pdet in https://github.com/duckdb/duckdb/pull/17751
• Revert "Avoid early-out when catalog lookup fails - instead finish all look-ups" by @Mytherin in https://github.com/duckdb/duckdb/pull/17805
• Support file rotation with WRITE_EMPTY_FILE false by @Mytherin in https://github.com/duckdb/duckdb/pull/17804
• Check page filtered out flag before reading it by @Damon07 in https://github.com/duckdb/duckdb/pull/17786
• Avoid saving ccache on pull_requests by @carlopi in https://github.com/duckdb/duckdb/pull/17810
• Support glibc 2.28 environments in 1.3.x by @taniabogatsch in https://github.com/duckdb/duckdb/pull/17814
• arrow_output_version option to produce arrow depending on a format version. by @pdet in https://github.com/duckdb/duckdb/pull/17791
• Internal [#5069]: Win32 Cast Simplification by @hawkfish in https://github.com/duckdb/duckdb/pull/17820
• local_agnostic::isspace to avoid spaces be depending on locale by @carlopi in https://github.com/duckdb/duckdb/pull/17808
• MultiFileReader: Make column mapping mode configurable per-file, instead of requiring it to be set globally by @Mytherin in https://github.com/duckdb/duckdb/pull/17817
• Fixup confict by @carlopi in https://github.com/duckdb/duckdb/pull/17831
• [Fix] Throw serialisation error when encountering invalid row IDs in WAL delete by @taniabogatsch in https://github.com/duckdb/duckdb/pull/17832
• remap_struct: Correctly reserve list vectors to deal with remapping larger-than-vector-size lists/maps by @Mytherin in https://github.com/duckdb/duckdb/pull/17836
• Fixup unique_ptr<T[], deleter>, now with working custom deleters by @carlopi in https://github.com/duckdb/duckdb/pull/17840
• Fix ICE with Windows ARM64 (1.3) by @staticlibs in https://github.com/duckdb/duckdb/pull/17846
• [Fix] WAL replay catalog error in AddForeignKeyConstraint by @taniabogatsch in https://github.com/duckdb/duckdb/pull/17830
• Add ducklake sha by @c-herrewijn in https://github.com/duckdb/duckdb/pull/17818
• add ducklake to internal_extensions by @c-herrewijn in https://github.com/duckdb/duckdb/pull/17811
• Use boolean operators instead of bitwise operators for (u)hugeint by @lnkuiper in https://github.com/duckdb/duckdb/pull/17862
• Bump httpfs by @Tmonster in https://github.com/duckdb/duckdb/pull/17863
• Function Serialization: adapt to removal of overloads by explicitly casting if argument types have changed by @Mytherin in https://github.com/duckdb/duckdb/pull/17867
• Add FinalizeLoad callback to catalogs, which can be called after the database is fully instantiated by @Mytherin in https://github.com/duckdb/duckdb/pull/17868
• Set query location for interval constants in all cases by @Mytherin in https://github.com/duckdb/duckdb/pull/17876
• Add support for option.scheduler_process_partial to the Executor by @bleskes in https://github.com/duckdb/duckdb/pull/17877
• Add array indices to nested paths in json_each/json_tree by @lnkuiper in https://github.com/duckdb/duckdb/pull/17878
• Add urllib3 dependency and improve fixture download reliability by @evertlammerts in https://github.com/duckdb/duckdb/pull/17880
• Add option to control parquet NaN pruning by @Maxxen in https://github.com/duckdb/duckdb/pull/17883
• Issue [#17781]: ASOF Predicate Binding by @hawkfish in https://github.com/duckdb/duckdb/pull/17889
• duckdb_base_std:: plus compile time test on discontinued functions by @carlopi in https://github.com/duckdb/duckdb/pull/17866
• Early-out on catalog errors to prevent GetDatabases by @taniabogatsch in https://github.com/duckdb/duckdb/pull/17865
• Fix mismatch between idx_t (64 bits) and size_t (implementation dependent) by @carlopi in https://github.com/duckdb/duckdb/pull/17898
• Skipping failing on OSX Release part by @hmeriann in https://github.com/duckdb/duckdb/pull/17899
• [Fix] Serialisation error for invalid block ID in index deserialization by @taniabogatsch in https://github.com/duckdb/duckdb/pull/17900
• Update MemoryTag when converting block to persistent by @lnkuiper in https://github.com/duckdb/duckdb/pull/17893
• Add support for deserializing multiple json-serialized statements by @Maxxen in https://github.com/duckdb/duckdb/pull/17902
• Deprecated std::stringstream by @carlopi in https://github.com/duckdb/duckdb/pull/17894
• Bump extensions by @Mytherin in https://github.com/duckdb/duckdb/pull/17905
• http_log.test: solve non-determinism at the test level by @carlopi in https://github.com/duckdb/duckdb/pull/17914
• Fixup https://github.com/duckdb/duckdb/pull/17775, correct boolean logic by @carlopi in https://github.com/duckdb/duckdb/pull/17912
• Make sure distance is always an int when doing version bumps in setup.py by @c-herrewijn in https://github.com/duckdb/duckdb/pull/17918
• bump azure, aws and httpfs by @samansmink in https://github.com/duckdb/duckdb/pull/17919
• bump spatial by @Maxxen in https://github.com/duckdb/duckdb/pull/17920
• Add v1.3.1 to version_map.json by @Flogex in https://github.com/duckdb/duckdb/pull/17922
• Bump Avro and Iceberg + fix by @carlopi in https://github.com/duckdb/duckdb/pull/17930
• Bump Avro and Iceberg in out_of_tree_extensions.cmake by @Tishj in https://github.com/duckdb/duckdb/pull/17923
• Enable Test extension (inside docker) step in build_extensions_dockerized by @hmeriann in https://github.com/duckdb/duckdb/pull/17916
• [chore] Skip 2 tests in autoloading mode (parquet + ZSTD) plus improve error message by @carlopi in https://github.com/duckdb/duckdb/pull/17935
• [tpcds] dsdgen currently assumes DuckTable, throw if that's not the case by @carlopi in https://github.com/duckdb/duckdb/pull/17934

Full Changelog: https://github.com/duckdb/duckdb/compare/v1.3.0...v1.3.1