Sail Files

Overview

You can find the release summary in the documentation.

The PySail Python package is available on PyPI.

What's Changed

• fix: use TOP for MSSQL schema queries by @ErikBPF in https://github.com/lakehq/sail/pull/2495
• refactor: separate data and lake sources by @lonless9 in https://github.com/lakehq/sail/pull/2417
• feat: prefix scan for system tables by @linhr in https://github.com/lakehq/sail/pull/2516
• chore: datafusion 55.0 by @lonless9 in https://github.com/lakehq/sail/pull/2383
• deps(npm): bump the all group with 2 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2506
• deps(cargo): bump the all group across 1 directory with 6 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2508
• refactor: unify lake row-level operation entry points by @lonless9 in https://github.com/lakehq/sail/pull/2418
• feat: support scoped Iceberg overwrite by @Robin-Everaars in https://github.com/lakehq/sail/pull/2473
• fix: preserve nested struct field schema by @lonless9 in https://github.com/lakehq/sail/pull/2520
• feat: system table for metrics by @linhr in https://github.com/lakehq/sail/pull/2521
• feat: support unbounded actor mailbox by @linhr in https://github.com/lakehq/sail/pull/2524
• refactor: remove async_trait for actor by @linhr in https://github.com/lakehq/sail/pull/2525
• feat: support UPDATE operation and unify COW/DV row operations for Delta Lake by @lonless9 in https://github.com/lakehq/sail/pull/2419
• feat: system store with memory and disk backends by @linhr in https://github.com/lakehq/sail/pull/2527
• chore: bump datasketches to 0.4.0 by @lonless9 in https://github.com/lakehq/sail/pull/2488
• feat: improve MERGE join strategies by clause and write mode by @lonless9 in https://github.com/lakehq/sail/pull/2529
• chore: bump Rust MSRV to 1.97.1 by @lonless9 in https://github.com/lakehq/sail/pull/2523
• feat: improve system store by @linhr in https://github.com/lakehq/sail/pull/2533
• feat: improve metrics setup by @linhr in https://github.com/lakehq/sail/pull/2534
• chore: remove the vendored pyarrow crate by @linhr in https://github.com/lakehq/sail/pull/2535
• fix: correct MERGE row preservation and Delta DV metrics by @lonless9 in https://github.com/lakehq/sail/pull/2532
• fix: preserve Iceberg snapshots and manifests during scoped overwrites by @lonless9 in https://github.com/lakehq/sail/pull/2531
• feat: improve system table reader by @linhr in https://github.com/lakehq/sail/pull/2541
• fix: correct Iceberg metadata discovery and manifest serialization by @lonless9 in https://github.com/lakehq/sail/pull/2542
• test: use realistic seeded data for ClickBench plan snapshots by @lonless9 in https://github.com/lakehq/sail/pull/2543
• deps(npm): bump the all group with 6 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2536
• deps(cargo): bump the all group across 1 directory with 3 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2538
• fix: correct partition manifest encoding and path formatting for Iceberg by @lonless9 in https://github.com/lakehq/sail/pull/2544
• fix: improve worker launch logic by @linhr in https://github.com/lakehq/sail/pull/2547
• fix: correct legacy collection sizes and table column regex selection by @shehabgamin in https://github.com/lakehq/sail/pull/2550
• test: pin sqlglot <30.18 for ibis compatibility by @lonless9 in https://github.com/lakehq/sail/pull/2552
• fix: coerce non-string inputs for formatted to_date by @shehabgamin in https://github.com/lakehq/sail/pull/2554
• fix: minor driver issues by @linhr in https://github.com/lakehq/sail/pull/2551
• fix: omit untyped null struct fields from JSON output by @shehabgamin in https://github.com/lakehq/sail/pull/2555
• feat: support reading multi-part checkpoints for Delta Lake by @lonless9 in https://github.com/lakehq/sail/pull/2546
• feat: implement vector cosine similarity by @rohankumardubey in https://github.com/lakehq/sail/pull/2557
• fix: let Delta writes inherit input parallelism by @zemin-piao in https://github.com/lakehq/sail/pull/2562
• test: migrate local object storage from MinIO to Silo by @lonless9 in https://github.com/lakehq/sail/pull/2574
• fix: handle URI-encoded file paths in Delta MERGE and UPDATE by @lonless9 in https://github.com/lakehq/sail/pull/2576
• deps(npm): bump the all group with 7 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2566
• deps(cargo): bump the all group with 9 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2568
• fix: minor issues in cluster mode by @linhr in https://github.com/lakehq/sail/pull/2591
• fix: correct partition count for rescaled stage input by @linhr in https://github.com/lakehq/sail/pull/2594
• fix: improve task assignment logic by @linhr in https://github.com/lakehq/sail/pull/2595
• chore: bump datafusion to 55.1.0 by @lonless9 in https://github.com/lakehq/sail/pull/2599
• deps(npm): bump the all group with 4 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2596
• deps(cargo): bump the all group with 8 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2598
• fix: apply runtime package security updates by @zemin-piao in https://github.com/lakehq/sail/pull/2602
• feat: optimize existence checks for Delta Lake and support projected EXISTS by @lonless9 in https://github.com/lakehq/sail/pull/2593
• fix: correct Delta statistics for dotted columns and all-null files by @lonless9 in https://github.com/lakehq/sail/pull/2604
• fix: support numeric arguments in btrim by @shehabgamin in https://github.com/lakehq/sail/pull/2614
• chore: align sail tests with cb by @shehabgamin in https://github.com/lakehq/sail/pull/2615
• feat: compressed stream and concurrent write for pipelined shuffle by @linhr in https://github.com/lakehq/sail/pull/2609
• fix: bucket balancing for tasks by @linhr in https://github.com/lakehq/sail/pull/2616
• fix: propagate batch size to worker by @linhr in https://github.com/lakehq/sail/pull/2623
• perf: reuse all-null arrays in abs by @shehabgamin in https://github.com/lakehq/sail/pull/2624
• fix: improve task launch performance by @linhr in https://github.com/lakehq/sail/pull/2626
• perf: optimize Parquet scan for Delta Lake by @lonless9 in https://github.com/lakehq/sail/pull/2613
• feat: batch coalescing for shuffle writer by @linhr in https://github.com/lakehq/sail/pull/2631
• feat: forward positional load paths to Python datasources by @jonasdedden in https://github.com/lakehq/sail/pull/2583
• feat: let an embedder choose the session factory by @alexy in https://github.com/lakehq/sail/pull/2630
• fix: normalize JDBC query subqueries by @ErikBPF in https://github.com/lakehq/sail/pull/2515
• fix: preserve row counts in zero-column Python datasource batches by @jonasdedden in https://github.com/lakehq/sail/pull/2584
• feat: support COW operations for Iceberg by @lonless9 in https://github.com/lakehq/sail/pull/2579
• feat: move batch coalescing from shuffle writer to reader by @linhr in https://github.com/lakehq/sail/pull/2632
• feat: implement vector L2 distance by @rohankumardubey in https://github.com/lakehq/sail/pull/2617
• fix: correct distributed lakehouse commits and nested-loop joins by @lonless9 in https://github.com/lakehq/sail/pull/2634
• perf: optimize Delta metadata aggregation by @lonless9 in https://github.com/lakehq/sail/pull/2608
• fix: preserve implicit struct field names in higher-order functions by @lonless9 in https://github.com/lakehq/sail/pull/2607
• test: cover string and date join key coercion by @shehabgamin in https://github.com/lakehq/sail/pull/2619
• fix: support LargeList columns for Iceberg by @lonless9 in https://github.com/lakehq/sail/pull/2640
• perf: early filter in join reorder by @linhr in https://github.com/lakehq/sail/pull/2605
• fix: implicitly cast a string value to double in round by @shehabgamin in https://github.com/lakehq/sail/pull/2641
• fix: support lambda predicates and map_filter by @shehabgamin in https://github.com/lakehq/sail/pull/2620
• fix: support time-only strings in timestamp conversions by @shehabgamin in https://github.com/lakehq/sail/pull/2621
• test: restore process timezone after Arrow UDF tests by @lonless9 in https://github.com/lakehq/sail/pull/2644
• fix: coerce timestamp string predicates with Spark semantics by @lonless9 in https://github.com/lakehq/sail/pull/2463
• deps(npm): bump the all group with 4 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2647
• deps(cargo): bump the all group across 1 directory with 4 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2649
• fix: rewrite binary grouping by @lonless9 in https://github.com/lakehq/sail/pull/2606
• feat: metadata aggregate for Iceberg by @lonless9 in https://github.com/lakehq/sail/pull/2636
• fix: align Pandas UDF execution and SQL behavior with Spark by @lonless9 in https://github.com/lakehq/sail/pull/2575
• feat: support v3 MOR operations for Iceberg by @lonless9 in https://github.com/lakehq/sail/pull/2655
• fix: reconcile catalog and Delta log schemas and fix column mapping reads for Delta Lake by @zemin-piao in https://github.com/lakehq/sail/pull/2646
• fix: windows Iceberg path handling and timestamp tests by @lonless9 in https://github.com/lakehq/sail/pull/2660
• fix: read Parquet files with second and millisecond timestamps by @james-willis in https://github.com/lakehq/sail/pull/2522
• test: run Python tests in parallel and run integration tests on schedule by @linhr in https://github.com/lakehq/sail/pull/2665
• feat: support repartition query hint by @pomykalakyle in https://github.com/lakehq/sail/pull/2590
• fix: include string statistics in describe by @pomykalakyle in https://github.com/lakehq/sail/pull/2589
• feat: custom Flight client transport by @linhr in https://github.com/lakehq/sail/pull/2664
• fix: resolve filter columns removed by projections by @shehabgamin in https://github.com/lakehq/sail/pull/2618
• fix: reduce partition count for small Parquet scan by @linhr in https://github.com/lakehq/sail/pull/2666
• feat: integrate mimalloc as the global allocator by @lonless9 in https://github.com/lakehq/sail/pull/2675
• fix: semi-join build side selection by @linhr in https://github.com/lakehq/sail/pull/2676
• deps(npm): bump the all group with 6 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2677
• docs: add spark-to-sail scanner mention by @everettroeth in https://github.com/lakehq/sail/pull/2673
• docs: update guide for Delta Lake and Iceberg by @lonless9 in https://github.com/lakehq/sail/pull/2656
• feat: expose window top-N optimizer config by @linhr in https://github.com/lakehq/sail/pull/2681
• feat: add built-in async Jev SQL functions by @shehabgamin in https://github.com/lakehq/sail/pull/2653
• fix: widen nanosecond timestamps for CSV schema inference by @alexy in https://github.com/lakehq/sail/pull/2672
• fix: update interval metadata by @linhr in https://github.com/lakehq/sail/pull/2685
• docs: polish docs by @linhr in https://github.com/lakehq/sail/pull/2686
• chore: prepare v0.7.2 by @linhr in https://github.com/lakehq/sail/pull/2683

New Contributors

• @jonasdedden made their first contribution in https://github.com/lakehq/sail/pull/2583

Full Changelog: https://github.com/lakehq/sail/compare/v0.7.1...v0.7.2