Sail Files

Overview

You can find the release summary in the documentation.

The PySail Python package is available on PyPI.

What's Changed

• test: improve function test selection by @linhr in https://github.com/lakehq/sail/pull/2352
• fix: retry Iceberg REST bootstrap authentication by @Robin-Everaars in https://github.com/lakehq/sail/pull/2348
• fix: validate explicit Parquet read schemas by @rohankumardubey in https://github.com/lakehq/sail/pull/2351
• test: initialize the global state when reusing an existing server by @davidlghellin in https://github.com/lakehq/sail/pull/2353
• refactor: move server and telemetry code by @linhr in https://github.com/lakehq/sail/pull/2355
• feat: support basic merge operation for Iceberg by @lonless9 in https://github.com/lakehq/sail/pull/2222
• refactor: improve child actor management by @linhr in https://github.com/lakehq/sail/pull/2359
• feat: list Spark 4.2 built-in functions by @lonless9 in https://github.com/lakehq/sail/pull/2325
• docs: add LangChain integration guide by @pomykalakyle in https://github.com/lakehq/sail/pull/2354
• deps(cargo): bump the all group with 6 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2366
• deps(npm): bump vue-tsc from 3.3.8 to 3.3.9 in the all group by @dependabot[bot] in https://github.com/lakehq/sail/pull/2363
• fix: make Python data source options case-insensitive by @ErikBPF in https://github.com/lakehq/sail/pull/2371
• chore: cargo update by @lonless9 in https://github.com/lakehq/sail/pull/2365
• fix: resolve Delta MERGE constraints by visible names by @alexy in https://github.com/lakehq/sail/pull/2374
• feat: add basic Celeborn lifecycle manager by @linhr in https://github.com/lakehq/sail/pull/2373
• fix: harden Iceberg MERGE semantics by @lonless9 in https://github.com/lakehq/sail/pull/2362
• feat: implement vector inner product by @jaideeppyne in https://github.com/lakehq/sail/pull/2360
• fix: propagate swallowed cascade errors in Iceberg drop_database by @Robin-Everaars in https://github.com/lakehq/sail/pull/2297
• fix: idempotent worker shutdown cleanup by @weixiuli in https://github.com/lakehq/sail/pull/2392
• feat: add basic Celeborn shuffle client by @linhr in https://github.com/lakehq/sail/pull/2378
• refactor: rename actor message and move actor options by @linhr in https://github.com/lakehq/sail/pull/2397
• test: align function tests with Spark values by @davidlghellin in https://github.com/lakehq/sail/pull/2375
• refactor: generalize task stream management by @linhr in https://github.com/lakehq/sail/pull/2398
• refactor: add task runner actor by @linhr in https://github.com/lakehq/sail/pull/2399
• fix: improve Delta protocol and checkpoint compatibility by @lonless9 in https://github.com/lakehq/sail/pull/2231
• feat: add Celeborn shuffle backend by @linhr in https://github.com/lakehq/sail/pull/2403
• fix: refine Celeborn shuffle backend implementation by @linhr in https://github.com/lakehq/sail/pull/2414
• feat: improve Celeborn transport and support connection reuse by @linhr in https://github.com/lakehq/sail/pull/2415
• fix: support three-argument dateadd by @rohankumardubey in https://github.com/lakehq/sail/pull/2429
• deps(npm): bump the all group with 6 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2425
• deps(cargo): bump the all group with 4 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2427
• fix: honor ANSI mode for invalid to_date strings by @lonless9 in https://github.com/lakehq/sail/pull/2405
• fix: align empty structs and to_json key ordering with Spark by @lonless9 in https://github.com/lakehq/sail/pull/2408
• fix: coerce string sum across aggregate and window paths by @lonless9 in https://github.com/lakehq/sail/pull/2410
• feat: support Celeborn revive logic by @linhr in https://github.com/lakehq/sail/pull/2423
• chore: always show gold data report by @linhr in https://github.com/lakehq/sail/pull/2431
• fix: shuffle output mode for scalar subquery by @linhr in https://github.com/lakehq/sail/pull/2432
• test: port HMS provider integration tests to Python by @lonless9 in https://github.com/lakehq/sail/pull/2416
• fix: align USING and NATURAL join output-key ownership with Spark by @lonless9 in https://github.com/lakehq/sail/pull/2411
• fix: lay the correctness groundwork for the DataFusion upgrade by @lonless9 in https://github.com/lakehq/sail/pull/2401
• fix: replace malformed UTF-8 in CSV reads by @lonless9 in https://github.com/lakehq/sail/pull/2412
• fix: align non-ANSI STRING-to-INT casts with Spark by @lonless9 in https://github.com/lakehq/sail/pull/2406
• fix: match Spark string and day-time interval addition semantics by @lonless9 in https://github.com/lakehq/sail/pull/2407
• fix: align CASE, IF, and COALESCE string/temporal coercion with Spark by @lonless9 in https://github.com/lakehq/sail/pull/2409
• fix: pin Iceberg writer metadata during planning by @lonless9 in https://github.com/lakehq/sail/pull/2402
• chore: update TPC-H benchmark results by @linhr in https://github.com/lakehq/sail/pull/2439
• chore: update README with the latest benchmark results by @linhr in https://github.com/lakehq/sail/pull/2440
• docs: add notes for benchmark by @linhr in https://github.com/lakehq/sail/pull/2441
• fix: preserve lambda bindings across projection pushdown by @lonless9 in https://github.com/lakehq/sail/pull/2413
• test: port Iceberg REST catalog tests to Python by @lonless9 in https://github.com/lakehq/sail/pull/2434
• test: add general proxy for Celeborn tests by @linhr in https://github.com/lakehq/sail/pull/2442
• fix: support Celeborn soft split properly by @linhr in https://github.com/lakehq/sail/pull/2443
• fix: support Python data source filter pushdown on PySpark 4.0 by @maxvwa in https://github.com/lakehq/sail/pull/2437
• test: fix flaky Celeborn partition split test by @lonless9 in https://github.com/lakehq/sail/pull/2447
• fix: correct SQL sequence syntax descriptor cardinality by @lonless9 in https://github.com/lakehq/sail/pull/2372
• perf: compile column regex patterns once per batch, not per row by @radmirnovii in https://github.com/lakehq/sail/pull/2448
• deps(cargo): bump the all group with 6 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2454
• deps(npm): bump the all group with 5 updates by @dependabot[bot] in https://github.com/lakehq/sail/pull/2452
• feat: expose some optimizer config options by @linhr in https://github.com/lakehq/sail/pull/2451
• chore: clippy 1.98.0 warnings by @lonless9 in https://github.com/lakehq/sail/pull/2460
• fix: enable metadata-only COUNT(1) for Delta tables by @lonless9 in https://github.com/lakehq/sail/pull/2461
• test: port Glue tests to Python by @lonless9 in https://github.com/lakehq/sail/pull/2445
• test: fix Iceberg row-level tests on Windows by @lonless9 in https://github.com/lakehq/sail/pull/2474
• fix: return signed row counts from delta commit plans by @lonless9 in https://github.com/lakehq/sail/pull/2475
• fix: correct DV scans and log replay planning for Delta Lake by @lonless9 in https://github.com/lakehq/sail/pull/2450
• feat: improve Celeborn integration by @linhr in https://github.com/lakehq/sail/pull/2465
• fix: improve sequence, sampling, and convert_tz parity by @shehabgamin in https://github.com/lakehq/sail/pull/2430
• feat: event processor for system tables by @linhr in https://github.com/lakehq/sail/pull/2477
• refactor: declare output nullability for scalar functions Spark always marks nullable by @davidlghellin in https://github.com/lakehq/sail/pull/2436
• test: port Unity Catalog tests to Python by @lonless9 in https://github.com/lakehq/sail/pull/2466
• fix: supervise cluster task panics by @rohankumardubey in https://github.com/lakehq/sail/pull/2480
• fix: preserve null rows in Delta DELETE by @lonless9 in https://github.com/lakehq/sail/pull/2321
• fix: disabled file cache configuration by @shehabgamin in https://github.com/lakehq/sail/pull/2481
• fix: enable partition pruning for Delta MERGE target scans by @lonless9 in https://github.com/lakehq/sail/pull/2478
• fix: allow multipart parts to upload concurrently by @shehabgamin in https://github.com/lakehq/sail/pull/2482
• test: remove flaky object store test by @linhr in https://github.com/lakehq/sail/pull/2485
• perf: decode only the projected columns when reading string-typed CSV by @shehabgamin in https://github.com/lakehq/sail/pull/2484
• feat: optimize grouped and exact aggregates from metadata for Delta Lake by @lonless9 in https://github.com/lakehq/sail/pull/2467
• feat: update Celeborn endpoint config by @linhr in https://github.com/lakehq/sail/pull/2486
• docs: add shuffle and checkpoint guides by @linhr in https://github.com/lakehq/sail/pull/2491
• fix: stabilize Iceberg row-level operations with storage shuffle by @lonless9 in https://github.com/lakehq/sail/pull/2490
• chore: prepare v0.7.1 by @linhr in https://github.com/lakehq/sail/pull/2487

New Contributors

• @ErikBPF made their first contribution in https://github.com/lakehq/sail/pull/2371
• @jaideeppyne made their first contribution in https://github.com/lakehq/sail/pull/2360
• @maxvwa made their first contribution in https://github.com/lakehq/sail/pull/2437

Full Changelog: https://github.com/lakehq/sail/compare/v0.7.0...v0.7.1