Lance Files

What's Changed

Breaking Changes 🛠

• feat(python)!: unify Python CreateIndex api with Rust implementation by @chenghao-guo in https://github.com/lancedb/lance/pull/4438

New Features 🎉

• feat(python): release GIL in dataset API by @lorinlee in https://github.com/lancedb/lance/pull/4389
• feat: expose decoder config at python side by @Xuanwo in https://github.com/lancedb/lance/pull/4335
• feat: allow empty projection by @westonpace in https://github.com/lancedb/lance/pull/4340
• feat(java): dataset supports reading transaction by @majin1102 in https://github.com/lancedb/lance/pull/4397
• feat: implement explain_plan and analyze_plan for merge_insert operations by @wjones127 in https://github.com/lancedb/lance/pull/4295
• feat(rust): support diff_meta api by @yanghua in https://github.com/lancedb/lance/pull/4404
• feat: auto-build Lance namespace and Spark doc in main Lance website by @jackye1995 in https://github.com/lancedb/lance/pull/4411
• feat: allow row ids (offsets, addresses) in filters to be translated into an optimized take op by @westonpace in https://github.com/lancedb/lance/pull/4394
• feat(java): java transaction supports merge operation by @majin1102 in https://github.com/lancedb/lance/pull/4419
• feat(java): transaction supports UpdateConfig operation by @majin1102 in https://github.com/lancedb/lance/pull/4406
• feat: allow configuring bss for encoding by @Xuanwo in https://github.com/lancedb/lance/pull/4393
• feat(java): java transaction supports rewrite operation by @majin1102 in https://github.com/lancedb/lance/pull/4414
• feat(java): java transaction support restore and reserveFragments operation by @majin1102 in https://github.com/lancedb/lance/pull/4435
• feat(java): introduce update in transaction by @fangbo in https://github.com/lancedb/lance/pull/4408
• feat: allow creating empty scalar indices by @wjones127 in https://github.com/lancedb/lance/pull/4033
• feat: support blob encoding in format 2.1 by @Xuanwo in https://github.com/lancedb/lance/pull/4449
• feat(java): introduce delete in transaction by @fangbo in https://github.com/lancedb/lance/pull/4441
• feat(rust): support update stable rowid without index by @yanghua in https://github.com/lancedb/lance/pull/4388

Bug Fixes 🐛

• fix: limit the depth of filter expression to 500 by @BubbleCal in https://github.com/lancedb/lance/pull/4403
• fix: make sure aws feature is applied to code correctly by @vlovich in https://github.com/lancedb/lance/pull/4398
• fix: fix target alignment incorrect by using LanceBuffer borrow_to_typed_slice by @Xuanwo in https://github.com/lancedb/lance/pull/4413
• fix: wrong OSS path append by @adrian-wang in https://github.com/lancedb/lance/pull/4424
• fix: too large data chunk generated by highly compressed yet nested data with RLE by @Xuanwo in https://github.com/lancedb/lance/pull/4431
• fix: remove explain in sql by @wojiaodoubao in https://github.com/lancedb/lance/pull/4428
• fix: panic message when LanceFileWriter is not closed and then got garbage collected by @wojiaodoubao in https://github.com/lancedb/lance/pull/4128
• fix: update broken links by @ddupg in https://github.com/lancedb/lance/pull/4447
• fix: fix bss been enabled in wrong by @Xuanwo in https://github.com/lancedb/lance/pull/4444
• fix: none ASCII path issue by @majin1102 in https://github.com/lancedb/lance/pull/4442
• fix: btree prewarm causes offset overflow by @BubbleCal in https://github.com/lancedb/lance/pull/4448
• fix: do not panic when reading slices of bitmap column in 2.0 reader by @westonpace in https://github.com/lancedb/lance/pull/4452

Performance Improvements 🚀

• perf: introduce assume for generating better code by @BubbleCal in https://github.com/lancedb/lance/pull/4439

Other Changes

• refactor: optimize MiniBlockRepIndex decode by decoding from bytes directly by @Xuanwo in https://github.com/lancedb/lance/pull/4440
• refactor: refactor encoding proto for 2.1 to cleanup and future-proof by @westonpace in https://github.com/lancedb/lance/pull/4434

New Contributors

• @adrian-wang made their first contribution in https://github.com/lancedb/lance/pull/4424

Full Changelog: https://github.com/lancedb/lance/compare/v0.32.1...v0.33.0