Lance Files

What's Changed

New Features 🎉

• feat(java): brings transaction api to Java module and support project by @majin1102 in https://github.com/lancedb/lance/pull/4219
• feat: add loading trace event by @yanghua in https://github.com/lancedb/lance/pull/4298
• feat: add file reader options in ReadParams by @Xuanwo in https://github.com/lancedb/lance/pull/4287
• feat: add matrix type by @BubbleCal in https://github.com/lancedb/lance/pull/4322
• feat(java): expose sql api for java api by @fangbo in https://github.com/lancedb/lance/pull/4328
• feat: add compression override support for encoding by @Xuanwo in https://github.com/lancedb/lance/pull/4310
• feat: add sdk_version in tracing events by @ddupg in https://github.com/lancedb/lance/pull/4347
• feat: add transaction properties for commit message handling by @chenghao-guo in https://github.com/lancedb/lance/pull/4078
• feat: allow configure encoding via field metadata by @Xuanwo in https://github.com/lancedb/lance/pull/4354
• feat: add FragmentSession by @wojiaodoubao in https://github.com/lancedb/lance/pull/4343
• feat(java): support configure transaction properties by @majin1102 in https://github.com/lancedb/lance/pull/4363
• feat(python): release GIL in LanceFileReader & LanceFileWriter by @lorinlee in https://github.com/lancedb/lance/pull/4383
• feat: add verify for encoding roundtrip tests by @Xuanwo in https://github.com/lancedb/lance/pull/4380
• feat(java): introduce overwrite and append in transaction by @majin1102 in https://github.com/lancedb/lance/pull/4327
• feat: support nullability in struct correctly by @Xuanwo in https://github.com/lancedb/lance/pull/4391

Bug Fixes 🐛

• fix: add hnsw range search support by @LeoReeYang in https://github.com/lancedb/lance/pull/4311
• fix: use debug format for object store provider errors by @jackye1995 in https://github.com/lancedb/lance/pull/4313
• fix: update missing lance perf asset by @emmanuel-ferdman in https://github.com/lancedb/lance/pull/4316
• fix: fix can not scan and take correct rowids after compaction when enable move stable rowid feature by @yanghua in https://github.com/lancedb/lance/pull/4289
• fix: update broken README links by @adi-ray in https://github.com/lancedb/lance/pull/4338
• fix: flaky test test_tracing.py by @ddupg in https://github.com/lancedb/lance/pull/4341
• fix: correct the error enum in case of missing positions for phrase queries by @BubbleCal in https://github.com/lancedb/lance/pull/4342
• fix: support count(*) by @wojiaodoubao in https://github.com/lancedb/lance/pull/4309
• fix: fix bug in trim_ranges which could cause filtered read to miss rows by @westonpace in https://github.com/lancedb/lance/pull/4351
• fix: process hung when using tracing events by @ddupg in https://github.com/lancedb/lance/pull/4356
• fix: make sure cloud features aren't implicitly enabled (lancedb/lancedb#2567) by @vlovich in https://github.com/lancedb/lance/pull/4362
• fix: implement limit and offset support for FTS and vector search by @wjones127 in https://github.com/lancedb/lance/pull/4318
• fix: remove the broken large binary test with dictionary encoding && fsst compression by @HaochengLIU in https://github.com/lancedb/lance/pull/4349
• fix: don't panic when reading files if the last row was deleted by @westonpace in https://github.com/lancedb/lance/pull/4396

Documentation 📚

• docs: migrate python docs to uv based by @Xuanwo in https://github.com/lancedb/lance/pull/4346
• docs: clean up table.proto by @jbapple in https://github.com/lancedb/lance/pull/4359
• docs: minor fix by @lorinlee in https://github.com/lancedb/lance/pull/4379

Performance Improvements 🚀

• perf: improve L2 distance perf by @BubbleCal in https://github.com/lancedb/lance/pull/4321
• perf: pass session down when checking existing table by @wjones127 in https://github.com/lancedb/lance/pull/3677

Other Changes

• Revert "feat: add matrix type (#4322)" by @BubbleCal in https://github.com/lancedb/lance/pull/4326
• refactor: unify uri of dataset trace events by @ddupg in https://github.com/lancedb/lance/pull/4323
• refactor: use previous instead v2 for our old encoding algo by @Xuanwo in https://github.com/lancedb/lance/pull/4336
• refactor: simplify substrait logic, allow session state to be injected by @westonpace in https://github.com/lancedb/lance/pull/4357

New Contributors

• @emmanuel-ferdman made their first contribution in https://github.com/lancedb/lance/pull/4316
• @chenghao-guo made their first contribution in https://github.com/lancedb/lance/pull/4325
• @adi-ray made their first contribution in https://github.com/lancedb/lance/pull/4338
• @fangbo made their first contribution in https://github.com/lancedb/lance/pull/4328
• @vlovich made their first contribution in https://github.com/lancedb/lance/pull/4362
• @lorinlee made their first contribution in https://github.com/lancedb/lance/pull/4379

Full Changelog: https://github.com/lancedb/lance/compare/v0.32.0...v0.32.1