Faiss Files

Highlights

• SuperKMeans — faster k-means via ADSampling + PDX progressive pruning (#5168)
• Metal GPU backend for Apple Silicon, starting with IndexFlat (#5144)
• RVV (RISC-V Vector) SIMD backend + RISC-V cross-compilation CI (#5156, [#5184])
• pip install support via scikit-build-core + cibuildwheel (#4862)
• SVS IVF index enabled; SVS upgraded to 0.3.0 (#4801, [#4999]); SVSVamana usable as IVF coarse quantizer via index_factory (#5175)
• TurboQuant (CPU) quantizer (#5049)
• HNSW improvements — optional persistent locks for incremental adds (#5031), prune_headroom to avoid O(n²) pruning (#4847), narrower add-time critical section (#4915)
• Extensive index deserialization hardening — validation and OOM/overflow protection across ~40 PRs (binary, IVF, HNSW, FastScan, ScalarQuantizer, AdditiveQuantizer, ProductQuantizer, SVS, VectorTransform, …)
• Major Dynamic Dispatch refactor across distances, hamming, partitioning, RaBitQ, FastScan; dynamic dispatch now builds on Windows (#5127)
• cuVS upgraded to 26.02 (#4945, [#4788]); CI moved to CUDA 13.2 (#5207); C++20 for cuVS (#4881); filtered search for cuVS indexes (#4858)
• OMP exception safety across IndexHNSW, IndexNNDescent, IndexNSG, IndexFlatCodes, IndexIVF (#5106, [#5105], [#5111], [#5133], [#5037], [#5053])

Full Changelog

Added

• [b7618f] Add SuperKMeans: faster k-means via ADSampling+PDX progressive pruning (#5168)
• [66c9d0] Add SuperKMeans foundations: math primitives, PDX layout, SIMD kernels
• [dc7afc] Support SVSVamana as IVF coarse quantizer via index_factory (#5175)
• [7a8e4d] CI: cross-compile for riscv64 with RVV dynamic dispatch (#5184)
• [032027] Introduce RVV (#5156)
• [417c53] Add NoneSIMDLevel context manager for cross-level reference checks (#5158)
• [46def4] Enable SVS IVF Index in FAISS (#4801)
• [585ba7] Add pip install support via scikit-build-core + cibuildwheel (#4862)
• [799bf3] Introduce early stop facilities for IVF, attempt 2 (#5160)
• [66cea5] Add Metal GPU backend for Apple Silicon (IndexFlat) (#5144)
• [17fd33] Add database-parallel flat search for few-query workloads (#5000)
• [c7f07b] Add C++ per-SIMD-level tests for distance utility functions (#5154)
• [024275] Add bit-exact RQ encode cross-level test (#5155)
• [2f68a3] Add per-SIMD-level tests for hamming.h utils + fix crosshamming bug (#5153)
• [a7116e] Add TurboQuant (CPU) (#5049)
• [11e5d3] Add optional persistent locks to IndexHNSW for incremental adds (#5031)
• [e82aa2] Add OMP exception capture helpers and migrate IndexIVF (#5111)
• [6299bf] Add QT_0bit to ScalarQuantizer for centroid-only IVF distance (#5079)
• [3030fe] Add filtered search for cuVS indexes (#4858)
• [f83ea5] Support IVFRaBitQSearchParameters in RaBitQFastScan scanner (#5081)
• [82a823] Add iterator callbacks for distance computation and heap admission (#5082)
• [b068fd] Support IO_FLAG_MMAP_IFC on Darwin (#5058)
• [113bf6] Expose fast_scan_code_size from non-IVF FastScan indices (#5077)
• [786d99] Expose a fast scan code size from fast scan indices (#5064)
• [95f1e4] OMP build diagnostics (#4842)
• [c4ac85] Add a balanced assignment function (#5050)
• [8402a6] Add unit tests for HNSW deserialization validation (#5033)
• [1dc5e1] V2 enable specifying a subset of SIMD levels that are implemented at dispatching time. (#4959)
• [e6f5c0] Add SIMD dynamic dispatch migration guide (#4973)
• [8177a3] Add nvidia-smi diagnostic call (#4960)
• [67fc66] Add CappedInvertedLists for per-list size limiting (#4880)
• [e57a89] introduce SingleQueryResultCollectHandler (#4926)
• [748c03] HNSW: add prune_headroom to avoid O(n^2) pruning/locking, headroom test (#4847)
• [1e4d22] Introduce an early stop threshold for Kmeans (#4894)
• [4a0fba] Add FastScanCodeScanner dispatch boundary with per-SIMD TUs (#4897)
• [0fcba2] Add sentinels to index format enums. (#4907)
• [40480f] Support limits on index deserialization loops — useful for tests (#4902)
• [1e4544] Add defaulted SIMDLevel template parameter to handler and scaler types (#4867)
• [abdd37] Add Pixi installation option to docs (#5214)

Changed

• [9d5491] Templatize SIMDResultHandler on SL, mark handle() final (#5223)
• [b63f23] Upgrade GitHub Actions CI from CUDA 12.6 to CUDA 13.2 (#5207)
• [a9f5ba] Move RISC-V fast_scan forwarders to dedicated impl-riscv.cpp (#5216)
• [893271] Validate SVS storage_kind via shared helper at all deserialization read sites (#5204)
• [bc490b] Make IVFPQSearchCagraConfig dtype fields settable from Python (#5191)
• [ffd372] Update pattern for index factory string with SVSVamana (#5201)
• [f37041] Extract Metal shaders to standalone .metal file with kernel wrapper class (#5167)
• [d98364] perf: reserve() to avoid rehashing/reallocation (#5193)
• [4eee3d] Add tolerance and assertion log to test_OIVFPQ (#5188)
• [2322af] faiss: parallelize post-BLAS reduction loop and end_multiple() in result handlers (#5185)
• [967eda] Replace FAISS_ASSERT with FAISS_THROW_FMT for unsupported SVSStorageKind (#5182)
• [715725] Enable verbose mode for TestPyPI publish step (#5181)
• [5e1231] Filter wheel artifacts before publishing to PyPI (#5179)
• [28b2b6] Inline PQ code distance kernels into scanner TUs (#5159)
• [c41405] Enforce minimum training set size for ScalarQuantizer, check nullptr (#5141)
• [44c197] Enable ccache in GitHub Actions cmake builds (#5157)
• [e09de2] Harden compute_query_factors: add preconditions, handle aliasing, remove dead checks
• [6a9936] Hoist per-iteration vector allocations in IndexIVFRaBitQFastScan::compute_LUT
• [c3460c] Convert binary index Hamming callers to dynamic dispatch (#5071)
• [ecac65] Convert hamming.cpp public API to dynamic dispatch (#5070)
• [8d1b96] Enable dynamic dispatch build on Windows (#5127)
• [a25bcb] Rely on cuVS for default values (#5140)
• [062284] Panorama Optimizations (#5041)
• [f4338a] Parameterize existing tests with @for_all_simd_levels (#5134)
• [9d5674] Wire 512-bit QBS kernels into fast scan DD dispatch (#5075)
• [bdf877] facebook-unused-include-check in IndexNNDescent.cpp (#5132)
• [632427] fbcode/faiss/invlists (#5048)
• [e23c66] Replace dispatch_HammingComputer with with_HammingComputer (#5126)
• [ed7e1f] Hoist SIMD dispatch outside loops in 4 call sites (#5074)
• [582246] Remove use of __AMDGCN_WAVEFRONT_SIZE macro with it being deprecated in ROCm 7 (#4619)
• [2dff11] Add braces around single-line control flow statements
• [cc0c08] Hoist SIMD dispatch out of approx_topk hot loops (#5073)
• [18f93f] Replace SSE3 intrinsics in Index2Layer with portable scalar code (#5076)
• [42765a] Optimize RaBitQ FastScan LUT construction for high-dimensional data (#5110)
• [ec5e70] Optimize multibit sign-bit unpacking in RaBitQ FastScan handlers (#5097)
• [3f127e] Thread qb/centered search params through FastScan LUT and handler (#5095)
• [3c19c5] Use std:: qualified math functions to avoid float-to-double promotion (#5088)
• [48d2b0] Replace typedef with using declarations (#5086)
• [e12deb] Remove extra semicolons after inline member function definitions (#5085)
• [5679d3] HNSW Cagra base-level entrypoint sampling parallel (#5068)
• [c4efc4] Remove bare simdlib type aliases from simdlib_dispatch.h (#5069)
• [4a63c9] Convert partitioning to dynamic dispatch (#5062)
• [f226be] Roll out @for_all_simd_levels to key test files (#5061)
• [afd694] fbcode/faiss/impl/HNSW.cpp (#5059)
• [658c43] Replace bare assert() with FAISS_THROW_IF_NOT in core index files (#5052)
• [364749] Re-enable ROCm runner for AMD GPU CI (#4854)
• [faeac8] Update outdated C++17 and Python 2 references (#5045)
• [77e602] Use HTTPS for GitHub links in README (#5043)
• [91843b] Use explicit ISA flags instead of -march/-mtune=sapphirerapids (#5034)
• [acac82] Merge .gitignore updates from faiss-gpu-cu132 into main (#4996)
• [8d8881] Make distance_compute_blas_threshold dimension-aware (#5022)
• [553c78] HNSW SIMD dispatch (#5015)
• [499b48] close the loop on ivf rabitq fast scan filter integration (#5026)
• [aef066] SVS 0.3.0 (#4999)
• [cc3460] reuse unpack buffer in multibit IVFRaBitQFastScan refinement (#5003)
• [d81584] Update Python type stubs: fix class hierarchy and add missing base classes (#5004)
• [d1c432] Convert rabitq_simd.h to runtime SIMD dispatch (#4912)
• [6bca96] Convert distances_fused/ to runtime SIMD dispatch (#4911)
• [04d0d5] facebook-unused-include-check in distances_simd.cpp (#5013)
• [7eb05f] facebook-unused-include-check in partition.cpp (#5012)
• [7e04fa] Update Python type stubs (init.pyi) to match current API (#4998)
• [db0e79] Use omp_get_max_threads() for OpenMP thread count (#4991)
• [9af838] Bump openblas from 0.3.30 to 0.3.32 in conda recipes (#4994)
• [c04891] Enable quieting FAISS warning "inverted lists not stored" (#4964)
• [b2ddb4] Loosen SVS findpackage version requirement (#4969)
• [d8c3d2] Upgrade to C++20 for CUVS (#4881)
• [94f2b5] Rename train() with queries to train_with_queries (#4955)
• [02d609] Upgrade cuVS Version to 26.02 (#4945)
• [55a47c] Upgrade cuVS Version to 26.02, switch various BUCK files over to 26.02 (#4788)
• [0d147a] Make READVECTOR byte limit configurable (#4928)
• [9962fb] HNSW: narrow critical section during add to avoid lock contention (#4915)
• [b1feeb] Convert approx_topk and residual_quantizer_encode_steps to SIMDLevel dynamic dispatch (#4923)
• [58a57e] Re-enable backwards compatibility tests for rabitq now that 1.14.1 released (#4924)
• [5dcef1] Refactor binary HNSW stats to use OpenMP reduction instead of destructor sync (#4910)
• [d8c1a9] move distances implementation into SIMD specific compile units (#4906)
• [ea11f0] Switch all search paths to FastScanCodeScanner and remove make_knn_handler (#4904)
• [57190f] Wire RaBitQ search through FastScanCodeScanner dispatch (#4903)
• [79e8ac] Parallelize compute_residuals in IndexIVFPQ (#4654)
• [eddea2] Extract RaBitQ result handler to impl/fast_scan/ (#4895)
• [734751] Extract PQ4 kernels to includable headers in impl/pq_4bit/ (#4868)
• [bb298a] convert simdlib in distances_simd.cpp (#4884)
• [8d8268] Templatize simdlib types on SIMDLevel (#4866)
• [c74809] Improve exception safety for the 'own_fields'/allocation pattern (#4864)
• [c6cf00] use level but not 0 in neighbor_range(i, level, &begin, &end) (line 198) (#5005)

Fixed

• [cb69d7] Fix Dq=None crash and np.empty nondeterminism in search_preassigned wrappers (#5221)
• [5c92c5] Fix flaky test_hnsw smoke test by increasing efSearch (#5222)
• [8d3cc9] Fix clone_index null return for IndexRowwiseMinMax (#5220)
• [ff1d54] Fix python_unnecessary_generator_set_comprehension issues in faiss/tests/test_fastscan_filter.py (#5215)
• [e23591] Fix FastScan DD regression by threading SIMD level through kernel functions (#5210)
• [172324] Fix IDSelectorBitmap conversion to cuVS bitset (#5211)
• [6bd749] Fix IDSelector leak via SearchParameters.sel setter (#5208)
• [6376bc] Fix CI: (cuVS) conda shards cache lock, (ARM) bump openblas, (SVS) fix LeanVec double-destruction (#5209)
• [4b5a73] Fix python_unnecessary_generator_set_comprehension issues in faiss/.../bench_fw/optimize.py (#5205)
• [6789bf] Validate IndexIDMap id_map and inner-index ntotal consistency to prevent search-time null deref (#5203)
• [f75081] Pin MKL to <2026 to avoid soname mismatch with pytorch (#5192)
• [bb2ce7] Fix 7 broken tests (#5197)
• [f323c0] Fix backward compat test (#5195)
• [23cd94] Reduce BinaryCagra test parameters to prevent CI timeouts (#5194)
• [85fc62] Validate ProductQuantizer M*ksub during deserialization to prevent oversized allocations (#5187)
• [6cef1b] Fix cuVS and ROCm CI conda environment failures (#5180)
• [4a471e] Pin conda <25.7 to fix Windows build (#5176)
• [a9c0d4] Work around GCC 12 miscompilation of AVX2 histogram (#5124)
• [40a8cc] Fix lints associated with early stop facilities for IVF (#5172)
• [1e69d5] Fix peak-memory spike when loading IVF invlists via IO_FLAG_MMAP_IFC (#5122)
• [994222] Fix add_sa_codes silently accepting non-int64 ids, corrupting stored labels (#5171)
• [c5ddbc] c_api: fix IndexShards own_indices getter/setter name mismatch (#5165)
• [8380e2] Fix avx512 unit test (#5161)
• [71448c] Fix issue with svs tests (#5162)
• [6c7044] Validate code_size during deserialization to prevent oversized allocations (#5151)
• [9dbb81] Set own_fields after reading Index2Layer sub-quantizer during deserialization (#5147)
• [01b22b] Fix memory leak warning for unordered_multimap in IndexIVFFlatDedup Python binding (#1667) (#5145)
• [130fc2] Fix flaky test (#5150)
• [dfce6e] Fix OMP exception safety in IndexHNSW search (#5133)
• [c62733] Fix sa_decode offset bug and integer overflow in IndexRefine (#5143)
• [1b4b99] Fix implicit integer precision loss from 64-bit to 32-bit (#5091)
• [b05689] Fix miscellaneous lint warnings (#5093)
• [ed11f2] Fix broken fbcode//faiss/tests:test_contrib - test_checkpoint (test_contrib.TestBigBatchSearch) (#5139)
• [277c53] Fix PCA training bug and memory safety issues in VectorTransform.cpp (#5138)
• [831254] Fix integer type truncation in IDSelectorBatch bloom filter mask (#5136)
• [9cbc8d] Fix integer overflow and unbounded loop in Clustering.cpp (#5130)
• [3c4056] Fix race condition in HNSW::add_with_locks (#5129)
• [6e64c5] Bump GCC pin in faiss-gpu conda recipe to fix AVX2 SIMD miscompilation (#5125)
• [58f6eb] Add per-read byte limit to SVS ReaderStreambuf to prevent OOM from corrupt index data (#5118)
• [817ecf] Validate inverted list entry sizes against deserialization byte limit (#5117)
• [e6be16] Validate VectorTransform dimension consistency during deserialization (#5115)
• [6707ea] Add is_trained check to IndexIVF search and range_search to prevent querying untrained indexes (#5114)
• [349df7] Validate IndexHNSW2Level storage type during deserialization and search (#5113)
• [edd6f3] Reject null quantizer during IVF index deserialization (#5112)
• [27078c] Fix OMP exception safety in IndexNNDescent and IndexNSG search (#5106)
• [99db15] Fix OMP exception safety in IndexFlatCodes search (#5105)
• [40ad64] Fix cuVS build (#5107)
• [57bf47] Fix int/size_t signedness mismatches in HNSW add (#5116)
• [f9f116] Validate k_factor during deserialization to prevent search-time OOM (#5104)
• [2d8232] Set code_size on IVFRaBitQFastScanScanner (#5099)
• [b92309] fix: GPU CAGRA copyFrom host-memory lifetime dependency and add regression test (#4968)
• [796977] Fix IndexBinaryMultiHash::reset() not clearing hash maps and avoid unnecessary copies (#5100)
• [b86066] fix: cagra bug (#4963)
• [91f663] Add value-initialization to uninitialized member variables (#5089)
• [61ed7d] Fix orphaned OpenMP directives in IndexRowwiseMinMax training functions (#5096)
• [740d3a] Fix FastScan for indices without own inverted lists + performance optimizations in RaBitQ handler (#5080)
• [7cc634] Suppress unused static constexpr variable warnings (#5087)
• [aa3ce3] Fix race condition, memory management, debug output, and hashtable lookup in sorting.cpp (#5078)
• [86d337] Validate FastScan M2 consistency during deserialization (#5056)
• [67f066] Validate SVS storage_kind during deserialization (#5055)
• [e89947] Default-initialize SVSStorageKind in IndexSVSVamana (#5054)
• [c1e48a] Add search-path bounds checking to IndexIVF::search1 (#5040)
• [97d257] Fix correctness bugs in NNDescent Nhood copy/move operations and gen_random bounds (#5072)
• [06ef48] Fix uncaught exceptions in IndexIVF::range_search_preassigned OpenMP region (#5053)
• [255250] Fix bugs in NSG search_on_graph and sync_prune (#5063)
• [7047dc] Fix priority_queue constructor inheritance for Apple Clang on macOS (#5065)
• [e0a1ae] Assorted Dynamic Dispatch cleanup fixes (#5060)
• [25296b] Fix type mismatch bug and remove dead code in IndexRowwiseMinMax::sa_decode_impl (#5051)
• [7b2705] Debug manifold crash with NaN (#5025)
• [9d3f97] Fix duplicate words and grammar typos across codebase (#5044)
• [84262f] Fix d_out check bug and add descriptive error messages to VectorTransform assertions (#5047)
• [ef28c6] Fix uncaught exceptions in IndexIVF::search_preassigned OpenMP region (#5037)
• [8008ca] Fix deadly signal in to_svs_metric() for unsupported metrics (#5032)
• [4b8da6] Fix IndexSVSVamana null-deref in deserialize_impl (#5029)
• [ab3da5] Cap ZnSphereCodecRec decode cache size (#5028)
• [d68050] Add missing input validation to IndexFlat::range_search and compute_distance_subset (#5038)
• [c6fa50] Validate IndexIDMap id_map size during deserialization (#5024)
• [408786] Validate AdditiveQuantizer dimension consistency during deserialization (#5023)
• [0ce4df] Fix AVX512_SPR build failure for 512-bit SIMD types (#5030)
• [a308ad] Harden ScalarQuantizer deserialization validation (#5020)
• [ab46dc] Validate qbs at deserialization time in FastScan index formats (#5019)
• [668da4] Validate IndexBinaryHash b <= code_size*8 during deserialization (#5011)
• [87bc9b] Validate IndexFastScan quantizer state during deserialization (#5009)
• [4c3915] Validate codebooks size at deserialization time in read_AdditiveQuantizer (#5008)
• [8c1f3f] read-heap-buffer-overflow (size 32) in float faiss::fvec_norm_L2sqr<>()
• [226ccf] Fix OSX arm64 nightly: bump libopenblas from 0.3.30 to 0.3.32 (#5006)
• [aeb170] large-malloc (15023964028 bytes) in __sanitizer::RunMallocHooks() (#5018)
• [8550f2] Validate VectorTransform dimensions during deserialization (#5007)
• [833d4a] out-of-memory in __gnu_cxx::new_allocator<>::allocate() (#5014)
• [d716be] Fix avx512_result_handlers build on aarch64 (#4965)
• [18c85e] Add deserialization vector limit enforcement for ResidualCoarseQuantizer (#4997)
• [b74f3c] Validate IndexBinaryMultiHash b and nhash*b bounds during deserialization (#4984)
• [b9f496] Add invlists null check to IndexBinaryIVF::search (#4980)
• [ba1981] Add input validation to IndexBinaryHNSW search preconditions (#4979)
• [d26c15] Validate binary index consistency during deserialization (#4978)
• [2c28ef] Validate RaBitQ qb during deserialization (#4983)
• [5a4d62] deadly-signal (vector::_M_default_append) in __clang_call_terminate (#4986)
• [075900] Fix compiler warnings for pip wheel builds (macOS + Windows) (#4989)
• [8a80b3] signal in faiss::ArrayInvertedListsPanorama::ArrayInvertedListsPanorama() (#4987)
• [12675a] out-of-memory in __gnu_cxx::new_allocator<>::allocate() (#4985)
• [0a8e51] Fix HNSW Panorama Perf Bug (#4974)
• [6dbf27] Validate VectorTransform data during deserialization (#4981)
• [3f7938] Fix MSVC link failure: remove __restrict from fvec_madd specializations (#4972)
• [d47b28] deadly-signal (vector::_M_default_append) in __clang_call_terminate (#4982)
• [f851c5] large-malloc (6241124352 bytes) in __sanitizer::RunMallocHooks() (#4977)
• [bd5ed0] Validate inverted lists pointer in IndexIVF operations (#4951)
• [398857] Handle empty index in IndexHNSWCagra base_level_only search (#4950)
• [d647b5] Validate graph index data during deserialization (#4949)
• [e5114c] fix: resolve compiler warnings in binary and miscellaneous Index files (#4940)
• [44f57b] fix: resolve compiler warnings in top-level FAISS source files (#4943)
• [b3a291] fix: resolve compiler warnings in test files (#4944)
• [88efcb] fix: resolve compiler warnings in core base classes and headers (#4933)
• [7acc42] fix: resolve compiler warnings in inverted lists and cppcontrib (#4939)
• [64a212] fix: resolve compiler warnings in IO, result handlers, and misc impl (#4936)
• [a05f13] fix: resolve compiler warnings in PQ, PreTransform, and Refine Index files (#4941)
• [764542] fix: resolve compiler warnings in IVF Index implementations (#4942)
• [f5142b] large-malloc (7046432072 bytes) in __sanitizer::RunMallocHooks() (#4967)
• [4db01e] fix: resolve compiler warnings in graph-based index implementations (#4956)
• [ccc3cd] Validate code vector sizes during index deserialization (#4948)
• [ec30fb] Use BitstringReader in IndexBinaryHash to avoid OOB reads (#4961)
• [fa8ddd] Fix flaky test for Panaorama/regular IVFFlat equivalence (#4954)
• [1e6729] Validate quantizer data sizes during index deserialization (#4947)
• [62a196] fix: resolve compiler warnings in distance and hamming utilities (#4937)
• [645a74] Re add cuvs to cmakelists.txt (#4953)
• [5287db] fix: resolve compiler warnings in graph-based index implementations (#4934)
• [a48e45] fix: resolve compiler warnings in quantizer implementations (#4935)
• [32ca42] fix: resolve compiler warnings in SIMD, sorting, and misc utilities (#4938)
• [52d9fc] Add overflow check in READVECTOR macro (#4946)
• [0f7e65] Enforce memory limit in read_ProductQuantizer (#4930)
• [9ea026] Validate codes vector size in BlockInvertedLists deserialization (#4920)
• [a4385d] Validate n_per_block and block_size in BlockInvertedLists deserialization (#4919)
• [cdb725] Validate id_map size matches ntotal in IndexBinaryIDMap deserialization (#4917)
• [05ca04] Validate per-entry ilsz in read_binary_multi_hash_map (#4916)
• [b02317] Fix MSVC OpenMP build: use idx_t instead of size_t in parallel for loop (#4922)
• [5b83ec] Fix OpenMP critical section contention in IndexBinaryHNSW search (#4909)
• [e96ba2] fix: decouple coarse quantizer from cuvs index reset (#4885)
• [9d6b2e] Additional input validation for index deserialization (#4899)
• [796bdf] Additional binary index input validation (#4898)
• [d0434b] Fix cloning for IVFFlatPanorama (#4887)
• [3e4c10] Validate dynamic_casts during index load (#4883)
• [47e53b] Fix backward compatibility to use latest version (#4855)

Deprecated

• [812010] Remove flaky test (#5169)
• [74dcee] Remove RaBitQStats debug counters from all RaBitQ handlers (#5102)
• [000eec] Delete old pq4_accumulate_loop files (replaced by dispatching.h) (#4905)

Compare: https://github.com/facebookresearch/faiss/compare/v1.14.1...v1.14.2