DownloadAnnoy (Approximate Nearest Neighbors Oh Yeah) is a C++ library with Python bindings to search for points in space that are close to a given query point. It also creates large read-only file-based data structures that are mmapped into memory so that many processes may share the same data. There are some other libraries to do nearest neighbor search. Annoy is almost as fast as the fastest libraries, (see below), but there is actually another feature that really sets Annoy apart: it has the ability to use static files as indexes. In particular, this means you can share index across processes. Annoy also decouples creating indexes from loading them, so you can pass around indexes as files and map them into memory quickly. Another nice thing of Annoy is that it tries to minimize memory footprint so the indexes are quite small.
Features
- If you want to find nearest neighbors and you have many CPU's, you only need to build the index once
- You can also pass around and distribute static files to use in production environment
- Any process will be able to load (mmap) the index into memory and will be able to do lookups immediately
- After running matrix factorization algorithms, every user/item can be represented as a vector in f-dimensional space
- Build index on disk to enable indexing big datasets that won't fit into memory
- Lets you share memory between multiple processes
Project Samples
