Open Source Go Vector Search Engines

Weaviate in a nutshell: Weaviate is a vector search engine and vector database. Weaviate uses machine learning to vectorize and store data, and to find answers to natural language queries. With Weaviate you can also bring your custom ML models to production scale. Weaviate in detail: Weaviate is a low-latency vector search engine with out-of-the-box support for different media types (text, images, etc.). It offers Semantic Search, Question-Answer-Extraction, Classification, Customizable Models (PyTorch/TensorFlow/Keras), and more. Built from scratch in Go, Weaviate stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance of a cloud-native database, all accessible through GraphQL, REST, and various language clients.

Milvus is an open-source vector database built to power embedding similarity search and AI applications. Milvus makes unstructured data search more accessible, and provides a consistent user experience regardless of the deployment environment. Milvus 2.0 is a cloud-native vector database with storage and computation separated by design. All components in this refactored version of Milvus are stateless to enhance elasticity and flexibility. Average latency measured in milliseconds on trillion vector datasets. Rich APIs designed for data science workflows. Consistent user experience across laptop, local cluster, and cloud. Embed real-time search and analytics into virtually any application. Milvus’ built-in replication and failover/failback features ensure data and applications can maintain business continuity in the event of a disruption. Component-level scalability makes it possible to scale up and down on demand.

Vald is a highly scalable distributed fast approximate nearest neighbor dense vector search engine. Vald is designed and implemented based on the Cloud-Native architecture. It uses the fastest ANN Algorithm NGT to search for neighbors. Vald has automatic vector indexing and index backup, and horizontal scaling which is made for searching from billions of feature vector data. Vald is easy to use, feature-rich and highly customizable as you needed. Usually, the graph requires locking during indexing, which causes stop-the-world. But Vald uses distributed index graphs so it continues to work during indexing. Vald implements it's own highly customizable Ingress/Egress filter. Which can be configured to fit the gRPC interface. Horizontal scalable on memory and cpu for your demand. Vald supports to auto backup feature using Object Storage or Persistent Volume which enables disaster recovery.

Vearch is the vector search infrastructure for deep learning and AI applications. Vearch is a distributed vector storage and retrieval system which can be easily extended to billions scale. Vearch implements a high-performance, lockless real-time vector indexing subsystem that utilizes various optimization techniques to support millisecond vector update and retrieval. End-to-end one-click deployment. Through the module of the plugin, a complete default visual search system can be deployed just with one click. Otherwise, you can easily customize your own image, video, or text feature extraction algorithm plugin. This GIF provides a clear demonstration of the project vearch usage and its internal structure. The use of vearch is mainly divided into three steps. Firstly, create DB and Space, then import your data, and finally, you can search on your own dataset.

A lightweight distributed vector search engine based on Vald codebase. Vald is an awesome highly scalable distributed vector search engine works on Kubernetes. It has great features such as file-based backup, and metrics-based ordering of Agents. Also, Vald is highly configurable using YAML files. It works without Kubernetes, single binary (less than 30MB), easy to run (can be configured by command-line options), and consists of Agent and Server. alvd has almost the same features as Vald's gateway-lb + discoverer and agent-ngt. alvd depends on Vald codebase, the files came from Vald (such as internal, pkg/vald. They are downloaded when running make command.) are excluded from my license and ownership.