Pinecone Rerank v0
Pinecone Rerank V0 is a cross-encoder model optimized for precision in reranking tasks, enhancing enterprise search and retrieval-augmented generation (RAG) systems. It processes queries and documents together to capture fine-grained relevance, assigning a relevance score from 0 to 1 for each query-document pair. The model's maximum context length is set to 512 tokens to preserve ranking quality. Evaluations on the BEIR benchmark demonstrated that Pinecone Rerank V0 achieved the highest average NDCG@10, outperforming other models on 6 out of 12 datasets. For instance, it showed up to a 60% boost on the Fever dataset compared to Google Semantic Ranker and over 40% on the Climate-Fever dataset relative to cohere-v3-multilingual or voyageai-rerank-2. The model is accessible through Pinecone Inference and is available to all users in public preview.
Learn more
IBM Watson Discovery
Find specific answers and trends from documents and websites using search powered by AI. Watson Discovery is AI-powered search and text-analytics that uses innovative, market-leading natural language processing to understand your industry’s unique language. It finds answers in your content fast and uncovers meaningful business insights from your documents, webpages and big data, cutting research time by more than 75%. Semantic search is much more than keyword search. Unlike traditional search engines, when you ask a question, Watson Discovery adds context to the answer. It quickly combs through content in your connected data sources, pinpoints the most relevant passage and provides the source documents or webpage. A next-level search experience with natural language processing that makes all necessary information easily accessible. Use machine learning to visually label text, tables and images, while surfacing the most relevant results.
Learn more
RankGPT
RankGPT is a Python toolkit designed to explore the use of generative Large Language Models (LLMs) like ChatGPT and GPT-4 for relevance ranking in Information Retrieval (IR). It introduces methods such as instructional permutation generation and a sliding window strategy to enable LLMs to effectively rerank documents. It supports various LLMs, including GPT-3.5, GPT-4, Claude, Cohere, and Llama2 via LiteLLM. RankGPT provides modules for retrieval, reranking, evaluation, and response analysis, facilitating end-to-end workflows. It includes a module for detailed analysis of input prompts and LLM responses, addressing reliability concerns with LLM APIs and non-deterministic behavior in Mixture-of-Experts (MoE) models. The toolkit supports various backends, including SGLang and TensorRT-LLM, and is compatible with a wide range of LLMs. RankGPT's Model Zoo includes models like LiT5 and MonoT5, hosted on Hugging Face.
Learn more
Azure AI Search
Deliver high-quality responses with a vector database built for advanced retrieval augmented generation (RAG) and modern search. Focus on exponential growth with an enterprise-ready vector database that comes with security, compliance, and responsible AI practices built in. Build better applications with sophisticated retrieval strategies backed by decades of research and customer validation. Quickly deploy your generative AI app with seamless platform and data integrations for data sources, AI models, and frameworks. Automatically upload data from a wide range of supported Azure and third-party sources. Streamline vector data processing with built-in extraction, chunking, enrichment, and vectorization, all in one flow. Support for multivector, hybrid, multilingual, and metadata filtering. Move beyond vector-only search with keyword match scoring, reranking, geospatial search, and autocomplete.
Learn more
