BentoML
Serve your ML model in any cloud in minutes. Unified model packaging format enabling both online and offline serving on any platform. 100x the throughput of your regular flask-based model server, thanks to our advanced micro-batching mechanism. Deliver high-quality prediction services that speak the DevOps language and integrate perfectly with common infrastructure tools. Unified format for deployment. High-performance model serving. DevOps best practices baked in. The service uses the BERT model trained with the TensorFlow framework to predict movie reviews' sentiment. DevOps-free BentoML workflow, from prediction service registry, deployment automation, to endpoint monitoring, all configured automatically for your team. A solid foundation for running serious ML workloads in production. Keep all your team's models, deployments, and changes highly visible and control access via SSO, RBAC, client authentication, and auditing logs.
Learn more
Pinecone
The AI Knowledge Platform.
The Pinecone Database, Inference, and Assistant make building high-performance vector search apps easy. Developer-friendly, fully managed, and easily scalable without infrastructure hassles.
Once you have vector embeddings, manage and search through them in Pinecone to power semantic search, recommenders, and other applications that rely on relevant information retrieval.
Ultra-low query latency, even with billions of items. Give users a great experience. Live index updates when you add, edit, or delete data. Your data is ready right away. Combine vector search with metadata filters for more relevant and faster results.
Launch, use, and scale your vector search service with our easy API, without worrying about infrastructure or algorithms. We'll keep it running smoothly and securely.
Learn more
RankLLM
RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking. It offers a suite of rerankers, pointwise models like MonoT5, pairwise models like DuoT5, and listwise models compatible with vLLM, SGLang, or TensorRT-LLM. Additionally, it supports RankGPT and RankGemini variants, which are proprietary listwise rerankers. It includes modules for retrieval, reranking, evaluation, and response analysis, facilitating end-to-end workflows. RankLLM integrates with Pyserini for retrieval and provides integrated evaluation for multi-stage pipelines. It also includes a module for detailed analysis of input prompts and LLM responses, addressing reliability concerns with LLM APIs and non-deterministic behavior in Mixture-of-Experts (MoE) models. The toolkit supports various backends, including SGLang and TensorRT-LLM, and is compatible with a wide range of LLMs.
Learn more
TILDE
TILDE (Term Independent Likelihood moDEl) is a passage re-ranking and expansion framework built on BERT, designed to enhance retrieval performance by combining sparse term matching with deep contextual representations. The original TILDE model pre-computes term weights across the entire BERT vocabulary, which can lead to large index sizes. To address this, TILDEv2 introduces a more efficient approach by computing term weights only for terms present in expanded passages, resulting in indexes that are 99% smaller than those of the original TILDE. This efficiency is achieved by leveraging TILDE as a passage expansion model, where passages are expanded using top-k terms (e.g., top 200) to enrich their content. It provides scripts for indexing collections, re-ranking BM25 results, and training models using datasets like MS MARCO.
Learn more
