RankLLMCastorini
|
||||||
Related Products
|
||||||
About
RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking. It offers a suite of rerankers, pointwise models like MonoT5, pairwise models like DuoT5, and listwise models compatible with vLLM, SGLang, or TensorRT-LLM. Additionally, it supports RankGPT and RankGemini variants, which are proprietary listwise rerankers. It includes modules for retrieval, reranking, evaluation, and response analysis, facilitating end-to-end workflows. RankLLM integrates with Pyserini for retrieval and provides integrated evaluation for multi-stage pipelines. It also includes a module for detailed analysis of input prompts and LLM responses, addressing reliability concerns with LLM APIs and non-deterministic behavior in Mixture-of-Experts (MoE) models. The toolkit supports various backends, including SGLang and TensorRT-LLM, and is compatible with a wide range of LLMs.
|
About
vLLM is a high-performance library designed to facilitate efficient inference and serving of Large Language Models (LLMs). Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project with contributions from both academia and industry. It offers state-of-the-art serving throughput by efficiently managing attention key and value memory through its PagedAttention mechanism. It supports continuous batching of incoming requests and utilizes optimized CUDA kernels, including integration with FlashAttention and FlashInfer, to enhance model execution speed. Additionally, vLLM provides quantization support for GPTQ, AWQ, INT4, INT8, and FP8, as well as speculative decoding capabilities. Users benefit from seamless integration with popular Hugging Face models, support for various decoding algorithms such as parallel sampling and beam search, and compatibility with NVIDIA GPUs, AMD CPUs and GPUs, Intel CPUs, and more.
|
|||||
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
|||||
Audience
Academic researchers and developers seeking a solution offering tools for implementing and evaluating listwise reranking with large language models
|
Audience
AI infrastructure engineers looking for a solution to optimize the deployment and serving of large-scale language models in production environments
|
|||||
Support
Phone Support
24/7 Live Support
Online
|
Support
Phone Support
24/7 Live Support
Online
|
|||||
API
Offers API
|
API
Offers API
|
|||||
Screenshots and Videos |
Screenshots and Videos |
|||||
Pricing
Free
Free Version
Free Trial
|
Pricing
No information available.
Free Version
Free Trial
|
|||||
Reviews/
|
Reviews/
|
|||||
Training
Documentation
Webinars
Live Online
In Person
|
Training
Documentation
Webinars
Live Online
In Person
|
|||||
Company InformationCastorini
Canada
github.com/castorini/rank_llm/
|
Company InformationvLLM
United States
vllm.ai
|
|||||
Alternatives |
Alternatives |
|||||
|
|
||||||
|
|
|
|||||
|
|
|
|||||
Categories |
Categories |
|||||
Integrations
OpenAI
Database Mart
Docker
Gemini
Gemini Enterprise
Hugging Face
KServe
Kubernetes
Llama
Mistral AI
|
Integrations
OpenAI
Database Mart
Docker
Gemini
Gemini Enterprise
Hugging Face
KServe
Kubernetes
Llama
Mistral AI
|
|||||
|
|
|