Search Results for "prometheus"

Prometheus-Eval is an open-source framework designed to evaluate the outputs of large language models using specialized evaluator models known as Prometheus. The project provides tools, datasets, and scripts that allow developers and researchers to measure the quality of LLM responses through automated scoring rather than relying solely on human evaluators.

...Requests can be distributed across vLLM, NVIDIA NIM, Ollama, OpenRouter, and other compatible endpoints. Routing strategies include random splits, LLM classification, signal-driven stage routing, escalation, and custom algorithms. Prometheus metrics track requests, errors, latency, tokens, and routing overhead. Developers can run it as a standalone server, launch coding agents through it, or embed its routing logic in Rust applications. The project is currently pre-alpha and explicitly marked experimental rather than production-ready.

Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API.

Jina is a framework that empowers anyone to build cross-modal and multi-modal applications on the cloud. It uplifts a PoC into a production-ready service. Jina handles the infrastructure complexity, making advanced solution engineering and cloud-native technologies accessible to every developer. Build applications that deliver fresh insights from multiple data types such as text, image, audio, video, 3D mesh, PDF with Jina AI’s DocArray. Polyglot gateway that supports gRPC, Websockets, HTTP,...