Search Results for "python programming language"

Simple, Pythonic building blocks to evaluate LLM applications.

LangKit is an open-source text metrics toolkit for monitoring language models. It offers an array of methods for extracting relevant signals from the input and/or output text, which are compatible with the open-source data logging library whylogs. Productionizing language models, including LLMs, comes with a range of risks due to the infinite amount of input combinations, which can elicit an infinite amount of outputs. The unstructured nature of text poses a challenge in the ML observability...

The repo contains standard OpenTelemetry instrumentations for LLM providers and Vector DBs, as well as a Traceloop SDK that makes it easy to get started with OpenLLMetry, while still outputting standard OpenTelemetry data that can be connected to your observability stack. If you already have OpenTelemetry instrumented, you can just add any of our instrumentations directly.

Langfuse is building open-source observability and analytics for LLM apps. Observability: Explore and debug complex logs & traces in a visual UI. Analytics: Improve performance of LLM apps. In particular, get a view on costs, latency and response quality using intuitive dashboards.