Latitude
Latitude is an open-source prompt engineering platform designed to help product teams build, evaluate, and deploy AI models efficiently. It allows users to import and manage prompts at scale, refine them with real or synthetic data, and track the performance of AI models using LLM-as-judge or human-in-the-loop evaluations. With powerful tools for dataset management and automatic logging, Latitude simplifies the process of fine-tuning models and improving AI performance, making it an essential platform for businesses focused on deploying high-quality AI applications.
Learn more
ChainForge
ChainForge is an open-source visual programming environment designed for prompt engineering and large language model evaluation. It enables users to assess the robustness of prompts and text-generation models beyond anecdotal evidence. Simultaneously test prompt ideas and variations across multiple LLMs to identify the most effective combinations. Evaluate response quality across different prompts, models, and settings to select the optimal configuration for specific use cases. Set up evaluation metrics and visualize results across prompts, parameters, models, and settings, facilitating data-driven decision-making. Manage multiple conversations simultaneously, template follow-up messages, and inspect outputs at each turn to refine interactions. ChainForge supports various model providers, including OpenAI, HuggingFace, Anthropic, Google PaLM2, Azure OpenAI endpoints, and locally hosted models like Alpaca and Llama. Users can adjust model settings and utilize visualization nodes.
Learn more
Langfuse
Langfuse is an open source LLM engineering platform to help teams collaboratively debug, analyze and iterate on their LLM Applications.
Observability: Instrument your app and start ingesting traces to Langfuse
Langfuse UI: Inspect and debug complex logs and user sessions
Prompts: Manage, version and deploy prompts from within Langfuse
Analytics: Track metrics (LLM cost, latency, quality) and gain insights from dashboards & data exports
Evals: Collect and calculate scores for your LLM completions
Experiments: Track and test app behavior before deploying a new version
Why Langfuse?
- Open source
- Model and framework agnostic
- Built for production
- Incrementally adoptable - start with a single LLM call or integration, then expand to full tracing of complex chains/agents
- Use GET API to build downstream use cases and export data
Learn more
LayerLens
LayerLens is an independent AI model evaluation platform for understanding how models perform through verified results across benchmarks, prompt-level results, agentic benchmarks, and audit-ready comparisons across vendors. It helps teams compare more than 200 AI models side by side, with transparent benchmarks, model comparison tools, and consistent evaluation methods for accuracy, latency, behavior, and real-world applicability. LayerLens is built for deep model analysis through Spaces, where teams can group benchmarks and evaluations, explore task strengths, and track performance patterns in context. It supports continuous evaluation by running ongoing evals across model versions, prompt changes, judge updates, and live traces, helping teams detect quality regressions, drift, silent failures, contamination, and policy issues before they affect production.
Learn more
