Braintrust Integrations

Reflexio is an AI-agent learning platform for developers building and operating agents. It ingests agent interactions, execution traces, tool calls, and user corrections through an SDK, REST API, CLI, or supported tracing-platform connections, then extracts recurring failures into reusable playbooks and user profiles. At runtime, an agent can retrieve relevant learned guidance before responding. The Hosted Enterprise portal lets teams inspect, edit, approve, reject, search, and evaluate learnings. Reflexio helps agents apply corrections across interactions without changing model weights. Teams can use the managed cloud service, configure their own model-provider keys, or self-host in AWS, Google Cloud, or Azure. Enterprise trace imports are available from Braintrust, Langfuse Cloud, and LangSmith Cloud; Reflexio also provides a mem0 wrapper and an OpenClaw integration.

evalship helps engineering teams test LLM-powered software through evaluation-suite audits and pull-request reviews. It checks scorers and judges for flaws, identifies missing coverage and evals that may pass incorrectly, and provides findings with code references and suggested fixes. Setup can help a coding agent configure GitHub Actions and open a setup pull request. On later pull requests, evalship compares results with the default branch, flags regressions, flaky evals, and changed prompts or model calls without evaluation coverage, and can provide a merge-gating status check. It reads results from supported test runners and LLM evaluation platforms. Evaluations run in the customer's GitHub Actions with their own model API keys; evalship reviews results rather than executing evaluations. Teams can review evaluation history and receive weekly reports.

Overmind is a model training and deployment platform for teams building AI agents. It ingests production traces through OpenTelemetry and creates datasets from trace data, file uploads, and connectors for Langfuse, LangSmith, Braintrust, and Galileo. Teams can inspect agent context and runtime behavior, prepare and score datasets, evaluate prompts and models against their own examples, and optimize agent behavior through scored changes. The platform supports LoRA and full fine-tuning of open-weight models, benchmarking with configured evaluators, and deployment through an OpenAI-compatible endpoint; trained weights can also be downloaded. Users can work in the hosted console or run the platform on their own infrastructure with Docker Compose. It is designed for AI engineering and product teams operating agents in production that want to test, improve, or specialize models using their own data.