Braintrust Integrations

evalship helps engineering teams test LLM-powered software with a review of their evals at setup and a report on every pull request. The review checks scorers, judges and prompts for flaws, such as a judge that scores 1.0 when its call fails, and gives each flaw a fix with code references. Setup is a prompt for your coding agent: it configures GitHub Actions and opens a setup pull request. On later pull requests, evalship compares results with the default branch, reports the evals a change really broke and the likely cause, sets flaky evals aside, and flags changed prompts or LLM calls that no eval exercises. Its check can block a merge that breaks an eval. It reads results from pytest, Vitest, Jest, promptfoo, Langfuse, Braintrust and LangSmith, with no SDK to add. Evals run in the customer's own GitHub Actions with their own model API keys; evalship only reads the results. Teams get a dashboard with each eval's history and a weekly report by email. Public repositories are free.

Reflexio is an AI-agent learning platform for developers building and operating agents. It ingests agent interactions, execution traces, tool calls, and user corrections through an SDK, REST API, CLI, or supported tracing-platform connections, then extracts recurring failures into reusable playbooks and user profiles. At runtime, an agent can retrieve relevant learned guidance before responding. The Hosted Enterprise portal lets teams inspect, edit, approve, reject, search, and evaluate learnings. Reflexio helps agents apply corrections across interactions without changing model weights. Teams can use the managed cloud service, configure their own model-provider keys, or self-host in AWS, Google Cloud, or Azure. Enterprise trace imports are available from Braintrust, Langfuse Cloud, and LangSmith Cloud; Reflexio also provides a mem0 wrapper and an OpenClaw integration.

Overmind is a model training and deployment platform for teams building AI agents. It ingests production traces through OpenTelemetry and creates datasets from trace data, file uploads, and connectors for Langfuse, LangSmith, Braintrust, and Galileo. Teams can inspect agent context and runtime behavior, prepare and score datasets, evaluate prompts and models against their own examples, and optimize agent behavior through scored changes. The platform supports LoRA and full fine-tuning of open-weight models, benchmarking with configured evaluators, and deployment through an OpenAI-compatible endpoint; trained weights can also be downloaded. Users can work in the hosted console or run the platform on their own infrastructure with Docker Compose. It is designed for AI engineering and product teams operating agents in production that want to test, improve, or specialize models using their own data.