Audience
Engineering teams and founders shipping AI features, who want to know when an AI call goes wrong.
About evalship
evalship tests, monitors and optimizes every AI call your product makes. Setup is one line for your coding agent: it writes evals for the prompts that have none and finds the flaws in the evals you have, each with fix instructions. Test: on every pull request, evalship says which evals the change broke and why, flags AI calls that no eval tests, and can block the merge. Monitor: evalship checks every AI call your app makes in production and emails you the moment one fails, overpromises, shares private data or breaks a rule your evals test, with the trace and a line for your coding agent to fix it. Optimize: your coding agent tries prompt and model changes on your machine, and evalship says which win beyond chance and by how much. Your evals keep running with your own keys, on your machine, in GitHub Actions or in GitLab CI. evalship reads results from pytest, Vitest, Jest, promptfoo, Langfuse, Braintrust or LangSmith, and works without an eval platform. Public repositories are free.