Athina AI
Monitor your LLMs in production, and discover and fix hallucinations, accuracy, and quality-related errors with LLM outputs seamlessly. Evaluate your outputs for hallucinations, misinformation, quality issues, and other bad outputs. Configurable for any LLM use case. Segment your data to analyze your cost, accuracy, response times, model usage, and feedback in depth. Search, sort, and filter through your inference calls, and trace through your queries, retrievals, prompts, responses, and feedback metrics to debug generations. Explore your conversations, understand what your users are talking about and how they feel, and learn which conversations ended badly. Compare your performance metrics across different models and prompts. Our insights will help you find the best-performing model for every use case. Our evaluators use your data, configurations, and feedback to get better and analyze the outputs better.
Learn more
Neum AI
No one wants their AI to respond with out-of-date information to a customer. Neum AI helps companies have accurate and up-to-date context in their AI applications. Use built-in connectors for data sources like Amazon S3 and Azure Blob Storage, vector stores like Pinecone and Weaviate to set up your data pipelines in minutes. Supercharge your data pipeline by transforming and embedding your data with built-in connectors for embedding models like OpenAI and Replicate, and serverless functions like Azure Functions and AWS Lambda. Leverage role-based access controls to make sure only the right people can access specific vectors. Bring your own embedding models, vector stores and sources. Ask us about how you can even run Neum AI in your own cloud.
Learn more
Martian
By using the best-performing model for each request, we can achieve higher performance than any single model. Martian outperforms GPT-4 across OpenAI's evals (open/evals). We turn opaque black boxes into interpretable representations. Our router is the first tool built on top of our model mapping method. We are developing many other applications of model mapping including turning transformers from indecipherable matrices into human-readable programs. If a company experiences an outage or high latency period, automatically reroute to other providers so your customers never experience any issues. Determine how much you could save by using the Martian Model Router with our interactive cost calculator. Input your number of users, tokens per session, and sessions per month, and specify your cost/quality tradeoff.
Learn more
Vellum AI
Bring LLM-powered features to production with tools for prompt engineering, semantic search, version control, quantitative testing, and performance monitoring. Compatible across all major LLM providers. Quickly develop an MVP by experimenting with different prompts, parameters, and even LLM providers to quickly arrive at the best configuration for your use case. Vellum acts as a low-latency, highly reliable proxy to LLM providers, allowing you to make version-controlled changes to your prompts – no code changes needed. Vellum collects model inputs, outputs, and user feedback. This data is used to build up valuable testing datasets that can be used to validate future changes before they go live. Dynamically include company-specific context in your prompts without managing your own semantic search infra.
Learn more