Alternatives to EvalsOne
Compare EvalsOne alternatives for your business or organization using the curated list below. SourceForge ranks the best alternatives to EvalsOne in 2026. Compare features, ratings, user reviews, pricing, and more from EvalsOne competitors and alternatives in order to make an informed decision for your business.
-
1
Agenta
Agenta
Agenta is an open-source LLMOps platform designed to help teams build reliable AI applications with integrated prompt management, evaluation workflows, and system observability. It centralizes all prompts, experiments, traces, and evaluations into one structured hub, eliminating scattered workflows across Slack, spreadsheets, and emails. With Agenta, teams can iterate on prompts collaboratively, compare models side-by-side, and maintain full version history for every change. Its evaluation tools replace guesswork with automated testing, LLM-as-a-judge, human annotation, and intermediate-step analysis. Observability features allow developers to trace failures, annotate logs, convert traces into tests, and monitor performance regressions in real time. Agenta helps AI teams transition from siloed experimentation to a unified, efficient LLMOps workflow for shipping more reliable agents and AI products.Starting Price: Free -
2
DeepEval
Confident AI
DeepEval is a simple-to-use, open source LLM evaluation framework, for evaluating and testing large-language model systems. It is similar to Pytest but specialized for unit testing LLM outputs. DeepEval incorporates the latest research to evaluate LLM outputs based on metrics such as G-Eval, hallucination, answer relevancy, RAGAS, etc., which uses LLMs and various other NLP models that run locally on your machine for evaluation. Whether your application is implemented via RAG or fine-tuning, LangChain, or LlamaIndex, DeepEval has you covered. With it, you can easily determine the optimal hyperparameters to improve your RAG pipeline, prevent prompt drifting, or even transition from OpenAI to hosting your own Llama2 with confidence. The framework supports synthetic dataset generation with advanced evolution techniques and integrates seamlessly with popular frameworks, allowing for efficient benchmarking and optimization of LLM systems.Starting Price: Free -
3
Maxim
Maxim
Maxim is an agent simulation, evaluation, and observability platform that empowers modern AI teams to deploy agents with quality, reliability, and speed. Maxim's end-to-end evaluation and data management stack covers every stage of the AI lifecycle, from prompt engineering to pre & post release testing and observability, data-set creation & management, and fine-tuning. Use Maxim to simulate and test your multi-turn workflows on a wide variety of scenarios and across different user personas before taking your application to production. Features: Agent Simulation Agent Evaluation Prompt Playground Logging/Tracing Workflows Custom Evaluators- AI, Programmatic and Statistical Dataset Curation Human-in-the-loop Use Case: Simulate and test AI agents Evals for agentic workflows: pre and post-release Tracing and debugging multi-agent workflows Real-time alerts on performance and quality Creating robust datasets for evals and fine-tuning Human-in-the-loop workflowsStarting Price: $29/seat/month -
4
TruLens
TruLens
TruLens is an open-source Python library designed to systematically evaluate and track Large Language Model (LLM) applications. It provides fine-grained instrumentation, feedback functions, and a user interface to compare and iterate on app versions, facilitating rapid development and improvement of LLM-based applications. Programmatic tools that assess the quality of inputs, outputs, and intermediate results from LLM applications, enabling scalable evaluation. Fine-grained, stack-agnostic instrumentation and comprehensive evaluations help identify failure modes and systematically iterate to improve applications. An easy-to-use interface that allows developers to compare different versions of their applications, facilitating informed decision-making and optimization. TruLens supports various use cases, including question-answering, summarization, retrieval-augmented generation, and agent-based applications.Starting Price: Free -
5
Orbit Eval
Turning Point HR Solutions Ltd
Orbit Eval is part of the Orbit Software Suite and is analytical job evaluation software. Job evaluation is a consistent & systematic process for defining the relative size or ranking of jobs within an organisation, by applying a consistent set of criteria to job roles. Analytical schemes offer a higher degree of rigour and objectivity. They enable a systematic approach to be applied providing a rationale as to why jobs are ranked differently. Application of the same method throughout the evaluation ensures consistency while minimising subjectivity and gender bias Orbit Eval is easy to use, very transparent and ensures consistency. The tool has been designed to be ‘owned’ by the organisation & requires minimal amounts of training. . It is hosted in the cloud with access permission levels. You can also input your current paper based scheme into the web-based data storage facility in Orbit Eval© to accommodate various systems including: NJC, GLPC & others. -
6
Trusys AI
Trusys
Trusys.ai is a unified AI assurance platform that helps organizations evaluate, secure, monitor, and govern artificial intelligence systems across their full lifecycle, from early testing to production deployment. It offers a suite of tools: TRU SCOUT for automated security and compliance scanning against global standards and adversarial vulnerabilities, TRU EVAL for comprehensive functional evaluation of AI applications (text, voice, image, and agent) assessing accuracy, bias, and safety, and TRU PULSE for real-time production monitoring with alerts for drift, performance degradation, policy violations, and anomalies. It provides end-to-end observability and performance tracking, enabling teams to catch unreliable output, compliance gaps, and production issues early. Trusys supports model-agnostic evaluation with a no-code, intuitive interface and integrates human-in-the-loop reviews and custom scoring metrics to blend expert judgment with automated metrics.Starting Price: Free -
7
Adaline
Adaline
Iterate quickly and ship confidently. Confidently ship by evaluating your prompts with a suite of evals like context recall, llm-rubric (LLM as a judge), latency, and more. Let us handle intelligent caching and complex implementations to save you time and money. Quickly iterate on your prompts in a collaborative playground that supports all the major providers, variables, automatic versioning, and more. Easily build datasets from real data using Logs, upload your own as a CSV, or collaboratively build and edit within your Adaline workspace. Track usage, latency, and other metrics to monitor the health of your LLMs and the performance of your prompts using our APIs. Continuously evaluate your completions in production, see how your users are using your prompts, and create datasets by sending logs using our APIs. The single platform to iterate, evaluate, and monitor LLMs. Easily rollbacks if your performance regresses in production, and see how your team iterated the prompt. -
8
Confident AI
Confident AI
Confident AI offers an open-source package called DeepEval that enables engineers to evaluate or "unit test" their LLM applications' outputs. Confident AI is our commercial offering and it allows you to log and share evaluation results within your org, centralize your datasets used for evaluation, debug unsatisfactory evaluation results, and run evaluations in production throughout the lifetime of your LLM application. We offer 10+ default metrics for engineers to plug and use.Starting Price: $39/month -
9
Revolution FTO
Wayne Enterprises
Documenting the training of new officers is serious business. Liability is generally determined by training or the lack of it. Our police and sheriff FTO evaluation software was created by sworn officers having over 23 years of experience in managing FTOs and training new officers. This software is web-based and allows your training officers to document all daily and monthly activities of your newer officers. Through an annual contract with your agency, we can provide 24/7 phone, web, and onsite technical support. You will get direct assistance from a developer of the software. Create evaluations in half the time. FTO's can only change the evals they create. Finalization prevents changes in evaluations. Use from any computer inside the department. Use dailies to create monthlies, trainees can log on and sign evals without FTO. Chronological one-button approval of evaluations. Create statistical reports and track the effectiveness of police academies. -
10
EvalExpert
AlgoDriven
EvalExpert empowers dealerships by giving them the vehicle appraisal tools to make data-driven decisions about used cars. We offer a fully automated, single platform for vehicle appraisal, price guidance and analysis. Our industry leading data, partnered with proprietary algorithms; help reduce paperwork, eliminate mistakes of manual entry, improve productivity & provide great service to your customers. Using our propriety algorithms and industry leading data, EvalExpert streamlines the appraisal process with our easy to use, 3 step appraisal process - scan the vehicles registration or VIN, take photos, enter current information & condition details - done! EvalExpert’s Web Dashboard instantly syncs all your dealerships evaluations from any device. It provides overview statistics for the dealership and sales team with the most advanced reporting tools available in the market. -
11
FinetuneDB
FinetuneDB
Capture production data, evaluate outputs collaboratively, and fine-tune your LLM's performance. Know exactly what goes on in production with an in-depth log overview. Collaborate with product managers, domain experts and engineers to build reliable model outputs. Track AI metrics such as speed, quality scores, and token usage. Copilot automates evaluations and model improvements for your use case. Create, manage, and optimize prompts to achieve precise and relevant interactions between users and AI models. Compare foundation models, and fine-tuned versions to improve prompt performance and save tokens. Collaborate with your team to build a proprietary fine-tuning dataset for your AI models. Build custom fine-tuning datasets to optimize model performance for specific use cases. -
12
viEval
viGlobal
Evaluate every professional's performance with ease, efficiency & precision. Your annual review process doesn't have to be time-consuming. With our help, simplify any number of evaluations into one easy annual workflow. We understand the results your professional services firm needs to capture, including performance on projects and client work. viEval is the best-in-class tool for performance evaluation of professional work. All client work and hours are automatically pulled in from billing systems, so evaluations can be completed quickly and easily. We build high-performance cultures with 360-degree annual evaluation and integration with real-time feedback for continuous improvement. Our system can be easily customized for any role, department, or practice area. Create a performance management process of any complexity with our intelligent process builder. Use our pre-built templates for professional services firms or design your own process to capture precise feedback. -
13
Valid Eval
Valid Eval
Complex group deliberations don't have to be painful. Whether you're tasked with ranking hundreds of competing proposals, judging a dozen live pitches, or managing a multi-phase innovation program, there's an easier way. A better way. Valid Eval is an online evaluation system for organizations that make and defend tough decisions. It's a secure SaaS platform that works efficiently at virtually any scale so you can involve as many applicants, subjects, domain experts, and judges as it takes to do the job right. Combining best practices from the learning sciences and systems engineering, Valid Eval delivers defensible, data driven results and provides robust reporting tools that help you measure and monitor performance and demonstrate mission alignment. Best of all, it provides an unprecedented degree of transparency that promotes accountability and builds trust in the process. -
14
Prompt flow
Microsoft
Prompt Flow is a suite of development tools designed to streamline the end-to-end development cycle of LLM-based AI applications, from ideation, prototyping, testing, and evaluation to production deployment and monitoring. It makes prompt engineering much easier and enables you to build LLM apps with production quality. With Prompt Flow, you can create flows that link LLMs, prompts, Python code, and other tools together in an executable workflow. It allows for debugging and iteration of flows, especially tracing interactions with LLMs with ease. You can evaluate your flows, calculate quality and performance metrics with larger datasets, and integrate the testing and evaluation into your CI/CD system to ensure quality. Deployment of flows to the serving platform of your choice or integration into your app’s code base is made easy. Additionally, collaboration with your team is facilitated by leveraging the cloud version of Prompt Flow in Azure AI. -
15
LayerLens
LayerLens
LayerLens is an independent AI model evaluation platform for understanding how models perform through verified results across benchmarks, prompt-level results, agentic benchmarks, and audit-ready comparisons across vendors. It helps teams compare more than 200 AI models side by side, with transparent benchmarks, model comparison tools, and consistent evaluation methods for accuracy, latency, behavior, and real-world applicability. LayerLens is built for deep model analysis through Spaces, where teams can group benchmarks and evaluations, explore task strengths, and track performance patterns in context. It supports continuous evaluation by running ongoing evals across model versions, prompt changes, judge updates, and live traces, helping teams detect quality regressions, drift, silent failures, contamination, and policy issues before they affect production. -
16
TensorZero
TensorZero
TensorZero is an open source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation. It creates a feedback loop for optimizing LLM applications, turning production metrics and human feedback into smarter, faster, and cheaper models and agents. The gateway lets teams integrate once and access every major LLM provider through a single unified API, including API and self-hosted models, with support for tool use, structured outputs, batch inference, embeddings, multimodal inputs, caching, routing, retries, fallbacks, load balancing, granular timeouts, usage tracking, custom rate limits, and provider-key protection. Built for performance in Rust, TensorZero is designed for extreme throughput and low-latency production workloads while still letting teams adopt only the components they need. Its observability layer stores inferences and feedback in the user’s own database, available programmatically or through the open source UI.Starting Price: Free -
17
Weavel
Weavel
Meet Ape, the first AI prompt engineer. Equipped with tracing, dataset curation, batch testing, and evals. Ape achieves an impressive 93% on the GSM8K benchmark, surpassing both DSPy (86%) and base LLMs (70%). Continuously optimize prompts using real-world data. Prevent performance regression with CI/CD integration. Human-in-the-loop with scoring and feedback. Ape works with the Weavel SDK to automatically log and add LLM generations to your dataset as you use your application. This enables seamless integration and continuous improvement specific to your use case. Ape auto-generates evaluation code and uses LLMs as impartial judges for complex tasks, streamlining your assessment process and ensuring accurate, nuanced performance metrics. Ape is reliable, as it works with your guidance and feedback. Feed in scores and tips to help Ape improve. Equipped with logging, testing, and evaluation for LLM applications.Starting Price: Free -
18
Plurai
Plurai
Plurai is the real-world trust platform for AI agents, built for simulation-driven evaluation, protection, and optimization that turns agents into trusted, continuously improving production systems. It helps teams train evals and guardrails tailored to their use case, bridging the gap from prototype to reliable production at scale. Plurai’s simulation platform prepares agents for the real world, not the lab, with hyper-realistic, product-tailored experimentation and evaluation that covers production complexity. It generates authentic multi-turn scenarios, personas, required artifacts, and tool mocking, using organizational PRDs, relevant sources, and policies to build a knowledge graph and expand edge-case coverage. Instead of relying on static datasets, manual test creation, or inconsistent LLM-as-a-judge methods, Plurai groups evaluations into structured, runnable experiments so teams can test new versions, measure regressions, and validate improvements before release.Starting Price: Free -
19
Instill Core
Instill AI
Instill Core is an all-in-one AI infrastructure tool for data, model, and pipeline orchestration, streamlining the creation of AI-first applications. Access is easy via Instill Cloud or by self-hosting from the instill-core GitHub repository. Instill Core includes: Instill VDP: The Versatile Data Pipeline (VDP), designed for unstructured data ETL challenges, providing robust pipeline orchestration. Instill Model: An MLOps/LLMOps platform that ensures seamless model serving, fine-tuning, and monitoring for optimal performance with unstructured data ETL. Instill Artifact: Facilitates data orchestration for unified unstructured data representation. Instill Core simplifies the development and management of sophisticated AI workflows, making it indispensable for developers and data scientists leveraging AI technologies.Starting Price: $19/month/user -
20
Selene 1
atla
Atla's Selene 1 API offers state-of-the-art AI evaluation models, enabling developers to define custom evaluation criteria and obtain precise judgments on their AI applications' performance. Selene outperforms frontier models on commonly used evaluation benchmarks, ensuring accurate and reliable assessments. Users can customize evaluations to their specific use cases through the Alignment Platform, allowing for fine-grained analysis and tailored scoring formats. The API provides actionable critiques alongside accurate evaluation scores, facilitating seamless integration into existing workflows. Pre-built metrics, such as relevance, correctness, helpfulness, faithfulness, logical coherence, and conciseness, are available to address common evaluation scenarios, including detecting hallucinations in retrieval-augmented generation applications or comparing outputs to ground truth data. -
21
doteval
doteval
doteval is an AI-assisted evaluation workspace that simplifies the creation of high-signal evaluations, alignment of LLM judges, and definition of rewards for reinforcement learning, all within a single platform. It offers a Cursor-like experience to edit evaluations-as-code against a YAML schema, enabling users to version evaluations across checkpoints, replace manual effort with AI-generated diffs, and compare evaluation runs on tight execution loops to align them with proprietary data. doteval supports the specification of fine-grained rubrics and aligned graders, facilitating rapid iteration and high-quality evaluation datasets. Users can confidently determine model upgrades or prompt improvements and export specifications for reinforcement learning training. It is designed to accelerate the evaluation and reward creation process by 10 to 100 times, making it a valuable tool for frontier AI teams benchmarking complex model tasks. -
22
EVALS
EVALS
EVALS is the most dynamic mobile skills assessment and tracking solution for public safety, providing students and instructors with powerful tools to enhance learning and performance. Record, stream, upload and review videos to reinforce the knowledge, skills, attitudes and beliefs associated with the proper process. Design realistic scenarios and situational evaluations that help students develop the specialized skills needed to be effective in the real world. Track on-the-job training hours and performance requirements using our unique Digital Taskbook and Time Tracking modules. Select the components you need to streamline and simplify your training evaluations, including Digital Taskbook, an embedded events calendar, attendance, and time tracking, private message boards, academic testing, and more. Access the platform from anywhere via a web-enabled device and use the iOS app to perform field and video assessments without an internet connection. -
23
Basalt
Basalt
Basalt is an AI-building platform that helps teams quickly create, test, and launch better AI features. With Basalt, you can prototype quickly using our no-code playground, allowing you to draft prompts with co-pilot guidance and structured sections. Iterate efficiently by saving and switching between versions and models, leveraging multi-model support and versioning. Improve your prompts with recommendations from our co-pilot. Evaluate and iterate by testing with realistic cases, upload your dataset, or let Basalt generate it for you. Run your prompt at scale on multiple test cases and build confidence with evaluators and expert evaluation sessions. Deploy seamlessly with the Basalt SDK, abstracting and deploying prompts in your codebase. Monitor by capturing logs and monitoring usage in production, and optimize by staying informed of new errors and edge cases.Starting Price: Free -
24
Noteweave
Noteweave
Noteweave is an Intelligent Research Machines platform that helps teams go from research to executable production plans. It is built to stress-test scientific research, translate papers into validated experiments, and run R&D faster from one research-first workspace. Deep Analysis pressure-tests methods, evaluations, and robustness so failure modes surface before they reach production, helping teams detect production faults in academic papers pre-emptively, find missing evals, set up discrepancies, or misleading robustness trends, and identify technical faults faster. Explore searches across millions of papers, datasets, and code repositories, then synthesizes them into runnable production plans with traceable evidence. Noteweave helps users discover relevant research signals across 3 million+ AI/ML publications, optimize plans against constraints such as GPU utilization, translate academic methods into reproducible steps, and validate evaluation strategies more reliably.Starting Price: $18.99 per month -
25
PointCab Origins
PointCab
PointCab Origins is your Swiss Army Knife for the evaluation of point cloud data, no matter from which laser scanner, and compatible with all CAD and BIM systems. From point cloud registration to the creation of vector lines and the transfer of your results into your CAD system, Origins offers you the perfect point cloud workflow. Origins automatically create a front, side, and top view (orthophotos) from the point cloud. Intuitive and easy to work with. Create floor plans & sections or measure areas, distances, volumes, and much more with just a few clicks. Even if you have little experience with point clouds, Origins is very intuitive to work with. Plus, our 2-minute tutorials will get you started in no time. Whether it's drones, terrestrial, or SLAM laser scanners, PointCab Origins processes all data. Of course, merging different point cloud data is no problem as well. PointCab Origns offers pro functions that meet sophisticated needs and use cases. -
26
Evalgent
Evalgent
Evalgent is an AI voice agent testing and evaluation platform. AI voice agents fail in production not because the technology is weak, but because demos use clean audio and cooperative users — real users don't. Evalgent catches failures before they reach production, cuts iteration cycles, and gets voice agents to revenue faster. HOW IT WORKS 1. Define: lock real scenarios and success criteria. 2. Run: run them under realistic human behavior. 3. Measure: see what works, what fails, and where limits lie. 3. Act: get clear, actionable insights on what to fix, tune, or deploy. FEATURES 1. Scenarios: define and generate test cases from agent instructions 2. Caller Profiles: simulate real users across accents, speech pace, and interruption patterns 3. Metrics: custom LLM-based and telemetry scoring across every conversation 4. Evaluations: structured campaigns with pass/fail verdicts and improvement recommendations 5. Reviews: human-in-the-loop correction with full audit trail -
27
Latitude
Latitude
Latitude is an open-source prompt engineering platform designed to help product teams build, evaluate, and deploy AI models efficiently. It allows users to import and manage prompts at scale, refine them with real or synthetic data, and track the performance of AI models using LLM-as-judge or human-in-the-loop evaluations. With powerful tools for dataset management and automatic logging, Latitude simplifies the process of fine-tuning models and improving AI performance, making it an essential platform for businesses focused on deploying high-quality AI applications.Starting Price: $0 -
28
ProdEval
Texas Computer Works
There is no such thing as a typical user of this system. Users include; independent reservoir engineers doing reserve reports, production engineers working up AFE’s and monitoring daily production, bank engineers tracking petroleum loan packages, CFOs tracking their borrowing base, property tax professionals assessing ad-valorem value, plus investors buying and selling producing properties. TCW’s ProdEval software is a quick and comprehensive Economic Evaluation system for both reserve reporting and prospect analysis. ProdEval has a very easy-to-use and straightforward approach to economic analysis and this methodology serves the user well. For example, the projecting of future production using sophisticated curve fitting techniques that allow the user to simply adjust the curves is one of the big factors that new users find attractive. The system is a rather open-ended system in that it accepts data from many sources; excel worksheets, commercial data sources. -
29
HoneyHive
HoneyHive
AI engineering doesn't have to be a black box. Get full visibility with tools for tracing, evaluation, prompt management, and more. HoneyHive is an AI observability and evaluation platform designed to assist teams in building reliable generative AI applications. It offers tools for evaluating, testing, and monitoring AI models, enabling engineers, product managers, and domain experts to collaborate effectively. Measure quality over large test suites to identify improvements and regressions with each iteration. Track usage, feedback, and quality at scale, facilitating the identification of issues and driving continuous improvements. HoneyHive supports integration with various model providers and frameworks, offering flexibility and scalability to meet diverse organizational needs. It is suitable for teams aiming to ensure the quality and performance of their AI agents, providing a unified platform for evaluation, monitoring, and prompt management. -
30
Tülu 3
Ai2
Tülu 3 is an advanced instruction-following language model developed by the Allen Institute for AI (Ai2), designed to enhance capabilities in areas such as knowledge, reasoning, mathematics, coding, and safety. Built upon the Llama 3 Base, Tülu 3 employs a comprehensive four-stage post-training process: meticulous prompt curation and synthesis, supervised fine-tuning on a diverse set of prompts and completions, preference tuning using both off- and on-policy data, and a novel reinforcement learning approach to bolster specific skills with verifiable rewards. This open-source model distinguishes itself by providing full transparency, including access to training data, code, and evaluation tools, thereby closing the performance gap between open and proprietary fine-tuning methods. Evaluations indicate that Tülu 3 outperforms other open-weight models of similar size, such as Llama 3.1-Instruct and Qwen2.5-Instruct, across various benchmarks.Starting Price: Free -
31
AfterQuery
AfterQuery
AfterQuery is an applied research platform designed to create high-quality training data for frontier artificial intelligence models by capturing how real experts think, reason, and solve problems in professional contexts. It focuses on transforming real-world work into structured datasets that go beyond simple outputs, encoding decision-making processes, tradeoffs, and contextual reasoning that traditional internet-sourced data cannot provide. It works directly with domain experts to generate supervised fine-tuning data, including prompt–response pairs and detailed reasoning traces, as well as reinforcement learning datasets with expert-designed prompts and grading frameworks that convert subjective judgment into scalable reward signals. It also builds custom agent environments across APIs and tools, enabling models to be trained and evaluated in realistic workflows, and captures computer-use trajectories that demonstrate how humans interact with software step by step. -
32
SnapEval 2.0
SnapEval
Instantly capture and share feedback ‘snapshots’ using smartphones and computers. Automatically incorporate feedback snapshots into a Performance Summary. Nominate a feedback snapshot for public recognition of performance excellence within the organization. Drag and drop to establish relationships. Explore organization structure ‘what ifs.’ Live access and file export sharing. Instantly create and send custom rich push notification messages to smartphones. Align employees with the organization’s values and goals. Gain comprehensive visibility into performance levels and trends across the firm. Automatically create professional evaluations using Continuous Feedback. Universal support of employee performance feedback for all job functions across all industries. Feedback is captured and shared in intuitive snapshots called ‘Evals’.Starting Price: $2.25 per user per month -
33
Pezzo
Pezzo
Pezzo is the open-source LLMOps platform built for developers and teams. In just two lines of code, you can seamlessly troubleshoot and monitor your AI operations, collaborate and manage your prompts in one place, and instantly deploy changes to any environment.Starting Price: $0 -
34
Laguna XS.2
Poolside
Laguna XS.2 is Poolside’s open-weight agentic coding model, built as the lightest and fastest model in the Laguna family. It is a 33B total-parameter Mixture of Experts model with 3B activated parameters, trained completely in-house on 30T tokens. As Poolside’s newest generation model open to the community, Laguna XS.2 is a second-generation architecture and the company’s first open-weight model, built on the lessons learned from training Laguna M.1 across synthetic data and reinforcement learning. The model is designed for agentic coding workflows, where it can code, act, iterate quickly, and perform best inside Poolside’s coding agent. Laguna XS.2 is positioned as a strong model for rapid agentic iteration, especially for developers and teams that need a compact, efficient coding model rather than a heavier frontier system. It is released under an Apache 2.0 license, allowing the community to evaluate, fine-tune, quantize, serve, and build on the weights.Starting Price: Free -
35
OpenEuroLLM
OpenEuroLLM
OpenEuroLLM is a collaborative initiative among Europe's leading AI companies and research institutions to develop a series of open-source foundation models for transparent AI in Europe. The project emphasizes transparency by openly sharing data, documentation, training, testing code, and evaluation metrics, fostering community involvement. It ensures compliance with EU regulations, aiming to provide performant large language models that align with European standards. A key focus is on linguistic and cultural diversity, extending multilingual capabilities to encompass all EU official languages and beyond. The initiative seeks to enhance access to foundational models ready for fine-tuning across various applications, expand evaluation results in multiple languages, and increase the availability of training datasets and benchmarks. Transparency is maintained throughout the training processes by sharing tools, methodologies, and intermediate results. -
36
Verta
Verta
Get everything you need to start customizing LLMs and prompts immediately, no PhD required. Starter Kits with model, prompt, and dataset suggestions matched to your use case allow you to begin testing, evaluating, and refining model outputs right away. Experiment with multiple models (proprietary and open source), prompts, and techniques simultaneously to speed up the iteration process. Automated testing and evaluation and AI-powered prompt and refinement suggestions enable you to run many experiments at once to quickly achieve high-quality results. Verta’s easy-to-use platform empowers builders of all tech levels to achieve high-quality model outputs quickly. Using a human-in-the-loop approach to evaluation, Verta prioritizes human feedback at key points in the iteration cycle to capture expertise and develop IP to differentiate your GenAI products. Easily keep track of your best-performing options from Verta’s Leaderboard. -
37
Katana
Foundry
Fast and formidable, Katana is a look development and lighting powerhouse that tackles creative challenges with ferocity and ease. It arms artists with the creative freedom and scalability to exceed the needs of today’s most demanding CG-rendering projects. Light at the speed of thought thanks to cutting-edge Lighting Tools, empowering artists to light whole sequences of shots with Katana’s industry-leading, multi-shot workflows. Katana’s Foresight Rendering workflows, comprised of Multiple Simultaneous Renders and Networked Interactive Rendering, provide artists with scalable feedback for faster iterations. Katana is made to drive look development of hero or high volume assets while seamlessly collaborating with shot production. USD-optimised tech, combined with multiple APIs, five different commercial renderers and an open-sourced Shotgun TK integration, make Katana the Swiss Army knife of your pipeline. -
38
Dynamiq
Dynamiq
Dynamiq is a platform built for engineers and data scientists to build, deploy, test, monitor and fine-tune Large Language Models for any use case the enterprise wants to tackle. Key features: 🛠️ Workflows: Build GenAI workflows in a low-code interface to automate tasks at scale 🧠 Knowledge & RAG: Create custom RAG knowledge bases and deploy vector DBs in minutes 🤖 Agents Ops: Create custom LLM agents to solve complex task and connect them to your internal APIs 📈 Observability: Log all interactions, use large-scale LLM quality evaluations 🦺 Guardrails: Precise and reliable LLM outputs with pre-built validators, detection of sensitive content, and data leak prevention 📻 Fine-tuning: Fine-tune proprietary LLM models to make them your ownStarting Price: $125/month -
39
OpenPipe
OpenPipe
OpenPipe provides fine-tuning for developers. Keep your datasets, models, and evaluations all in one place. Train new models with the click of a button. Automatically record LLM requests and responses. Create datasets from your captured data. Train multiple base models on the same dataset. We serve your model on our managed endpoints that scale to millions of requests. Write evaluations and compare model outputs side by side. Change a couple of lines of code, and you're good to go. Simply replace your Python or Javascript OpenAI SDK and add an OpenPipe API key. Make your data searchable with custom tags. Small specialized models cost much less to run than large multipurpose LLMs. Replace prompts with models in minutes, not weeks. Fine-tuned Mistral and Llama 2 models consistently outperform GPT-4-1106-Turbo, at a fraction of the cost. We're open-source, and so are many of the base models we use. Own your own weights when you fine-tune Mistral and Llama 2, and download them at any time.Starting Price: $1.20 per 1M tokens -
40
Dify
Dify
Dify is an open-source platform designed to streamline the development and operation of generative AI applications. It offers a comprehensive suite of tools, including an intuitive orchestration studio for visual workflow design, a Prompt IDE for prompt testing and refinement, and enterprise-level LLMOps capabilities for monitoring and optimizing large language models. Dify supports integration with various LLMs, such as OpenAI's GPT series and open-source models like Llama, providing flexibility for developers to select models that best fit their needs. Additionally, its Backend-as-a-Service (BaaS) features enable seamless incorporation of AI functionalities into existing enterprise systems, facilitating the creation of AI-powered chatbots, document summarization tools, and virtual assistants. -
41
Light Table
Light Table
Connects you to your creation with instant feedback and showing data values flow through your code. Easily customizable from keybinds to extensions to be completely tailored to your specific project. Try new ideas quickly and easily. Ask questions about your software, to give you a more profound understanding of your code. Embed anything you want, from graphs to games to running visualizations. Everything from eval and debugging to a fuzzy finder for files and commands to fit seamlessly into your workflow. An elegant, lightweight, beautifully designed layout so your IDE is no longer cluttered. No more printing to the console in order to view your results. Simply evaluate your code and the results will be displayed inline. Developer tools should be open source. Every bit of Light Table's code is available to the community because none of us are as smart as all of us. -
42
Entry Point AI
Entry Point AI
Entry Point AI is the modern AI optimization platform for proprietary and open source language models. Manage prompts, fine-tunes, and evals all in one place. When you reach the limits of prompt engineering, it’s time to fine-tune a model, and we make it easy. Fine-tuning is showing a model how to behave, not telling. It works together with prompt engineering and retrieval-augmented generation (RAG) to leverage the full potential of AI models. Fine-tuning can help you to get better quality from your prompts. Think of it like an upgrade to few-shot learning that bakes the examples into the model itself. For simpler tasks, you can train a lighter model to perform at or above the level of a higher-quality model, greatly reducing latency and cost. Train your model not to respond in certain ways to users, for safety, to protect your brand, and to get the formatting right. Cover edge cases and steer model behavior by adding examples to your dataset.Starting Price: $49 per month -
43
Double Time Docs
Double Time Docs
Answer multiple choice, fill-in and short answer questions about your student's background, observation and assessments. Fill in custom Comment boxes whenever you need to include special information not covered by our questions. As you fill out the questions, at any time you can preview the evaluation report. Full sentences are created based on your answers to the questions. The sentences use the student's name and correct pronoun every time. No more name or pronoun mistakes! When you're satisfied with your evaluation report (which won't take long!), download it to your computer as a Word Document or automatically create a Google Doc on your Google Drive and fine-tune it to your heart's content. Time is one of our most precious commodities. With the increase in caseload, referrals and assessments, there is not enough time in the school day to write an evaluation. On average, it can take more than 3 hours to write a Pediatric SLP, OT or PT evaluation report. DTD can cut that in half.Starting Price: $7 per month -
44
CALIBRAT
TalentBridge Technologies
Assessing a large number of candidates is cumbersome & painstaking task. This platform organizes and streamlines the manual processes in simple steps to conduct assessments online with easy-to-follow administration, scoring, and interpretation. You can pay to match your assessment needs. Cost-effective approach for assessment requirements with access to all the platform features. Eliminate the logistical costs of paper-based assessments. Auto-evaluation or platform aided evaluation reduces evaluation efforts thereby reducing costs incurred in traditional paper-based methods. Individual judgment about candidates tend to be subjective and prone to error. Standardised assessments can help to reduce such subjective judgements and help in making the accurate and effective decision about candidates. -
45
Airtrain
Airtrain
Query and compare a large selection of open-source and proprietary models at once. Replace costly APIs with cheap custom AI models. Customize foundational models on your private data to adapt them to your particular use case. Small fine-tuned models can perform on par with GPT-4 and are up to 90% cheaper. Airtrain’s LLM-assisted scoring simplifies model grading using your task descriptions. Serve your custom models from the Airtrain API in the cloud or within your secure infrastructure. Evaluate and compare open-source and proprietary models across your entire dataset with custom properties. Airtrain’s powerful AI evaluators let you score models along arbitrary properties for a fully customized evaluation. Find out what model generates outputs compliant with the JSON schema required by your agents and applications. Your dataset gets scored across models with standalone metrics such as length, compression, coverage.Starting Price: Free -
46
Laminar
Laminar
Laminar is an open source all-in-one platform for engineering best-in-class LLM products. Data governs the quality of your LLM application. Laminar helps you collect it, understand it, and use it. When you trace your LLM application, you get a clear picture of every step of execution and simultaneously collect invaluable data. You can use it to set up better evaluations, as dynamic few-shot examples, and for fine-tuning. All traces are sent in the background via gRPC with minimal overhead. Tracing of text and image models is supported, audio models are coming soon. You can set up LLM-as-a-judge or Python script evaluators to run on each received span. Evaluators label spans, which is more scalable than human labeling, and especially helpful for smaller teams. Laminar lets you go beyond a single prompt. You can build and host complex chains, including mixtures of agents or self-reflecting LLM pipelines.Starting Price: $25 per month -
47
PROBIS Expert
emproc
The cloud-based multi-project controlling software PROBIS Expert for the real estate industry is able to control and evaluate the costs of complex projects efficiently and transparently. Despite highly complex content, the platform is intuitive to use and therefore easy to handle for all project participants. Data is available in real-time and everywhere. The project structures are graphically prepared and clearly displayed. This provides a central overview, evaluation and analysis of the costs of different projects. The experts at emproc SYS, who developed this software are themselves experienced controllers who provide international customers with assistance in structuring and optimizing digital processes and overall control. Dashboard. Extensive reporting in real time. Individual configurable, user-friendly data view. Transparent comparability of any cost scenarios. Property developers, project managers or financial institutes can employ PROBIS Expert to create reporting -
48
Scale Evaluation
Scale
Scale Evaluation offers a comprehensive evaluation platform tailored for developers of large language models. This platform addresses current challenges in AI model assessment, such as the scarcity of high-quality, trustworthy evaluation datasets and the lack of consistent model comparisons. By providing proprietary evaluation sets across various domains and capabilities, Scale ensures accurate model assessments without overfitting. The platform features a user-friendly interface for analyzing and reporting model performance, enabling standardized evaluations for true apples-to-apples comparisons. Additionally, Scale's network of expert human raters delivers reliable evaluations, supported by transparent metrics and quality assurance mechanisms. The platform also offers targeted evaluations with custom sets focusing on specific model concerns, facilitating precise improvements through new training data. -
49
Snowglobe
Snowglobe
Snowglobe is a high-fidelity simulation engine that helps AI teams test LLM applications at scale by simulating real-world user conversations before launch. It generates thousands of realistic, diverse dialogues by creating synthetic users with distinct goals and personalities that interact with your chatbot’s endpoints across varied scenarios, exposing blind spots, edge cases, and performance issues early. Snowglobe produces labeled outcomes so teams can evaluate behavior consistently, generate high-quality training data for fine-tuning, and iteratively improve model performance. Designed for reliability work, it addresses risks like hallucinations and RAG fragility by stress-testing retrieval and reasoning in lifelike workflows rather than narrow prompts. Getting started is fast: connect your bot to Snowglobe’s simulation environment and, with an API key for your LLM provider, run end-to-end tests in minutes.Starting Price: $0.25 per message -
50
Microsoft Foundry Models
Microsoft
Microsoft Foundry Models is a unified model catalog that gives enterprises access to more than 11,000 AI models from Microsoft, OpenAI, Anthropic, Mistral AI, Meta, Cohere, DeepSeek, xAI, and others. It allows teams to explore, test, and deploy models quickly using a task-centric discovery experience and integrated playground. Organizations can fine-tune models with ready-to-use pipelines and evaluate performance using their own datasets for more accurate benchmarking. Foundry Models provides secure, scalable deployment options with serverless and managed compute choices tailored to enterprise needs. With built-in governance, compliance, and Azure’s global security framework, businesses can safely operationalize AI across mission-critical workflows. The platform accelerates innovation by enabling developers to build, iterate, and scale AI solutions from one centralized environment.