Search Results for "test"
Sort By:
Showing 144 open source projects for "test"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
1
Ragas
Supercharge Your LLM Application Evaluations
Objective metrics, intelligent test generation, and data-driven insights for LLM apps. Ragas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications. Say goodbye to time-consuming, subjective assessments and hello to data-driven, efficient evaluation workflows. Don't have a test dataset ready? We also do production-aligned test set generation. -
2
CodiumAI Cover-Agent
CodiumAI Cover-Agent: An AI-Powered Tool for Automated Test Generation
CodiumAI Cover Agent aims to help efficiently increasing code coverage, by automatically generating qualified tests to enhance existing test suites. -
3
TTRL
Test-Time Reinforcement Learning
TTRL is an open-source framework for test-time reinforcement learning in large language models, with a particular focus on reasoning tasks where ground-truth labels are not available during inference. The project addresses the problem of how to generate useful reward signals from unlabeled test-time data, and its central insight is that common test-time scaling practices such as majority voting can be repurposed into reward estimates for online reinforcement learning. ... -
4
Qodo Cover
AI tool that generates tests to improve code coverage quickly
...It supports scanning entire repositories to automatically detect test files, gather relevant context, and extend test suites accordingly. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Giskard
Collaborative & Open-Source Quality Assurance for all AI models
...Giskard automatically generates relevant tests based on the vulnerabilities detected by the scan. You can easily customize the tests depending on your use case by defining domain-specific data slicers and transformers as fixtures of your test suites. -
6
PML
The easiest way to use deep metric learning in your application
This library contains 9 modules, each of which can be used independently within your existing codebase, or combined together for a complete train/test workflow. To compute the loss in your training loop, pass in the embeddings computed by your model, and the corresponding labels. The embeddings should have size (N, embedding_size), and the labels should have size (N), where N is the batch size. The TripletMarginLoss computes all possible triplets within the batch, based on the labels you pass into it. ... -
7
aisuite
Simple, unified interface to multiple Generative AI providers
...Using an interface similar to OpenAI's, aisuite makes it easy to interact with the most popular LLMs and compare the results. It is a thin wrapper around Python client libraries and allows creators to seamlessly swap out and test responses from different LLM providers without changing their code. Today, the library is primarily focused on chat completions. We will expand it to cover more use cases in the near future. Currently supported providers are - OpenAI, Anthropic, Azure, Google, AWS, Groq, Mistral, HuggingFace and Ollama. To maximize stability, aisuite uses either the HTTP endpoint or the SDK for making calls to the provider. -
8
FullTClash
General proxy performance testing tool based on Clash using Telegram
...The front end part uses Telegram API as the interactive interface, which needs to be used in conjunction with Telegram, that is, a Telegram robot (bot), FullTClash bot is a Telegram robot (hereinafter referred to as bot) carrying its test tasks. -
9
amrlib
A python library that makes AMR parsing, generation and visualization
...A QT-based GUI to facilitate the conversion of sentences to graphs and back to sentences. Methods to plot AMR graphs in both the GUI and as library functions. Training and test code for both the StoG and GtoS models. A SpaCy extension that allows direct conversion of SpaCy Docs and Spans to AMR graphs. Sentence to Graph alignment routines FAA_Aligner (Fast_Align Algorithm), based on the ISI aligner code detailed in this paper. RBW_Aligner (Rule Based Word) for a simple, single token to single node alignment. -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
10
FuzzyAI Fuzzer
A powerful tool for automated LLM fuzzing
...The framework can be integrated into development pipelines to continuously test AI APIs and detect weaknesses before deployment. FuzzyAI provides testing tools, datasets, and evaluation workflows that help researchers measure how well models resist harmful instructions or attempts to bypass safety mechanisms. -
11
PaddleX
PaddlePaddle End-to-End Development Toolkit
...Users only need to put pictures belonging to the same category in the same folder. When the model is trained, we need to divide the training set, the validation set and the test set. Therefore, we need to divide the above data. Using the paddlex command, the data set can be randomly divided into 70% training set, 20% validation set and 10% test set. If you use the PaddleX visualization client for model training, the data set division function is integrated in the client, and you do not need to use command division by yourself. -
12
Freqtrade
Free, open source crypto trading bot
Freqtrade is a free and open-source crypto trading bot written in Python. It is designed to support all major exchanges and be controlled via Telegram or WebUI. It contains backtesting, plotting, and money management tools as well as strategy optimization by machine learning. Always start by running a trading bot in Dry-run and do not engage money before you understand how it works and what profit/loss you should expect. We strongly recommend you have basic coding skills and Python... -
13
ASSERT
Requirement-driven evaluation harness for AI agents and LLM
...It turns natural-language specifications, policies, product requirements, and launch criteria into structured tests that can be reviewed, executed, scored, and improved. The pipeline derives behavior categories, generates single-turn and multi-turn test cases, runs them against a target system, and uses an LLM judge to score conversations against the stated policies. It can evaluate hosted models, custom agents, multi-agent systems, REST clients, and frameworks such as LangGraph, CrewAI, AutoGen, DSPy, LlamaIndex, and OpenAI Agents SDK. ASSERT is designed to close the gap between what a system is supposed to do and what evaluation actually measures. ... -
14
PySpur
Visual tool for building, testing, and deploying AI agent workflows
PySpur is a visual development environment designed to help AI engineers build, test, and iterate on agent-based workflows more efficiently. It provides a structured playground where users can define test cases, construct agents either through Python code or a graphical interface, and continuously refine their behavior. It addresses common challenges in AI agent development such as prompt tuning difficulties and lack of visibility into workflow execution. -
15
SWE-agent
SWE-agent takes a GitHub issue and tries to automatically fix it
...GPT-4) into software engineering agents that can resolve issues in real GitHub repositories. On the SWE-bench, the SWE-agent resolves 12.47% of issues, achieving state-of-the-art performance on the full test set. We accomplish our results by designing simple LM-centric commands and feedback formats to make it easier for the LM to browse the repository, and view, edit, and execute code files. We call this an Agent-Computer Interface (ACI). -
16
Evidently
Evaluate and monitor ML models from validation to production
Evidently is an open-source Python library for data scientists and ML engineers. It helps evaluate, test, and monitor ML models from validation to production. It works with tabular, text data and embeddings. -
17
Rogue
AI Agent Evaluator & Red Team Platform
Rogue is an open-source evaluation and red-team framework designed to test the reliability, safety, and policy compliance of AI agents. The platform automatically interacts with an AI agent by generating dynamic scenarios and multi-turn conversations that simulate real-world interactions. Instead of relying solely on static test scripts, Rogue uses an agent-as-a-judge architecture where one agent probes another agent to detect failures or unexpected behaviors. -
18
tf2onnx
Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX
...TensorFlow has many more ops than ONNX and occasionally mapping a model to ONNX creates issues. tf2onnx will use the ONNX version installed on your system and installs the latest ONNX version if none is found. We support and test ONNX opset-13 to opset-17. opset-6 to opset-12 should work but we don't test them. If you want the graph to be generated with a specific opset, use --opset in the command line, for example --opset 13. When running under tf-2.x tf2onnx will use the tensorflow V2 controlflow. -
19
Deepchecks
Test Suites for validating ML models & data
Deepchecks is the leading tool for testing and for validating your machine learning models and data, and it enables doing so with minimal effort. Deepchecks accompany you through various validation and testing needs such as verifying your data’s integrity, inspecting its distributions, validating data splits, evaluating your model and comparing between different models. While you’re in the research phase, and want to validate your data, find potential methodological problems, and/or validate... -
20
Mistral Inference
Official inference library for Mistral models
Open and portable generative AI for devs and businesses. We release open-weight models for everyone to customize and deploy where they want it. Our super-efficient model Mistral Nemo is available under Apache 2.0, while Mistral Large 2 is available through both a free non-commercial license, and a commercial license. -
21
skfolio
Python library for portfolio optimization built on top of scikit-learn
...It supports a wide range of allocation methods, from classical mean-variance optimization to modern techniques that rely on clustering, factor models, and risk-based allocations. The framework also includes tools for evaluating portfolio performance under different market conditions, enabling users to test robustness and reduce the risk of overfitting. -
22
AgentOps
Python SDK for agent monitoring, LLM cost tracking, benchmarking, etc.
Industry-leading developer platform to test and debug AI agents. We built the tools so you don't have to. Visually track events such as LLM calls, tools, and multi-agent interactions. Rewind and replay agent runs with point-in-time precision. Keep a full data trail of logs, errors, and prompt injection attacks from prototype to production. Native integrations with the top agent frameworks. -
23
MTEB
MTEB: Massive Text Embedding Benchmark
Text embeddings are commonly evaluated on a small set of datasets from a single task not covering their possible applications to other tasks. It is unclear whether state-of-the-art embeddings on semantic textual similarity (STS) can be equally well applied to other tasks like clustering or reranking. This makes progress in the field difficult to track, as various models are constantly being proposed without proper evaluation. To solve this problem, we introduce the Massive Text Embedding... -
24
langrocks
Tools like web browser, computer access and code runner for LLMs
Langrocks is a programming language experimentation toolkit that enables developers to create, test, and optimize custom programming languages. -
25
Harvey LAB
A benchmark built to evaluate and improve agent capabilities
...It combines a dataset of legal assignments with an execution harness that runs agents against those tasks. Each benchmark task includes instructions, working documents, and rubrics that define successful completion. The environment is designed to test practical legal workflows rather than isolated question-answering ability. Evaluation tools score agent outputs and support reports and comparative experiment runs. Documentation includes an end-to-end M&A data-room example covering setup, execution, scoring, and result analysis. The project is intended to help researchers and developers identify weaknesses and measure improvements in legal AI agents.
