Search Results for "benchmark testing"

ARC-AGI is a benchmark dataset and experimental framework designed to evaluate and advance artificial general intelligence by testing systems on abstract reasoning tasks that require human-like problem-solving abilities. It consists of a curated set of tasks where models must infer patterns from input-output examples and apply those rules to new unseen cases, without relying on memorization or prior training data.

A unified, comprehensive and efficient recommendation library. We design general and extensible data structures to unify the formatting and usage of various recommendation datasets. We implement more than 100 commonly used recommendation algorithms and provide formatted copies of 28 recommendation datasets. We support a series of widely adopted evaluation protocols or settings for testing and comparing recommendation algorithms. RecBole is developed based on Python and PyTorch for...

Kinetics-I3D, developed by Google DeepMind, provides trained models and implementation code for the Inflated 3D ConvNet (I3D) architecture introduced in the paper “Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset” (CVPR 2017). The I3D model extends the 2D convolutional structure of Inception-v1 into 3D, allowing it to capture spatial and temporal information from videos for action recognition. This repository includes pretrained I3D models on the Kinetics dataset, with...

This repository is designed to provide a minimal benchmark framework comparing commonly used machine learning libraries in terms of scalability, speed, and classification accuracy. The focus is on binary classification tasks without missing data, where inputs can be numeric or categorical (after one-hot encoding). It targets large scale settings by varying the number of observations (n) up to millions and the number of features (after expansion) to about a thousand, to stress test different...

This project is now maintained at GitHub: https://github.com/DigitalInBlue/Celero Celero is a cross-platform open source C++ benchmarking library. Written with an API similar to Google Test, Celero utilizes C++11 features, offers tested benchmarking capabilities with microsecond precision, baselining, fixtures, and easy test setup. Use the baseline feature to measure/benchmark algorithm performance against a known case to more accurately rate performance of solutions.