Open Source C++ Reinforcement Learning Frameworks

AirSim is an open-source, cross platform simulator for drones, cars and more vehicles, built on Unreal Engine with an experimental Unity release in the works. It supports software-in-the-loop simulation with popular flight controllers such as PX4 & ArduPilot and hardware-in-loop with PX4 for physically and visually realistic simulations. It is developed as an Unreal plugin that can simply be dropped into any Unreal environment. AirSim's development is oriented towards the goal of creating a platform for AI research to experiment with deep learning, computer vision and reinforcement learning algorithms for autonomous vehicles. For this purpose, AirSim also exposes APIs to retrieve data and control vehicles in a platform independent way. AirSim is fully enabled for multiple vehicles. This capability allows you to create multiple vehicles easily and use APIs to control them.

This is the official C++ source code repository of the Bullet Physics SDK: real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc. We are developing a new differentiable simulator for robotics learning, called Tiny Differentiable Simulator, or TDS. The simulator allows for hybrid simulation with neural networks. It allows different automatic differentiation backends, for forward and reverse mode gradients. TDS can be trained using Deep Reinforcement Learning, or using Gradient based optimization (for example LFBGS). In addition, the simulator can be entirely run on CUDA for fast rollouts, in combination with Augmented Random Search. This allows for 1 million simulation steps per second. It is highly recommended to use PyBullet Python bindings for improved support for robotics, reinforcement learning and VR. Use pip install pybullet and checkout the PyBullet Quickstart Guide.

ViZDoom allows developing AI bots that play Doom using only the visual information (the screen buffer). It is primarily intended for research in machine visual learning, and deep reinforcement learning, in particular. ViZDoom is based on ZDOOM, the most popular modern source-port of DOOM. This means compatibility with a huge range of tools and resources that can be used to create custom scenarios, availability of detailed documentation of the engine and tools and support of Doom community. Async and sync single-player and multi-player modes. Fast (up to 7000 fps in sync mode, single-threaded). Lightweight (few MBs). Customizable resolution and rendering parameters. Access to the depth buffer (3D vision). Automatic labeling of game objects visible in the frame. Access to the list of actors/objects and map geometry.ViZDoom API is reinforcement learning friendly (suitable also for learning from demonstration, apprenticeship learning or apprenticeship via inverse reinforcement learning.

In an optical network scenario which consists of multiple nodes (whiteboxes) at its edges and ROADMs in-between, the coherent transceiver average laser configuration time is improved. The process is evaluated according to a testbed setup. This is facilitated in the appropriate lab equipment (or via simulation when required). For that purpose, a software agent (Netconf server) residing at the whiteboxes, is developed receiving input from the Software-Defined Networking (SDN) packet controller (PacketCTL - a Netconf client). Then, configuration of the local transceiver laser frequencies of the controlled pluggable devices takes place, for facilitating the connectivity in-between the ROADM network. Also, the agent records and reports back telemetry data (feedback) which is used by the PacketCTL's resource-allocating mechanism to improve efficiency within the network topology.

Closed Loop Simulation System (CLSquare) is an integrated architecture to train, test and compare reinforcement learning controllers on different plants. CLSquare provides simulated plants as well as interfaces to real plants.

EnvPool is a fast, asynchronous, and parallel RL environment library designed for scaling reinforcement learning experiments. Developed by SAIL at Singapore, it leverages C++ backend and Python frontend for extremely high-speed environment interaction, supporting thousands of environments running in parallel on a single machine. It's compatible with Gymnasium API and RLlib, making it suitable for scalable training pipelines.

The Lattice Visualizer 9 provides a comprehensive computational platform for studying emergent cognition, successfully integrating heterogeneous physical, neurodynamic, and adaptive regulation mechanisms. The core strength of the LV9 framework lies in its intrinsic coupling: the physical structure is influenced by information flow, the information flow is regulated by the energetic state, and the energetic state is homeostatically tuned by the Self-Organized Criticality (SOC) feedback loop, ensuring the system consistently operates near its most computationally optimal point. The multithreaded architecture, though facing inherent synchronization constraints during the Pulse Transfer phase, ensures high throughput for the most computationally demanding phases (Apply Updates and CycleRL). Furthermore, the 8 x 8 XOR syndrome memory architecture provides the necessary self-healing robustness against transient corruption, validating the "thread-safe" and "self-healing" claims.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games. OpenSpiel supports n-player (single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and sequential, strictly turn-taking and simultaneous-move, perfect and imperfect information games, as well as traditional multiagent environments such as (partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel also includes tools to analyze learning dynamics and other common evaluation metrics. Games are represented as procedural extensive-form games, with some natural extensions. The core API and games are implemented in C++ and exposed to Python. Algorithms and tools are written both in C++ and Python. To try OpenSpiel in Google Colaboratory, please refer to open_spiel/colabs subdirectory.

Parallel Reinforcement Evolutionary Artificial Neural Networks (PREANN) is a framework of flexible multi-layer ANN's with reinforcement learning based on genetic algorithms and a parallel implementation (using XMM registers and NVIDIA's CUDA).

RL++ is an easy to use modular open source library for Reinforcement Learning written in C++. It includes learning algorithms (TD, Sarsa, Q) as well as the implementation of value function representations (LookupTable, TileCoding, Neuronal Network).

Reinforcement learning and simulation-based search in computer Go

Highly modularized Reinforcement Learning library for real/simulation robots to learn behaviors. Our ultimate goal is to develop an artificial intelligence (AI) program with which the robots can learn to behave as their users wish.

We present TorchCraft, a library that enables deep learning research on Real-Time Strategy (RTS) games such as StarCraft: Brood War, by making it easier to control these games from a machine learning framework, here Torch. This white paper argues for using RTS games as a benchmark for AI research, and describes the design and components of TorchCraft. TorchCraft is a BWAPI module that sends StarCraft data out over a ZMQ connection. This lets you parse StarCraft data and interact with BWAPI from anywhere. The TorchCraft client should be installed from C++, Python, or Lua. We provide off-the-shelf solutions for Python and Lua.

General purpose agents using reinforcement learning. Combines radial basis functions, temporal difference learning, planning, uncertainty estimations, and curiosity. Intended to be an out-of-the-box solution for roboticists and game developers.

Vowpal Wabbit is a machine learning system that pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning. There is a specific focus on reinforcement learning with several contextual bandit algorithms implemented and the online nature lending to the problem well. Vowpal Wabbit is a destination for implementing and maturing state-of-the-art algorithms with performance in mind. The input format for the learning algorithm is substantially more flexible than might be expected. Examples can have features consisting of free-form text, which is interpreted in a bag-of-words way. There can even be multiple sets of free-form text in different namespaces. Similar to the few other online algorithm implementations out there. There are several optimization algorithms available with the baseline being sparse gradient descent (GD) on a loss function.