BRAINSTORMERS PUBLIC SOURCE CODE RELEASE
The source code of the Brainstormers' RoboCup Soccer Simulation 2D World Champion team 2005 has been made publicly available at the end of 2005. That source code release contains also a lot of our results in applying Reinforcement Learning in the simulated soccer domain - some learning algorithms as well as behaviors that exploit the results of learning.
The Brainstormers have been participating in RoboCup's soccer simulation tournaments since 1998. Ever since a number of successes could be achieved, including multiple World Vice Champion titles and the World Champion title at RoboCup 2005 in Osaka, at RoboCup 2007 in Atlanta, and at RoboCup 2008 in Suzhou.
The underlying and encouraging research goal in the development of the Brainstormers has always been to exploit AI and machine learning techniques wherever possible. In particular, the development, investigation, and successful application of Reinforcement Learning (RL) approaches in a complex domain like robotic soccer is one of the main stimulating factors of our work.
In the current public source code release you will find
may find a lot of our results in applying Reinforcement Learning in the simulated soccer domain - some learning algorithms as well as behaviors that exploit the results of learning (e.g. those behaving greedily with respect to a value function represented by a neural network). The following learned modules are part of this release:
* learned skills, such as ball kicking (see NeuroKick and NeuroKick05), ball intercepting (see NeuroIntercept), going to a position (see NeuroGo2Pos)
* learned medium-level capabilities, like passing (see NeuroWball, LearnWball)
* learned team-level capabilities, such as cooperatively scoring a goal (see Score04) or positioning (see NeuroPositioning)
* function approximators to represent state and state-action value functions (mostly multi-layer neural networks)
By the end 2005, the source code of our team has been made publicly available. This way, we hope to make a contribution to the entire soccer simulation community, in particular to new teams for which, as is known, it is difficult to overcome basic problems, such as developing a reliable world model or basic skills. For a more thorough description of the contents of the release, you should refer to the documentation file.
°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
Due to frequent request, the release tar ball
BS05PubRel.tar.gz has been made available on
Sourceforge as well (see
http://sourceforge.net/projects/bsrelease).
°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
The Brainstormers' code has been developed in C++ under Linux. We have tested compiling it with GNU C++ compilers (g++/gcc) up to version 3.3.
It is distributed under the terms of the GNU General Public License. It comes without any warranty and support.
Contact: Thomas Gabel
References:
Riedmiller, M., Buck, S., Merke, A., Ehrmann, R., Thate, O., Dilger, S., Sinner,
A., Hofmann, A., Frommberger, L.: Karlsruhe Brainstormers - Design Principles.
In: RoboCup-1999: Robot Soccer World Cup III, LNCS. Springer (1999)
Riedmiller, M., Merke, A., Meier, D., Hoffmann, A., Sinner, A., Thate, O., Kill,
C., Ehrmann, R.: Karlsruhe Brainstormers - A Reinforcement Learning Way to
Robotic Soccer. In Jennings, A., Stone, P., eds.: RoboCup-2000: Robot Soccer
World Cup IV, LNCS. Springer, Melbourne, Australia (2000)
Merke, A., Riedmiller, M.: Karlsruhe Brainstormers – A Reinforcement Learning
Way to Robotic Soccer II. In Birk, A., Coradeschi, S., Tadokoro, S., eds.: RoboCup-
2001: Robot Soccer World Cup V, LNCS. Springer, Seattle, USA (2001) 322–327
Riedmiller, M., Merke, A., Hoffmann, A., Withopf, D., Nickschas, M., Zacharias,
F.: Brainstormers 2002 - Team Description. In Birk, A., Coradeschi, S., Tadokoro,
S., eds.: RoboCup-2002: Robot Soccer World Cup VI, LNCS. Springer, Fukuoka,
Japan (2002)
Withopf, D., Riedmiller, M.: Comparing Different Methods to Speed-up Reinforcement
Learning in a Complex Domain. In: Proceedings of the International
Conference on Systems, Man, and Cybernetics, Big Island, USA (2003)
Riedmiller, M., Merke, A.: Using Machine Learning Techniques in Complex Multi-
Agent Domains. In Stamatescu, I., Menzel, W., Richter, M., Ratsch, U., eds.:
Adaptivity and Learning. Springer (2003)
Riedmiller, M., Merke, A., Nowak, W., Nickschas, M., Withopf, D.: Brainstormers
2003 - Team Description. In: RoboCup 2003: Robot Soccer World Cup VII, LNCS,
Padua, Italy, Springer (2003)
Riedmiller, M., Merke, A.,Withopf, D.: Brainstormers 2004 - Team Description. In:
Team Description Papers on CD-ROMfor Proceedings of RoboCup 2004. Springer,
Lisbon, Portugal (2004)
Gabel, T., Riedmiller, M.: CBR for State Value Function Approximation in Reinforcement
Learning. In: Proceedings of the 6th International Conference on
Case-Based Reasoning (ICCBR 2005), Chicago, USA, Springer (2005) 206–221
Withopf, D., Riedmiller, M.: Effective Methods for Reinforcement Learning in
Large Multi-Agent Domains. it - Information Technology Journal 47 (2005)
Riedmiller, M., Gabel, T., Knabe, J., Strasdat, H.: Brainstormers 2D — Team
Description 2005. In: RoboCup-2005: Robot SoccerWorld Cup IX, LNCS. Springer
(2005).
D. Withopf and M. Riedmiller:
Comparing different methods to speed-up reinforcement learning in a complex domain
In Proc. of the Int. Conference on Systems, Man and Cybernetics, 2005, Big Island, USA, October 2005
Gabel, T., Riedmiller, M.: Learning a Partial Behavior for a Competitive Robotic
Soccer Agent. KI Zeitschrift, KI 20(2), BöttcherIT Verlag, May 2006.
M. Riedmiller and T. Gabel and R. Hafner and S. Lange and M. Lauer:
Die Brainstormers: Entwurfsprinzipien lernfähiger autonomer Roboter
In Informatik-Spektrum 29(3), pages 175-190. Springer, June 2006.
M. Riedmiller and T. Gabel:
Brainstormers 2D - Team Description 2006
In G. Lakemeyer, E. Sklar, D. Sorrenti, T. Takahashi, editors, RoboCup 2006: Robot Soccer World Cup X, LNCS. Springer, 2006. (CD Supplement)
M. Riedmiller and T. Gabel:
Brainstormers 2D - Team Description 2007
In U. Visser, F. Ribeiro, T. Ohashi, F. Dellaert, editors, RoboCup 2007: Robot Soccer World Cup XI, LNCS. Springer, 2007. (CD Supplement)
M. Riedmiller and T. Gabel:
On Experiences in a Complex and Competitive Gaming Domain: Reinforcement Learning Meets RoboCup
In Proceedings of the 3rd IEEE Symposium on Computational Intelligence and Games (CIG 2007), pages 17-23. IEEE Press, April 2007.
T. Gabel, M. Riedmiller, and F. Trost:
A Case Study on Improving Defense Behavior in Soccer Simulation 2D: The NeuroHassle Approach
In L. Iocchi, H. Matsubara, A. Weitzenfeld, C. Zhou, editors, RoboCup 2008: Robot Soccer World Cup XII, LNCS. Springer, 2008.
M. Riedmiller, T. Gabel, F. Trost, T. Schwegmann:
Brainstormers 2D - Team Description 2008
In L. Iocchi, H. Matsubara, A. Weitzenfeld, C. Zhou, editors, RoboCup 2008: Robot Soccer World Cup XII, LNCS. Springer, 2008. (CD Supplement)
M. Riedmiller, T. Gabel, R. Hafner, and S. Lange.
Reinforcement Learning for Robot Soccer.
In Autonomous Robots, 27(1), pages 55-73. Springer, 2009.
T. Gabel, M. Riedmiller:
Brainstormers 2D - Team Description 2009
In Baltes, J.; Lagoudakis, M.G.; Naruse, T.; Shiry, S., editors, RoboCup 2009: Robot Soccer World Cup XIII, LNCS. Springer, 2009.
T. Gabel and M. Riedmiller.
On Progress in RoboCup: The Simulation League Showcase.
In RoboCup 2010: Robot Soccer World Cup XIV, Singapore. Springer, 2010.
