[simspark-devel] Fwd: 6 Months research practicum on Simspark, ideas and suggestions

[Patrick G. <Pat...@on...>]

Hello,

my name is Patrick, I'm a 24 year old Masters student from the 
University of Koblenz (Germany), currently studying at the Osaka 
University for 6 months. Here I am going to work on the Simspark 
environment within the context of a "research" practicum under the 
watchful eyes of Joschka Boedecker. (Actually it wont be "researching", 
but I will be focusing on solely one project)

My major subject is Computational Visualistics, so my plans for 
developing on Simspark are mostly improving usability and/or graphical 
presentation. From the Todo- and wishlist in the Wiki I could target (or 
at least support) some of:
- New interface for different rendering engines, add OGRE support
- Enable new agent-server protocol to support using image sensor and 
other things via new rendering engine.
- GUI for our simulator (possibly based on RsgEdit + ZigoratAssistant, 
incl. easy construction of new robots and simulations, moving robots on 
the field, etc.)
- Support Referee enhancements and trainer enhancements: (Drop a player 
anywhere on the field. Drop the ball at any place (as drop ball, free 
kick left or right) as is possible in 2D. Play Mode Control etc..Control 
e.g. via Monitor)
- Nice 3D visualization, including a stadium, animated crowd, automatic 
commentator, slow-motion replay, etc.
- Training Capabilities through agent-monitor communication

As i have about five and a half months of time I could probably connect 
some of these tasks, but of course i will have to focus. For example 
building a User Interface for Server, Monitor and Client Support (maybe 
like WeBots) along with the possibilty to connect new modules for a 
RsgEditor and new functionality would be possible.
At the same time, Peter Tissen (a friend of mine who is working here 
too) is implementing Bullet as a pluggable Physics Engine optionally 
choosable for the Simspark Simulation.

First of all, to me it is most important to get feedback by those, who 
are already experienced with Simspark. What is needed? What would be 
nice to have/doable in this amount of time? What is definitely NOT 
needed? Whatever will be done has to be accepted and integrated by the 
community, so it can be of use and of course can be maintained and 
extended in the future by everyone. So to begin with, in the following I 
have a list of ideas and questions you may comment with whatever comes 
to your mind. The more detailed, the better.**

*What should a central user interface contain*

- starting server, monitor and agents
- controlling the server (game control, message overview, physics 
engine, log control)
- controlling the monitor (views, rendering engine, video captures, game 
camera)
- controlling the agents (message overview, message sending?, debug 
interface?)
- creating and testing agents? (loading from collada?, annotating with 
receptors and effectors?, coding?, visualize behaviour? control interface?)
- be extendable to incorporate rsgeditor, further monitors, agent 
visualizations...
/
Questions/: Is running most agents as simspark plugin possible? Or is it 
better to offer some kind of debug messaging interface? How are agents 
debugged in practice and could a central user interface working together 
with server and visualization (see next point) be of help?

My preference for an easy-to-create and -extend up-to-date GUI is 
definitely QT.

*What should an improved graphics engine/monitor be able to do?*

- Visualizing server or agent internal states? (view of agents, rule 
checking, effector usage, agent state visualization)
- Improve Debugging/Logging by server and/or agents with a documentation 
interface (visualizing and recording on request)
- graphics engine choosable? (currently built-in as fallback, 
Ogre/Irrlicht as improvement). e.g. bind ogre to core scene graph via 
bridge pattern.
- Shading, 1st alternative: GLSL Shading environment attached to ogre
- pro: multiple render targets -> realistic lighting, improved 
materials, point of view rendering for every agent, realistic sight 
detection
-> implementation of Screen Space Lighting
-> implementation of Screen Space Ambient occlusion
- pro: Further rendering engines besides ogre can be connected the same way
     - con: bound to Ogre. When the interface should fail somehow, there 
wouldnt be any improved shading anymore.
- Shading, 2nd alternative: basic GLSL Shading environment attached to 
the monitors core, improving current indepandant shading ( allowing MRT 
Shading too)
     - pro: lightweight, same shading options like 1st option
     - pro: easy and fast to implement, so i will have time for other 
things
     - pro: not dependant to ogre -> one needs no knowledge about ogre
     - con: developing new shaders will be harder, as no combination is 
possible natively
     - con: lacking ogres support for Object loading, environmental 
animation, scenegraph, materials etc.
- Object Shadows
     - simple object shadows (blob under objects)-> easy, with ogre for free
     - directional shadow for each object -> easy because of flat ground,
     - shadow mapping (allow shadows on objects)-> time consuming).
- Inner Shadows (Screen Space Ambient Occlusion, SSAO): only possible 
with shader integration
- additional objects to render: seats, crowd, goal net, corner flags 
(further objects would be easier when using ogre)
- graphics improvement: textures, sky, weather
- environment simulation (wind on grass, changing sky...)

The main problem is how to improve the current monitor or create an 
alternative one, that is easy to use and (most important) easy to 
maintain. Various new possibilities like visual debugging and recording 
of server internal states (rcssserver3d) and the agents could be 
integrated into the visualization (or maybe better the GUI). I guess 
since rewriting the monitor from scratch could lead to discarding any 
new implementation with the stable one after im done, it my be better to 
make Ogre a pluggable option to the current monitor. New possibilities 
for the monitor would most likely create the need of extending the 
Server/Monitor MessageProtocol. Both implementing Ogre as pluggable 
rendering engine and simply improving the engine already in use are 
viable options. However, while creating a simple (yet good looking) 
improved rendering style for the Monitor is an easy task, connecting 
Ogre to the scene graph is not.

So the Question: Is it better to simply implement a better 
visualization, leaving much more time for graphical gimmicks, the GUI or 
debug/point-of-view visualization and so on, or is it preferable to 
create a flexible graphics-interface for the monitor and implement Ogre 
as the first one, even if there wont be much more time for the actual 
visualization and possibly more important tasks (central gui).

This said for the two possible development fields... Again, the most 
important thing is the acceptance in the community. I definitely dont 
want to create an interface or program that cannot be used or maintained 
in the future, should I not be able to continue.
Please send me E-Mails with comments, ideas, suggestions and anything 
helpful you might come up with.

Thanks a lot ;)

Patrick Geib