Re: [simspark-devel] Simspark Proxy Implementation

[Sander v. D. <sgv...@gm...>]

On Sun, Feb 24, 2013 at 3:26 PM, Stefan Glaser <gla...@gm...>wrote:

> Hey all,
>
> first of all, great to see some discussion going on again! Thank you for
> your great input!
>
> About security:
> [...]
>
> Though I agree it is important to think about security, what is the
benefit for a team in bypassing the proxy? It's purpose is to not
disadvantage a team in case of severe network delays and keep timing
constant, if a team circumvents it it only exposes itself to random timing.
In any case if it is an issue we can make the server (or the firewall)
enforce a maximum amount of connections, and have the proxy open that
amount when it starts.


> About providing results to the agents:
> What about holding the result file in the standard robocup user home
> (where it is probably written) and put a sym link to it in each home
> directory? So each agent should always find the actual result file in their
> home directory. Or are there restrictions/drawbacks in accessing such a
> file in the robocup user home?
>

That is an option. Maybe giving direct read access to the file would be
easier, as teams sometimes delete the symlinks from their directory on
accident.


> The only valid argument in my opinion for the RSG-format is that teams
> already have a parser. When using xml for example, it could be better
> visualized (e.g. via xsl as html-page), one could have validation of the
> document via DTD or Schema and there exist a lot of visual editors,
> enforcing and suggesting tags according to the DTD/Schema. On the other
> hand it may be a bit harder to edit directly from command line. But as
> Hedayat mentioned, we can provide different result formats later, also with
> respect to the Score Board Sander mentioned. Don't get me wrong, I don't
> want to exclude the RSG-format. I just wanted to mention, that it is not
> neccessarily the best choice for consistency and administration.
>

If something generates it as XML, we can also use XSLT to generate RSG from
it. What I wanted to say is, we just have to guarantee RSG will be there,
regardless of how it is generated. With that said, I do agree with the
benefits of XML. I'd also put JSON or YAML in the considerations, but any
can be generated from the other.


> Sander, do you have any documentation with respect to the RCSimControl?
>

Unfortunately not more than the in-code documentation at the moment. I am
working on it a bit more atm, I'll in any case make it so that Doxygen
generates something useful.


> Regards,
> Stefan
>
>
>
> Am 23.02.2013 18:58, schrieb Sander van Dijk:
>
>> Hey all,
>>
>> Few things from me to add to the discussion:
>>
>> - Regardless from implementation and discussion about which program should
>> be responsible, let's tackle first how teams access it. I for now am in
>> favour of files written somewhere that are accessible to all teams, e.g. a
>> special user's ('results') home dir that is readable for everybody. I
>> agree
>> with Hedayat that they should be in any case RSG. This would be the
>> easiest
>> for teams to implement, and we could even make them by hand in case we
>> don't manage to create the automated tools in time, which as OC makes me
>> most comfortable :).
>>
>> - Just FYI before too much redundant work is done, the organization for
>> Eindhoven is looking into using some competition wide centralized 'Score
>> Board' application that has been developed. I am awaiting more details on
>> what it actually is, but it looks like something that makes the gathering
>> and publication of results automatic, and perhaps we can plug into it.
>>
>> - Finally, I'd just like to point RCSimControl (
>> http://launchpad.net/**rcsimcontrol <http://launchpad.net/rcsimcontrol>),
>> which I developed during the RoboCup
>> funded project in which I also created the TBB multithreading for the
>> server. I always planned to use it to create a more advanced league
>> manager
>> as well (it controls multiple parallel instances; I use it to run a small
>> scale tournament for qualification on our super cluster). But development
>> of any other tools by somebody who actually has the time to invest in it
>> is
>> of course always welcome ;).
>>
>> Cheers,
>> Sander
>>
>>
>> On Mon, Feb 18, 2013 at 7:39 PM, Hedayat Vatankhah <hed...@gm...
>> >wrote:
>>
>>    Hi Stefan & Klaus,
>>> (and others! I thought that it is better to add this discussion to
>>> simspark-devel, hope that you don't mind!)
>>> About monitor, yes actually in all games that I'm in OC I do setup
>>> firewall rules to close all connections except the expected ones as
>>> Stefan
>>> said. But no, I didn't anything for communication between agents through
>>> sockets (however, I have tried some other restrictions like making the
>>> log
>>> files and directories write-only).
>>> I was not trying to imply that our security is perfect now, but security
>>> should be considered anyway. Also, the problem with the proxy could be
>>> probably much more 'useful' than communication between agents. I also
>>> didn't want to imply that we should not use proxy until it is completely
>>> secure. Even having a plan for security is great.
>>>
>>> Anyway, the security aspect is not necessarily complex. Stefan provided
>>> an
>>> idea, and actually it derived me to think of an even simpler idea... or
>>> actually that it is probably already secure enough! :P :D How? Well, we
>>> will run the proxy before the agents anyway, right? So, the proxy can
>>> make
>>> the expected number of connections to the simulator before running the
>>> agents. Now, we know that no more connections should be made to the
>>> simulator, so agents cannot connect to the simulator anyway. We can have
>>> a
>>> simple method to monitor the number of agents connected to proxies too
>>> (for
>>> example, league manager might query the number of agents and print them
>>> on
>>> screen or something). This probably have a little implementation work for
>>> the proxy too. However, Stefan ideas are great too, specially about the
>>> competition manager.
>>>
>>> About providing the results, RSG format is selected (I guess!) because
>>> agents already have an RSG parser, so they can easily parse them. Using
>>> another format for the agents mean that they should implement yet another
>>> parser. Therefore, I'm actually in favor of using RSG format for that.
>>> However, the results might be generated also in another format for other
>>> purposes.
>>>
>>> But, no, simspark should not be responsible for providing the results,
>>> and
>>> even if it is, it should get the results from another place. Actually,
>>> I've
>>> said previously (probably in TC ML or simspark-devel) that ideally we
>>> would
>>> need a kind of 'results server'. Since we run games in parallel, teams
>>> should be actually informed about all results not just the ones played in
>>> that cluster. There are even considerations such as the order and time of
>>> playing games in different clusters (there should be probably a
>>> 'schedule',
>>> and teams are informed about the results of the games which are past in
>>> schedule even if we were actually able to play some more games in the
>>> other
>>> cluster; or we should really run the games according to the schedule).
>>> Teams might query the results from the result server themselves, or
>>> rcssserver3d (which is certainly about competitions) can query them and
>>> provide it to agents. The results server can generate results in other
>>> formats usable for publishing, like using XML and XSL as Stefan
>>> mentioned.
>>>
>>> Finally, it might be more interesting if the competition manager can
>>> manage all games rather than the games running on a single cluster (in
>>> that
>>> case, it can actually play the role of the results server too). Consider
>>> each cluster as a soccer field, competition manager can schedule the
>>> games
>>> and then start the games on the selected fields. Or we might not consider
>>> clusters as fields and consider them completely equal, and dispatch each
>>> game when any cluster is available. Another policy could be to run all
>>> games of a single group on the same cluster to make sure that conditions
>>> are equal for each group. Anyway, when it dispatches a game, it might
>>> also
>>> provide the results for that game too. Well, this idea should not
>>> necessarily be implemented in the initial versions, but might be a goal
>>> which should be considered in design.
>>>
>>> Regards,
>>> Hedayat
>>>
>>>
>>> *Stefan Glaser <gla...@gm...> <gla...@gm...>* wrote on
>>>
>>> Mon, 18 Feb 2013 13:28:57 +0100:
>>>
>>> Hello together,
>>>
>>> sorry that I jump into the discussion, but I also spend some time in
>>> thinking of these problems.
>>>
>>> In general, the monitor issue you mentioned is not really a problem.
>>> Since
>>> the agent teams and the monitor usually run on different machines and the
>>> monitor port is fixed, a simple firewall rule should be sufficient to
>>> prevent this situation. The intermediate communication between agents
>>> though is tough.
>>>
>>> Now the proxy. The main target for me with the first version was simply
>>> to
>>> have a proxy. Since the proxy is only needed for competitions, I felt it
>>> should be transparent to both sides. Given this and that the current
>>> implementation uses random port assignments, there is no possibility for
>>> the server to distinguish connections from a proxy and connections coming
>>> directly from an agent. On the other hand, the current proxy
>>> implementation
>>> simply accepty new connections in an endless loop and has no information
>>> about the competition at all.
>>> In order to not touch the server code, I came up with the following idea.
>>> Extend the proxy to run (and maybe monitoring) the agent processes. If an
>>> agent is started by the proxy, the proxy can expect to get a connection
>>> request for each agent it started and can already reserve a connection to
>>> the server for it. If it doesn't get a connection request together with
>>> an
>>> init-message within let's say 10 seconds, the proxy simply kills the
>>> previously started agent again and prompts a warning. This way the game
>>> wouldn't start up and someone would notice that something is wrong.
>>> We could use the same starting scripts for the agents (just not starting
>>> a
>>> whole team, instead only one agent), but we would need to run a proxy
>>> instance for each match, with some more parameters. Should be not too
>>> much
>>> of a change in the depending scripts.
>>>
>>>
>>> In case I find time to develop the competition manager GUI I have in mind
>>> (an advanced version of the league manager), my target is to use an own
>>> interface to the proxy server to run/stop/monitor agent processes and
>>> their
>>> network statistics. This way a proxy can stay running the whole day and
>>> is
>>> instructed directly from the competition manager GUI.
>>> When talking about the competition manager GUI, I also thought about
>>> providing the current results of the competition to the agents. I wonder
>>> why the SimSpark server should take care of this and why the RSG format
>>> should be used in this case. When thinking of the SimSpark server as a
>>> tool
>>> for research, providing competition abilities is just not needed and
>>> irrelevant i.e. for learning. In my opinion, if we play a competition,
>>> the
>>> server is only responsible for simulating the games, not for organizing
>>> them. If we intend to have a central file, containing the whole
>>> competition
>>> results, why not using an xml-file, which can be easily published and
>>> displayed as html-page to humans, using xsl? And if this file already
>>> exists, why not publishing it over the internet?
>>> All this could be addressed by a separate application, providing a nice
>>> GUI to manage and monitor competitions. This GUI can also be used to
>>> display the results for visitors at Robocup events. By integrating i.e.
>>> RoboViz, we could even watch the games directly in the competition
>>> manager
>>> and play logfiles of the best scenes to amuse visitors. And, think about
>>> a
>>> team which want to test their new agent binary at a competition. They
>>> open
>>> the competition manager in training mode and can either start a game
>>> against another own binary, or ask other teams to enter their user
>>> password
>>> in order to allow the use of their binary for one test match. Test
>>> matches
>>> are then automatically played under competition conditions. This would
>>> speed up test-time of teams when setting up things on the competition
>>> machines for test games.
>>>
>>> I'm really looking forward to develop such a competition manager. I think
>>> this would be a great contribution to the Robocup events, which I can
>>> easily provide, since I'm familiar with most technical stuff involved.
>>> With
>>> this I could take care of providing the actual results of the competition
>>> to the agents.
>>> I'm not sure yet how I'll find time. But I'm kind of eager to start ;)
>>>
>>> What do you think?
>>>
>>> Sorry, got quite a long mail meanwhile. I intend to put a description of
>>> my thoughts into the wiki, too.
>>>
>>> Wish you all the best,
>>> Stefan
>>>
>>>
>>>
>>>
>>>
>>> Am 18.02.2013 08:53, schrieb Klaus Dorer:
>>>
>>> Hi Hedayat,
>>>
>>> hmmm, security aspects are important. But are we checking that no agent
>>> is
>>> connecting as monitor? Or are we checking that no agent opens a socket to
>>> another agent? I think no. Therefore I wonder if we have to be more
>>> strict
>>> here. If yes, is this something we have to do on application level, or
>>> can
>>> we do it on a different level, e.g. opening that port only for a specific
>>> application?
>>>
>>> Greetings
>>> Klaus
>>>
>>> Am 16.02.2013 19:20, schrieb Hedayat Vatankhah:
>>>
>>> Hi Stefan,
>>> Sorry for my lack of response. I'm still planning to put some time to
>>> answer your first email (but in brief, the security aspects look more
>>> complicated than the proxy itself. For example, agents must be unable to
>>> impersonate themselves as the proxy(ies)).
>>> And thank you a lot for reporting on the progress. I'll certainly look
>>> into the wiki page and will provide my comments (if any!). :)
>>>
>>> Thanks a lot.
>>> Hedayat
>>>
>>>
>>> /*Stefan Glaser <gla...@gm...> <gla...@gm...>*/ wrote
>>>
>>> on Sat, 16 Feb 2013 19:13:50 +0100:
>>>
>>> Hello Hedayat,
>>>
>>> I was eager to implement something, so I spend some time implementing the
>>> proxy server for SimSpark.
>>> It was quite simple and straight forward to implement and first tests
>>> look
>>> good. We intend to test the proxy over the internet, just to get a
>>> subjective impression ;) In addition we also want to test it on a local
>>> network, monitoring the traffic to the server with wireshark, to get some
>>> numbers. Once that is done, I would try to release it somehow.
>>>
>>> If you are interested in the concept and want to get some information
>>> until we release the source code, I already put a description into the
>>> wiki:
>>> http://simspark.sourceforge.**net/wiki/index.php/Agent_Proxy<http://simspark.sourceforge.net/wiki/index.php/Agent_Proxy>
>>>
>>> Wish you all the best,
>>> Stefan
>>>
>>>
>>> Am 13.02.2013 14:10, schrieb Stefan Glaser:
>>>
>>> Hello Hedayat,
>>>
>>> as Klaus mentioned, we would take care of implementing the proxy approach
>>> for the next competitions.
>>>
>>> We would realize it in Java - I hope that doesn't result in any
>>> problem...
>>>
>>> Now, appart from informing you about that we would take care of the
>>> implementation, I wanted to ask you about some thoughts with respect to
>>> the
>>> proxy. The basic idea is clear, having an intermediate component,
>>> measuring
>>> the time on the client machines. But I wonder, is there anything else
>>> critical, which one has to be aware of in the implementation? E.g. some
>>> situation which a team can explore in order to gain benefits over other
>>> teams?
>>>
>>> Cheers,
>>> Stefan
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------**------------------------------**
>>> ------------------
>>> The Go Parallel Website, sponsored by Intel - in partnership with
>>> Geeknet,
>>> is your hub for all things parallel software development, from weekly
>>> thought
>>> leadership blogs to news, videos, case studies, tutorials, tech docs,
>>> whitepapers, evaluation guides, and opinion stories. Check out the most
>>> recent posts - join the conversation now.
>>> http://goparallel.sourceforge.**net/<http://goparallel.sourceforge.net/>
>>> ______________________________**_________________
>>> Simspark Generic Physical MAS Simulator
>>> simspark-devel mailing list
>>> simspark-devel@lists.**sourceforge.net<sim...@li...>
>>> https://lists.sourceforge.net/**lists/listinfo/simspark-devel<https://lists.sourceforge.net/lists/listinfo/simspark-devel>
>>>
>>>
>>>
>>
>


-- 
Adaptive Systems Research Group
Department of Computer Science
University of Hertfordshire
United Kingdom