simspark-devel

Re: [simspark-devel] pixel camera perceptor

[Feng X. <hen...@ma...>]

Hi Joschka,

    It is a good idea! This makes our simulation much more realistic!

    Howerever, it may be hard for teams' work. Actually, I have talked
about this with Yuan some days ago and we both think that most of the teams
will not support this.

    But, how about provide the pixel camera as a additional perceptor and 
weaken the original vision perceptor? And next year, pure pixel camera?

Cheers!



Feng Xue
2008-04-10



发件人： Joschka Boedecker
发送时间： 2008-04-10 19:26:21
收件人： sim...@li...
抄送： 
主题： [simspark-devel] pixel camera perceptor

Hi all,
I've been thinking about how to implement a pixel camera perceptor for  
Simspark using hardware-accelerated OpenGL Render to Texture via the  
Frame Buffer Object extension (see, e.g. [1]). I'm not proposing we  
should adopt this in the 3D league, but it will be handy for doing  
research.
One problem I see is that this is a sensor, but it relies on rendering  
also. In the multi-threading implementation, these two are seperated  
as far as I remember, right? Maybe it is not a problem, since the  
rendering only needs to produce the image, which can be copied in the  
thread for the perceptors, but I'm not sure about this, so I thought  
I'll ask for ideas here.
Another thing I've been wondering about is whether it would be worth  
it to compress the images before sending it to the agent or not. Of  
course this will depend on color depth and size of the image, as well  
as network delay, but encoding (and decoding on the agent side) would  
also take time. For small images and a fast local network without much  
delay, it might be overkill I guess. Again, any ideas are welcome.
Cheers,
Joschka
[1] http://www.gamedev.net/reference/articles/article2331.asp
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Simspark Generic Physical MAS Simulator
simspark-devel mailing list
sim...@li...
https://lists.sourceforge.net/lists/listinfo/simspark-devel

Re: [simspark-devel] pixel camera perceptor

[Sander v. D. <sgv...@gm...>]

Hey all,

>From the point of view of RoboCup I myself am more in favor of adding extra
information to the vision perceptor like a rotation matrix. In my view the
3D simulation should offer things that the real leagues can't handle yet. At
the moment it might be unreal for humanoid robots to perceive body posture
and orientation, but I think this should be possible in the future.

On the other hand, a pixel camera is realistic and is good from a generic
simulator point of view (and perhaps an entertainment point of view during
matches). However for the RoboCup 3D league I'd say it's a step back, why
have our teams mess with a problem that all other leagues are already
working on.

In the discussion of simspark 'vs' webots it might have a good thing to have
though, because it seemed like one of the things why webots is perceived as
more advanced.

So my ideas are as follows:
- The current vision perceptor should be expanded/restricted: restrict field
of vision and like Joschka said, make the robot's head movable. Add vision
noise again. For agents to be able to have a higher level strategy
concerning teammates and opponents, vision objects should atleast get a
rotation matrix relative to the agent and preferably the data of other
robots should be expanded. Full position of all body parts might be a bit
too much data, perhaps just torso, head, feet and hands.

- The pixel camera should be implemented in such a way that the pixel data
can be received by the agent and also by the monitor, so we can show cool
first person action during matches. Yuan's idea to make a seperate camera
server seems like a solution for this, and also I think the possibility of
doing this extra rendering on seperate machines is nice. However, somehow it
doesn't seem pretty in my head, but for the moment I don't have a better
idea yet :)

Regards,

Sander

On Thu, Apr 10, 2008 at 3:14 PM, Yuan Xu <xuy...@gm...> wrote:

> Hi again,
>
> I think the vision can not get rotation matrix, it is unreal!
> We should consider the results after vision processing in real robots.
> Then it will be easier for teams when using pixel image perceptor.
>
> > Yes, good point. I think we could have a camera that has a restricted
> field
> > of view (omni-vision cameras are banned even in the real humanoid league
> > from this year on), but that will be tied to the orientation of it's
> parent
> > body, so that it can be changed with joint effectors, and we don't need
> any
> > camera effectors for that. This is what we could use for Nao then.
> >
> > Also, there were some comments that the vision information that is sent
> to
> > the agents should be extended, but it is not clear to me yet what should
> be
> > included. Some people suggested sending a rotation matrix (or a
> quaternion)
> > for the objects that an agent sees, for instance. What do you guys think
> > would be most useful?
> >
>
> --
> Best wishes!
>
> Xu Yuan
> School of Automation
> Southeast University, Nanjing, China
>
> mail:   xuy...@gm...
>          xy...@ya...
> web:   http://xuyuan.cn.googlepages.com
> --------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Simspark Generic Physical MAS Simulator
> simspark-devel mailing list
> sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simspark-devel
>

Re: [simspark-devel] pixel camera perceptor

[Markus R. <rol...@un...>]

Hi,

Sander van Dijk wrote:
[...]
> On the other hand, a pixel camera is realistic and is good from a 
> generic simulator point of view (and perhaps an entertainment point of 
> view during matches). However for the RoboCup 3D league I'd say it's a 
> step back, why have our teams mess with a problem that all other leagues 
> are already working on.

I agree with Sander that switching to the pixel camera is a step back as 
it raises the bar for teams to do basic things like self localization. 
In order to get more strategic and interesting matches staying with the 
VisionPerceptor would be better.

But having some nice 'first robot' view would be nice nevertheless ;)

[...]
> - The pixel camera should be implemented in such a way that the pixel 
> data can be received by the agent and also by the monitor, so we can 
> show cool first person action during matches. Yuan's idea to make a 
> seperate camera server seems like a solution for this, and also I think 
> the possibility of doing this extra rendering on seperate machines is 
> nice. However, somehow it doesn't seem pretty in my head, but for the 
> moment I don't have a better idea yet :)

In order to provide some first person robot views in the monitor there 
is no need for any server support. The additional viewport can be 
entirely rendered in the monitor process.

I also like Yuans idea to offload the rendering to some external process.

cheers,
Markus

Re: [simspark-devel] pixel camera perceptor

[Joschka B. <jos...@am...>]

Hi Feng,

thanks for your quick reply.

On Apr 10, 2008, at 8:48 PM, Feng Xue wrote:

> It is a good idea! This makes our simulation much more realistic!
>
>     Howerever, it may be hard for teams' work. Actually, I have talked
> about this with Yuan some days ago and we both think that most of  
> the teams
> will not support this.

You're probably right. But I don't want to suggest it for the league's  
competitions anyways (at least not for this year), it's more for doing  
research with robots that need realistic vision information.

>  But, how about provide the pixel camera as a additional perceptor

Exaclty, that's what I had in mind.

> and weaken the original vision perceptor?

Yes, good point. I think we could have a camera that has a restricted  
field of view (omni-vision cameras are banned even in the real  
humanoid league from this year on), but that will be tied to the  
orientation of it's parent body, so that it can be changed with joint  
effectors, and we don't need any camera effectors for that. This is  
what we could use for Nao then.

Also, there were some comments that the vision information that is  
sent to the agents should be extended, but it is not clear to me yet  
what should be included. Some people suggested sending a rotation  
matrix (or a quaternion) for the objects that an agent sees, for  
instance. What do you guys think would be most useful?

> And next year, pure pixel camera?

>

That's something that should be discussed on the sserver-three-d list  
and in the TC, I guess. I can imagine that many people will be against  
it (even though it is easy to use standard vision solutions like  
Intel's OpenCV, etc.). This is related to a much deeper question of  
clearly defining the goals of the 3D Soccer Simulation League I  
guess ;-)

Cheers,
Joschka

Re: [simspark-devel] pixel camera perceptor

[Yuan X. <xuy...@gm...>]

Hi again,

I think the vision can not get rotation matrix, it is unreal!
We should consider the results after vision processing in real robots.
Then it will be easier for teams when using pixel image perceptor.

> Yes, good point. I think we could have a camera that has a restricted field
> of view (omni-vision cameras are banned even in the real humanoid league
> from this year on), but that will be tied to the orientation of it's parent
> body, so that it can be changed with joint effectors, and we don't need any
> camera effectors for that. This is what we could use for Nao then.
>
> Also, there were some comments that the vision information that is sent to
> the agents should be extended, but it is not clear to me yet what should be
> included. Some people suggested sending a rotation matrix (or a quaternion)
> for the objects that an agent sees, for instance. What do you guys think
> would be most useful?
>

-- 
Best wishes!

Xu Yuan
School of Automation
Southeast University, Nanjing, China

mail:   xuy...@gm...
          xy...@ya...
web:   http://xuyuan.cn.googlepages.com
--------------------------------------------------