From: Hesham <hes...@gm...> - 2008-01-27 13:35:47
|
Hi all, After profiling the server I saw SDL_GetTicks() is taking around 25% of the time (in the tests I used 7 robots and with the single thread mode). To make long short, I saw this loop (while) in simulationserver.cpp: void SimulationServer::Run(int argc, char** argv) { .... if (mAutoTime) { AdvanceTime(mSimStep); cout<<"AutoTime"<<endl; } else { while (int(mSumDeltaTime*100) < int(mSimStep*100)) { inputCtr->StartCycle();// advance the time } } .... Since SimulationServer::Step() takes care of this case (int(mSumDeltaTime*100) < int(mSimStep*100)) I thought that while in Run() is not necessary. So after commenting it out, SDL_GetTicks() takes less than 10% of the time: void SimulationServer::Run(int argc, char** argv) { .... if (mAutoTime) { AdvanceTime(mSimStep); cout<<"AutoTime"<<endl; } else { inputCtr->StartCycle();// advance the time } .... BTW, the other day I came across the multi-threaded version of ODE by Intel. I think it's worth a try if Yuan has time to help :-) There is something wrong with the multi-threaded mode of the server on my system. So first I need to find out what's the problem. Bests, Hesham |
From: Yuan X. <xuy...@gm...> - 2008-01-27 14:31:49
|
Hi Hesham and all, > After profiling the server I saw SDL_GetTicks() is taking around 25% of the > time (in the tests I used 7 robots and with the single thread mode). To make > long short, I saw this loop (while) in simulationserver.cpp: > .... > Since SimulationServer::Step() takes care of this case > (int(mSumDeltaTime*100) < int(mSimStep*100)) I thought that while in Run() > is not necessary. So after commenting it out, SDL_GetTicks() takes less than > 10% of the time: The phenomenon is reasonable. Since the time queries are needed for timing in single thread, the percentage of queries means that your machine are light underload( not too busy ), I guess your robots did not do anything. You will notice that the percentage of queries will lower if robots are collides... But if you remove the codes, the timer is noneffective. Then if the robots collides, the server will very very slow, otherwise the simulation time elapse very quickly. You may ask why use query to get time, ok, I do not think it is a good idea. The server cycle is 20ms, but the time precision of Linux is only 10ms, some function such as "sleep" also can not help on it. In fact, SDL_GetTicks() is used to get "preciser"(seemly). And it is possibly that SDL causes the Input problem in multi-threads. I have some idea to this: 1. to get precise time, use RealTime-Linux as platform [It will narrow OS platform] 2. use a new thread to time, The timing is in InputControl thread in current multi-thread implementation, but query is also used(It can be improved little). [It may make the timing-thread busy, and can not guarantee precision.] 3. use other timer instead SDLTimer, such as boost::xtime. [It need more investigation.] > BTW, the other day I came across the multi-threaded version of ODE by Intel. Sounds interesting. > I think it's worth a try if Yuan has time to help :-) There is something > wrong with the multi-threaded mode of the server on my system. So first I > need to find out what's the problem. I am free recently. Give some hints of your problem. -- Best wishes! Xu Yuan School of Automation Southeast University, Nanjing, China mail: xuy...@gm... xy...@ya... web: http://xuyuan.cn.googlepages.com -------------------------------------------------- |
From: Hedayat V. <hed...@ai...> - 2008-01-27 19:57:46
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> <span>Hi,<br> First, I think that the loop isn't a problem too. And I wonder if SDL_GetTicks() creates any major problem now.<br> But the multi-threaded ODE might be really helpful. <br> <br> Second, would you please how do you profile the server? Each of us gets different results and I wonder why is so. Also, for the profiling results I provided in my previous mail I used valgrind's tools. As it is really slow, I used just one robot in those tests. Please tell me about the ways you use for profiling the server.<br> <br> Finally, It would be really nice if you tell me your opinions or the mistakes in the patch I sent with my email recently(in the email with subject: "some experiments with the server"). Also, I'm really interested to know about the answer of the 4th question.<br> <br> Thanks anyway,<br> Hedayat<br> <br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>"Yuan Xu" <a class="moz-txt-link-rfc2396E" href="mailto:xuy...@gm..."><xuy...@gm...></a></b></i> wrote on 01/27/2008 06:01:45 PM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">Hi Hesham and all, </pre> <blockquote type="cite"> <pre wrap="">After profiling the server I saw SDL_GetTicks() is taking around 25% of the time (in the tests I used 7 robots and with the single thread mode). To make long short, I saw this loop (while) in simulationserver.cpp: .... Since SimulationServer::Step() takes care of this case (int(mSumDeltaTime*100) < int(mSimStep*100)) I thought that while in Run() is not necessary. So after commenting it out, SDL_GetTicks() takes less than 10% of the time: </pre> </blockquote> <pre wrap=""><!----> The phenomenon is reasonable. Since the time queries are needed for timing in single thread, the percentage of queries means that your machine are light underload( not too busy ), I guess your robots did not do anything. You will notice that the percentage of queries will lower if robots are collides... But if you remove the codes, the timer is noneffective. Then if the robots collides, the server will very very slow, otherwise the simulation time elapse very quickly. You may ask why use query to get time, ok, I do not think it is a good idea. The server cycle is 20ms, but the time precision of Linux is only 10ms, some function such as "sleep" also can not help on it. In fact, SDL_GetTicks() is used to get "preciser"(seemly). And it is possibly that SDL causes the Input problem in multi-threads. I have some idea to this: 1. to get precise time, use RealTime-Linux as platform [It will narrow OS platform] 2. use a new thread to time, The timing is in InputControl thread in current multi-thread implementation, but query is also used(It can be improved little). [It may make the timing-thread busy, and can not guarantee precision.] 3. use other timer instead SDLTimer, such as boost::xtime. [It need more investigation.] </pre> <blockquote type="cite"> <pre wrap="">BTW, the other day I came across the multi-threaded version of ODE by Intel. </pre> </blockquote> <pre wrap=""><!----> Sounds interesting. </pre> </blockquote> <br> </body> </html> |
From: Hesham <hes...@gm...> - 2008-01-27 21:38:37
|
Hi Hedayat and Yuan, Hedayat, sorry I didn't find time to reply your email earlier. Regarding that interface for the physics engines, it sounds a good idea to either use it directly or write something like that for the server. But apparently it's a good idea to first test the server with multi-threaded ODE. I was reading Intel Threading Building Blocks book, you can find the first chapter at: http://softwarecommunity.intel.com/isn/downloads/intel_tbb_ch01_for_promo.pdf In chapter 11 they explain two methods and mention the complete code is on the website, I tried to find it but so far no luck. (If you don't have access to the book, let me know I can send you that section.) I'm using the Google profiler(/sampler), it's pretty straightforwards. I tried to write about it on the wiki but it asked me to login, and using my sorceforge ID/password it failed. Anyway I'll write it within this week and email it to you, so we can use the same method. And to me it makes sense that we get different results(/hot spots), because of different systems (HW&SW) we use. As far as I know a good example is ATLAS (math-atlas.sf.net), using scripts generates the optimized code for different systems. Next weekend I'll try to work more on these issues, and probably until then will know the answer of your 4th question. Yuan, the code of SDL_GetTicks() is simple, it doesn't give any more precision. I didn't get some parts of what you said. Maybe a silly question but, the server is supposed to have sharp 20ms cycles? I thought the previous timing method (SPADES) was replaced by the new one to get rid of the problems we had with timers, and more rely on time stamps. But you suggested using RTL to have more precise timers. Sorry maybe I got the whole thing wrong, I'll go through the code again. But I would appreciate it if you (or others) could please remind me the new idea for timing, maybe by referring to old emails - I couldn't find them. Thanks, Hesham On 27/01/2008, Hedayat Vatankhah <hed...@ai...> wrote: > > Hi, > First, I think that the loop isn't a problem too. And I wonder if > SDL_GetTicks() creates any major problem now. > But the multi-threaded ODE might be really helpful. > > Second, would you please how do you profile the server? Each of us gets > different results and I wonder why is so. Also, for the profiling results I > provided in my previous mail I used valgrind's tools. As it is really slow, > I used just one robot in those tests. Please tell me about the ways you use > for profiling the server. > > Finally, It would be really nice if you tell me your opinions or the > mistakes in the patch I sent with my email recently(in the email with > subject: "some experiments with the server"). Also, I'm really interested to > know about the answer of the 4th question. > > Thanks anyway, > Hedayat > > *"Yuan Xu" <xuy...@gm...> <xuy...@gm...>* wrote on > 01/27/2008 06:01:45 PM: > > Hi Hesham and all, > > > After profiling the server I saw SDL_GetTicks() is taking around 25% of the > time (in the tests I used 7 robots and with the single thread mode). To make > long short, I saw this loop (while) in simulationserver.cpp: > > .... > Since SimulationServer::Step() takes care of this case > (int(mSumDeltaTime*100) < int(mSimStep*100)) I thought that while in Run() > is not necessary. So after commenting it out, SDL_GetTicks() takes less than > > 10% of the time: > > The phenomenon is reasonable. > Since the time queries are needed for timing in single thread, > the percentage of queries means that your machine are light underload( > not too busy ), > I guess your robots did not do anything. You will notice that the > > percentage of queries will lower if robots are collides... > But if you remove the codes, the timer is noneffective. Then if the > robots collides, the server will very very slow, otherwise the > simulation time elapse very quickly. > > You may ask why use query to get time, ok, I do not think it is a good idea. > The server cycle is 20ms, but the time precision of Linux is only 10ms, > some function such as "sleep" also can not help on it. > > In fact, SDL_GetTicks() is used to get "preciser"(seemly). > And it is possibly that SDL causes the Input problem in multi-threads. > > I have some idea to this: > 1. to get precise time, use RealTime-Linux as platform [It will > > narrow OS platform] > 2. use a new thread to time, The timing is in InputControl thread in > current multi-thread implementation, but query is also used(It can be > improved little). [It may make the timing-thread busy, and can not > > guarantee precision.] > 3. use other timer instead SDLTimer, such as boost::xtime. [It need > more investigation.] > > BTW, the other day I came across the multi-threaded version of ODE by Intel. > > Sounds interesting. > > > |
From: Yuan X. <xuy...@gm...> - 2008-01-28 04:23:04
|
Hi all, > > Next weekend I'll try to work more on these issues, and probably until then > will know the answer of your 4th question. > > 4) What's the reason for stepping in discreet steps in SimulationServer::Step()? > Will it change anything with ODE? Yes, dWorldStep(20ms)+dWorldStep(20ms) != dWorldStep(40ms). Because, the collision detection is missing in dWorldStep(40ms). > Yuan, the code of SDL_GetTicks() is simple, it doesn't give any more > precision. I didn't get some parts of what you said. Yes, the precision depends on the OS, then I remind the RTL, and I know it is not suitable. > Maybe a silly question > but, the server is supposed to have sharp 20ms cycles? The cycle duration effects the performance of robot controller. Note that the control cycle is less than 10ms in real robot. The longer cycle the harder for the controller. And the HMDP[Joschka] may solve this, but it will make teams to change their codes a lot. > I thought the > previous timing method (SPADES) was replaced by the new one to get rid of > the problems we had with timers, and more rely on time stamps. But you > suggested using RTL to have more precise timers. The SPADES do not count the real time, it counts how many jiffies used by the agent. It can measure the CPU cost of the agent, but if the agent spend a lot of time on other work, such as communication, writing logs, the game will be very slow. Jelle Kok said that some games last less than 10min, but some other games need more than 1 hour! > Sorry maybe I got the whole > thing wrong, I'll go through the code again. But I would appreciate it if > you (or others) could please remind me the new idea for timing, maybe by > referring to old emails - I couldn't find them. > The timer was designed just like the 2D server: fixed cycle duration, the agent should sent action in time, otherwise the action will not be executed. The difference between 2D and 3D only are the duration and implementation. Best wishes! Xu Yuan |
From: Hedayat V. <hed...@ai...> - 2008-01-29 20:01:29
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> Hi Hesham,<br> Thanks for your answer. I'm agree with you about experimenting with a multi-threaded ODE. In fact, I searched for a multi-threaded ode and I found a paper but no code (but it was not what you mentioned). I will search more and I hope that we can find it.<br> I'll try to download the pdf and will notify you if I can't get it. Thanks.<br> Also, I'll try the google profiler too. <br> <br> Have a nice time,<br> Hedayat<br> <span><br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>Hesham <a class="moz-txt-link-rfc2396E" href="mailto:hes...@gm..."><hes...@gm...></a></b></i> wrote on 01/28/2008 01:08:32 AM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:1d7...@ma..." type="cite"><br> Hi Hedayat and Yuan,<br> <br> Hedayat, sorry I didn't find time to reply your email earlier. Regarding that interface for the physics engines, it sounds a good idea to either use it directly or write something like that for the server. But apparently it's a good idea to first test the server with multi-threaded ODE. I was reading Intel Threading Building Blocks book, you can find the first chapter at: <a moz-do-not-send="true" href="http://softwarecommunity.intel.com/isn/downloads/intel_tbb_ch01_for_promo.pdf" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://softwarecommunity.intel.com/isn/downloads/intel_tbb_ch01_for_promo.pdf</a><br> In chapter 11 they explain two methods and mention the complete code is on the website, I tried to find it but so far no luck. (If you don't have access to the book, let me know I can send you that section.)<br> <br> I'm using the Google profiler(/sampler), it's pretty straightforwards. I tried to write about it on the wiki but it asked me to login, and using my sorceforge ID/password it failed. Anyway I'll write it within this week and email it to you, so we can use the same method. And to me it makes sense that we get different results(/hot spots), because of different systems (HW&SW) we use. As far as I know a good example is ATLAS (<a moz-do-not-send="true" href="http://math-atlas.sf.net">math-atlas.sf.net</a>), using scripts generates the optimized code for different systems.<br> <br> Next weekend I'll try to work more on these issues, and probably until then will know the answer of your 4th question.<br> <br> Yuan, the code of <span>SDL_GetTicks() is simple, it doesn't give any more precision. I didn't get some parts of what you said. Maybe a silly question but, the server is supposed to have sharp 20ms cycles? I thought the previous timing method (SPADES) was replaced by the new one to get rid of the problems we had with timers, and more rely on time stamps. But you suggested using RTL to have more precise timers. Sorry maybe I got the whole thing wrong, I'll go through the code again. But I would appreciate it if you (or others) could please remind me the new idea for timing, maybe by referring to old emails - I couldn't find them.<br> <br> Thanks,<br> Hesham<br> </span><br> <div><span class="gmail_quote">On 27/01/2008, <b class="gmail_sendername">Hedayat Vatankhah</b> <<a moz-do-not-send="true" href="mailto:hed...@ai..." target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">hed...@ai...</a>> wrote:</span> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div style="direction: ltr;" bgcolor="#ffffff" text="#000000"><span>Hi,<br> First, I think that the loop isn't a problem too. And I wonder if SDL_GetTicks() creates any major problem now.<br> But the multi-threaded ODE might be really helpful. <br> <br> Second, would you please how do you profile the server? Each of us gets different results and I wonder why is so. Also, for the profiling results I provided in my previous mail I used valgrind's tools. As it is really slow, I used just one robot in those tests. Please tell me about the ways you use for profiling the server.<br> <br> Finally, It would be really nice if you tell me your opinions or the mistakes in the patch I sent with my email recently(in the email with subject: "some experiments with the server"). Also, I'm really interested to know about the answer of the 4th question.<br> <br> Thanks anyway,<br> Hedayat<br> <br> <i><b>"Yuan Xu" <a moz-do-not-send="true" href="mailto:xuy...@gm..." target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"><xuy...@gm...></a></b></i> wrote on 01/27/2008 06:01:45 PM:</span> <div><span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" type="cite"> <pre>Hi Hesham and all, </pre> <blockquote type="cite"> <pre>After profiling the server I saw SDL_GetTicks() is taking around 25% of the time (in the tests I used 7 robots and with the single thread mode). To make long short, I saw this loop (while) in simulationserver.cpp: .... Since SimulationServer::Step() takes care of this case (int(mSumDeltaTime*100) < int(mSimStep*100)) I thought that while in Run() is not necessary. So after commenting it out, SDL_GetTicks() takes less than 10% of the time: </pre> </blockquote> <pre>The phenomenon is reasonable. Since the time queries are needed for timing in single thread, the percentage of queries means that your machine are light underload( not too busy ), I guess your robots did not do anything. You will notice that the percentage of queries will lower if robots are collides... But if you remove the codes, the timer is noneffective. Then if the robots collides, the server will very very slow, otherwise the simulation time elapse very quickly. You may ask why use query to get time, ok, I do not think it is a good idea. The server cycle is 20ms, but the time precision of Linux is only 10ms, some function such as "sleep" also can not help on it. In fact, SDL_GetTicks() is used to get "preciser"(seemly). And it is possibly that SDL causes the Input problem in multi-threads. I have some idea to this: 1. to get precise time, use RealTime-Linux as platform [It will narrow OS platform] 2. use a new thread to time, The timing is in InputControl thread in current multi-thread implementation, but query is also used(It can be improved little). [It may make the timing-thread busy, and can not guarantee precision.] 3. use other timer instead SDLTimer, such as boost::xtime. [It need more investigation.] </pre> <blockquote type="cite"> <pre>BTW, the other day I came across the multi-threaded version of ODE by Intel. </pre> </blockquote> <pre>Sounds interesting. </pre> </blockquote> <br> </span></div> </div> </blockquote> </div> <br> </blockquote> </body> </html> |
From: Hedayat V. <hed...@ai...> - 2008-01-29 20:01:19
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> <span>Hi Yuan,<br> <br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>"Yuan Xu" <a class="moz-txt-link-rfc2396E" href="mailto:xuy...@gm..."><xuy...@gm...></a></b></i> wrote on 01/28/2008 07:53:01 AM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">Hi all, </pre> <blockquote type="cite"> <pre wrap="">Next weekend I'll try to work more on these issues, and probably until then will know the answer of your 4th question. 4) What's the reason for stepping in discreet steps in SimulationServer::Step()? Will it change anything with ODE? </pre> </blockquote> <pre wrap=""><!----> Yes, dWorldStep(20ms)+dWorldStep(20ms) != dWorldStep(40ms). Because, the collision detection is missing in dWorldStep(40ms). </pre> </blockquote> Thanks. But there is a problem (I think), it'll be OK if we are occasionally forced to do more than one step in one cycle, but if it happens usually which means that the server is unable to finish a single step in the desired time (which is the current situation I think), the number of required steps will grow exponentially. So what?! I don't know :( <br> Ideally, it shouldn't happen at all or at least in some rare cases, but when it happens, the server will not be able to catch up with the time. <br> Things might be better if we call dWorldStep with the real elapsed time which will at least prevent accumulation of server delays. (Considering the current slowness of the server (at least in my system!), this may change nothing at all, but it might be better when the server is optimized a bit?!)<br> <br> Thanks a lot,<br> Hedayat<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">..... Best wishes! Xu Yuan </pre> </blockquote> </body> </html> |
From: Yuan X. <xuy...@gm...> - 2008-01-31 03:05:40
|
Hi Hedayat, You are right. It is only like 2D server, but the latter does not need so much computation. As I said in my report about the timer, it depends on the server performance and hardware. Currently, some powerful PC can handle the 2v2 game, such as the PC in the China Open 07.( I do not know the specifies, sorry ). But I am sure it can not run 11v11. What we can do is optimize the server, otherwise make the computation time (real time) longer than simulation time. However, longer computation time will make game time longer ( it is not desired by organiger ), and more important, it is not suitable to the machine learning, because the experiment is even slower than the real robot. On the other side, the current server can not run properly in some old computer. To trade-off, maybe a parameter can imported as `simRate' = simulation time / real time. And the user configure it themselves according their hardware. But it may be hard to newbie. > Thanks. But there is a problem (I think), it'll be OK if we are > occasionally forced to do more than one step in one cycle, but if it happens > usually which means that the server is unable to finish a single step in the > desired time (which is the current situation I think), the number of > required steps will grow exponentially. So what?! I don't know :( > Ideally, it shouldn't happen at all or at least in some rare cases, but > when it happens, the server will not be able to catch up with the time. > Things might be better if we call dWorldStep with the real elapsed time > which will at least prevent accumulation of server delays. (Considering the > current slowness of the server (at least in my system!), this may change > nothing at all, but it might be better when the server is optimized a bit?!) > |
From: Hedayat V. <hed...@ai...> - 2008-01-31 14:01:57
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> <span>Hi Yuan,<br> OK, it seems that you are not agree with using a single step (equal the time spent since the last step) in dWorldStep instead of discreet time steps. Not a problem since both of situations are undesirable. <br> <br> About your suggestion, the parameter could have a second state: "auto" which could be the default one. In this mode, the server can recognize the slowness of the running environment and adjusts itself according to that. Also it can print error messages when it runs slower than real time so that the user will recognize it. What's your opinion?<br> <br> Thanks,<br> Bye<br> <br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>"Yuan Xu" <a class="moz-txt-link-rfc2396E" href="mailto:xuy...@gm..."><xuy...@gm...></a></b></i> wrote on 01/31/2008 06:35:37 AM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">Hi Hedayat, You are right. It is only like 2D server, but the latter does not need so much computation. As I said in my report about the timer, it depends on the server performance and hardware. Currently, some powerful PC can handle the 2v2 game, such as the PC in the China Open 07.( I do not know the specifies, sorry ). But I am sure it can not run 11v11. What we can do is optimize the server, otherwise make the computation time (real time) longer than simulation time. However, longer computation time will make game time longer ( it is not desired by organiger ), and more important, it is not suitable to the machine learning, because the experiment is even slower than the real robot. On the other side, the current server can not run properly in some old computer. To trade-off, maybe a parameter can imported as `simRate' = simulation time / real time. And the user configure it themselves according their hardware. But it may be hard to newbie. </pre> <blockquote type="cite"> <pre wrap=""> Thanks. But there is a problem (I think), it'll be OK if we are occasionally forced to do more than one step in one cycle, but if it happens usually which means that the server is unable to finish a single step in the desired time (which is the current situation I think), the number of required steps will grow exponentially. So what?! I don't know :( Ideally, it shouldn't happen at all or at least in some rare cases, but when it happens, the server will not be able to catch up with the time. Things might be better if we call dWorldStep with the real elapsed time which will at least prevent accumulation of server delays. (Considering the current slowness of the server (at least in my system!), this may change nothing at all, but it might be better when the server is optimized a bit?!) </pre> </blockquote> </blockquote> </body> </html> |
From: Yuan X. <xuy...@gm...> - 2008-01-31 03:18:08
|
Hi Hesham, Our team binary in China Open 07 has been published in http://xuyuan.cn.googlepages.com/SEU-RedSun-Final.tar.gz, and our team binary in RC07 is here: http://xuyuan.cn.googlepages.com/SEU-RC07-binary.tar.gz Hope it can help. 2008/1/30, Hesham <hes...@gm...>: > > Hi Yuan, Salam Hedayat, > > Yuan, you're absolutely right. I need to have kind of stress test as well to > have a better insight into the server performance. As you know for a such a > long time I haven't worked on a team for 3D soccer. So I was wondering if > you can send me a binary that (with normal expected behaviours) makes ODE > take a long time to simulate. Like a specific collision or whatever. I'm > using the test agents that stand and just move their arms and send/hear > messages. I've started optimizing ODE, it's not easy but so far it's > promising, on the weekend I'll let you know the progress. > > Thanks in advances, > Hesham > > > On 28/01/2008, Yuan Xu <xuy...@gm...> wrote: > > Hi all, > > > > > > > > Next weekend I'll try to work more on these issues, and probably until > then > > > will know the answer of your 4th question. > > > > > > 4) What's the reason for stepping in discreet steps in > SimulationServer::Step()? > > > Will it change anything with ODE? > > > > Yes, dWorldStep(20ms)+dWorldStep(20ms) != > dWorldStep(40ms). > > Because, the collision detection is missing in dWorldStep(40ms). > > > > > > > Yuan, the code of SDL_GetTicks() is simple, it doesn't give any more > > > precision. I didn't get some parts of what you said. > > > > Yes, the precision depends on the OS, then I remind the RTL, and I > > know it is not suitable. > > > > > Maybe a silly question > > > but, the server is supposed to have sharp 20ms cycles? > > > > The cycle duration effects the performance of robot controller. > > Note that the control cycle is less than 10ms in real robot. > > The longer cycle the harder for the controller. > > And the HMDP[Joschka] may solve this, but it will make teams to change > > their codes a lot. > > > > > I thought the > > > previous timing method (SPADES) was replaced by the new one to get rid > of > > > the problems we had with timers, and more rely on time stamps. But you > > > suggested using RTL to have more precise timers. > > > > The SPADES do not count the real time, it counts how many jiffies used > > by the agent. > > It can measure the CPU cost of the agent, but if the agent spend a lot > > of time on other work, such as communication, writing logs, the game > > will be very slow. Jelle Kok said that some games last less than > > 10min, but some other games need more than 1 hour! > > > > > Sorry maybe I got the whole > > > thing wrong, I'll go through the code again. But I would appreciate it > if > > > you (or others) could please remind me the new idea for timing, maybe by > > > referring to old emails - I couldn't find them. > > > > > > > The timer was designed just like the 2D server: fixed cycle duration, > > the agent should sent action in time, otherwise the action will not be > > executed. The difference between 2D and 3D only are the duration and > > implementation. > > > > > > Best wishes! > > > > Xu Yuan > > > > -- Best wishes! Xu Yuan School of Automation Southeast University, Nanjing, China mail: xuy...@gm... xy...@ya... web: http://xuyuan.cn.googlepages.com -------------------------------------------------- |
From: Hesham <hes...@gm...> - 2008-02-19 20:38:48
|
Hi Yuan, Sorry for getting back to this late. On 27/01/2008, Yuan Xu <xuy...@gm...> wrote: 1. to get precise time, use RealTime-Linux as platform [It will narrow OS platform] Apparently the new kernels already have high resolution timers (they have problem just with old cpus). I can get nanosecond with this program on my laptop: #include <ctime> #include <iostream> using namespace std; int main() { float temp; struct timespec ts1; struct timespec ts2; clock_gettime( CLOCK_REALTIME, &ts1 ); for (size_t i=0; i<50000000; i++) temp = 10.0 + 0.1; clock_gettime( CLOCK_REALTIME, &ts2 ); long long int count1 = static_cast<long long>(1000000000UL)*static_cast<long long>(ts1.tv_sec) + static_cast<long long>(ts1.tv_nsec); long long int count2 = static_cast<long long>(1000000000UL)*static_cast<long long>(ts2.tv_sec) + static_cast<long long>(ts2.tv_nsec); cout<<(count2-count1)<<endl; return 0; } And also there is no need to worry about new multi-core CPUs, they have synchronized cores (at least that's the case with Intel, according to the comments in the kernel code). I did some tests with nanosleep() too. The error is around 0.5 msec (much less than 10 msec that you mentioned), so it shows we may can use it if that error is acceptable (to me it is). Bests, Hesham On 27/01/2008, Yuan Xu <xuy...@gm...> wrote: > > Hi Hesham and all, > > > > After profiling the server I saw SDL_GetTicks() is taking around 25% of > the > > time (in the tests I used 7 robots and with the single thread mode). To > make > > long short, I saw this loop (while) in simulationserver.cpp: > > .... > > Since SimulationServer::Step() takes care of this case > > (int(mSumDeltaTime*100) < int(mSimStep*100)) I thought that while in > Run() > > is not necessary. So after commenting it out, SDL_GetTicks() takes less > than > > 10% of the time: > > The phenomenon is reasonable. > Since the time queries are needed for timing in single thread, > the percentage of queries means that your machine are light underload( > not too busy ), > I guess your robots did not do anything. You will notice that the > percentage of queries will lower if robots are collides... > But if you remove the codes, the timer is noneffective. Then if the > robots collides, the server will very very slow, otherwise the > simulation time elapse very quickly. > You may ask why use query to get time, ok, I do not think it is a good > idea. > The server cycle is 20ms, but the time precision of Linux is only 10ms, > some function such as "sleep" also can not help on it. > In fact, SDL_GetTicks() is used to get "preciser"(seemly). > And it is possibly that SDL causes the Input problem in multi-threads. > > I have some idea to this: > 1. to get precise time, use RealTime-Linux as platform [It will > narrow OS platform] > 2. use a new thread to time, The timing is in InputControl thread in > current multi-thread implementation, but query is also used(It can be > improved little). [It may make the timing-thread busy, and can not > guarantee precision.] > 3. use other timer instead SDLTimer, such as boost::xtime. [It need > more investigation.] > > > BTW, the other day I came across the multi-threaded version of ODE by > Intel. > > Sounds interesting. > > > I think it's worth a try if Yuan has time to help :-) There is something > > wrong with the multi-threaded mode of the server on my system. So first > I > > need to find out what's the problem. > > I am free recently. Give some hints of your problem. > > > -- > Best wishes! > > Xu Yuan > School of Automation > Southeast University, Nanjing, China > > mail: xuy...@gm... > xy...@ya... > web: http://xuyuan.cn.googlepages.com > -------------------------------------------------- > |
From: Yuan X. <xuy...@gm...> - 2008-02-20 07:44:10
Attachments:
test_timer.cpp
|
Hi Hesham, Thanks for your comments and code. In fact, SDL_GetTicks() call clock_gettime(CLOCK_MONOTONIC, &now), if it's available[1]. But the clocks are different, the SDL use CLOCK_MONOTONIC. I am not sure which is better, since clock_getres() with the two clocks returns the same precision in my systeam(Suse10.2): they are both ( 0 4000250). And you mentioned the nanosleep() error is around 0.5 msec. But I think it is the same mean with the 10 msec precision. I have done a test on different sleeps: usleep, boost::thread::sleep and nanosleep. The results showed that there are nearly the same: Input(ms) boost usleep nanosleep gettimeofday clock_gettime 0 4 4 4.03694 0.000556 0.000409 1 4.01191 4.0041 4.00091 1.00193 1.00087 2 4.044 4.01201 4.02362 2.00263 2.00181 3 8.03601 8.08801 8.07893 3.00388 3.01715 4 8.00801 7.9996 8.01276 4.00417 4.00744 5 8.46001 12.0281 10.2573 5.00107 5.0015 6 8.02001 8.02391 8.2374 6.04204 6.01252 7 12.024 12.008 12.0635 7.00603 7.01054 8 12.052 12.532 12.0045 8.0039 8.00347 9 13.8839 12.056 12.0787 9.04589 9.01147 The program which I used to test have been attached, you can compile it by: "g++ test_timer.cpp -lrt -lboost_thread" I noticed that SDL_Delay() is implemented by a while loop(also see [1]), which is similar as our current code. The last two rows of above table are results of 'while loop' method with gettimeofday() and clock_gettime(). I think I can make a conclusion: if we use 'sleep', no matter which one, the precision can not guaranteed; if we use 'while loop' the precision is better(It is also tested in my program), but it will cost lots of computation. [1] http://www.libsdl.org/cgi/viewvc.cgi/trunk/SDL/src/timer/unix/SDL_systimer.c?view=markup > > Apparently the new kernels already have high resolution timers (they have > problem just with old cpus). I can get nanosecond with this program on my > laptop: > > #include <ctime> > #include <iostream> > using namespace std; > int main() > { > float temp; > struct timespec ts1; > struct timespec ts2; > clock_gettime( CLOCK_REALTIME, &ts1 ); > for (size_t i=0; i<50000000; i++) > temp = 10.0 + 0.1; > clock_gettime( CLOCK_REALTIME, &ts2 ); > long long int count1 = static_cast<long > long>(1000000000UL)*static_cast<long long>(ts1.tv_sec) + > static_cast<long long>(ts1.tv_nsec); > long long int count2 = static_cast<long > long>(1000000000UL)*static_cast<long long>(ts2.tv_sec) + > static_cast<long long>(ts2.tv_nsec); > cout<<(count2-count1)<<endl; > return 0; > } > > And also there is no need to worry about new multi-core CPUs, they have > synchronized cores (at least that's the case with Intel, according to the > comments in the kernel code). > > I did some tests with nanosleep() too. The error is around 0.5 msec (much > less than 10 msec that you mentioned), so it shows we may can use it if that > error is acceptable (to me it is). > > Bests, > Hesham > > > On 27/01/2008, Yuan Xu <xuy...@gm...> wrote: > > Hi Hesham and all, > > > > > > > After profiling the server I saw SDL_GetTicks() is taking around 25% of > the > > > time (in the tests I used 7 robots and with the single thread mode). To > make > > > long short, I saw this loop (while) in simulationserver.cpp: > > > .... > > > Since SimulationServer::Step() takes care of this case > > > (int(mSumDeltaTime*100) < int(mSimStep*100)) I thought that while in > Run() > > > is not necessary. So after commenting it out, SDL_GetTicks() takes less > than > > > 10% of the time: > > > > The phenomenon is reasonable. > > Since the time queries are needed for timing in single thread, > > the percentage of queries means that your machine are light underload( > > not too busy ), > > I guess your robots did not do anything. You will notice that the > > percentage of queries will lower if robots are collides... > > But if you remove the codes, the timer is noneffective. Then if the > > robots collides, the server will very very slow, otherwise the > > simulation time elapse very quickly. > > You may ask why use query to get time, ok, I do not think it is a good > idea. > > The server cycle is 20ms, but the time precision of Linux is only 10ms, > > some function such as "sleep" also can not help on it. > > In fact, SDL_GetTicks() is used to get "preciser"(seemly). > > And it is possibly that SDL causes the Input problem in multi-threads. > > > > I have some idea to this: > > 1. to get precise time, use RealTime-Linux as platform [It will > > narrow OS platform] > > 2. use a new thread to time, The timing is in InputControl thread in > > current multi-thread implementation, but query is also used(It can be > > improved little). [It may make the timing-thread busy, and can not > > guarantee precision.] > > 3. use other timer instead SDLTimer, such as boost::xtime. [It need > > more investigation.] > > > > > BTW, the other day I came across the multi-threaded version of ODE by > Intel. > > > > Sounds interesting. > > > > > I think it's worth a try if Yuan has time to help :-) There is something > > > wrong with the multi-threaded mode of the server on my system. So first > I > > > need to find out what's the problem. > > > > I am free recently. Give some hints of your problem. > > > > > > -- > > Best wishes! > > > > Xu Yuan > > School of Automation > > Southeast University, Nanjing, China > > > > mail: xuy...@gm... > > xy...@ya... > > web: http://xuyuan.cn.googlepages.com > > -------------------------------------------------- > > > > -- Best wishes! Xu Yuan School of Automation Southeast University, Nanjing, China mail: xuy...@gm... xy...@ya... web: http://xuyuan.cn.googlepages.com -------------------------------------------------- |