From: Yuan X. <xuy...@gm...> - 2008-03-07 08:29:45
|
Hi Hedayat and all, I take a look at current multi-thread implementation. But I can not understand it exactly, could you help me? 1). in simulationserver.cpp 00491 Step(); 00492 ControlEvent(CE_EndCycle); The Step() calls mSceneServer->Pre/PostPhysicsUpdate() again, I think it is redundant, what should be called maybe mSceneServer->PhysicsUpdate(mSimStep), right? Place ControlEvent(CE_EndCycle) here is for rendering function in the main thread? The original code can not work with Mesa-6.4.1, it can work with Mesa-6.5.21 and ATI fglrx 8.31.5 . I checked the ChanegeLog of Mesa, there are some bug-fixes of thread safe in Mesa-6.5. Can you test that? 2). Synchronization by the barrier just make all the threads run one by one, it is not the real parallel. The original idea is make SimControlNodes run parallel, and PhysicsUpdate can also run at the same time. Since the scene tree can be used as cache. PrePhysicsUpdate copy data from scene tree to ODE, PosPhysicsUpdate copy data from ODE to scene tree. PhysicsUpdate only effects ODE internal data. ( To Joschka, Sorry, I do not finished the document work about single-thread and multi-thread. I want to make things clear firstly.) -- Best wishes! Xu Yuan School of Automation Southeast University, Nanjing, China mail: xuy...@gm... xy...@ya... web: http://xuyuan.cn.googlepages.com -------------------------------------------------- |
From: Hedayat V. <hed...@ai...> - 2008-03-07 10:41:02
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> <span><br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>"Yuan Xu" <a class="moz-txt-link-rfc2396E" href="mailto:xuy...@gm..."><xuy...@gm...></a></b></i> wrote on 03/07/2008 11:59:43 AM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">Hi Hedayat and all, I take a look at current multi-thread implementation. </pre> </blockquote> Hi Yuan,<br> Thanks a lot for looking at the code and for your comments.<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">But I can not understand it exactly, could you help me? 1). in simulationserver.cpp 00491 Step(); 00492 ControlEvent(CE_EndCycle); The Step() calls mSceneServer->Pre/PostPhysicsUpdate() again, I think it is redundant, what should be called maybe mSceneServer->PhysicsUpdate(mSimStep), right? </pre> </blockquote> You should know better than me. When I was writing the code, my main focus was on correctness. I was not sure about those functions and I didn't look further to see if they are redundant. I decided to wait for people who know it already! :). Yes, regarding to your description about Pre/PostPhysicsUpdate() functions, it seems that they are redundant. So, if updating GameControlServer is unnecessary too, Step() can be removed and using mSceneServer->PhysicsUpdate(mSimStep) before call to PostPhysicsUpdate() (with all required time calculations) should be enough. <br> So, please let me know if updating GameControlServer is necessary and I'll remove redundant calls. I have not investigated it thoroughly, but it seems that there is no need to update GameControlServer here. Am I right?<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap=""> Place ControlEvent(CE_EndCycle) here is for rendering function in the main thread? The original code can not work with Mesa-6.4.1, it can work with Mesa-6.5.21 and ATI fglrx 8.31.5 . I checked the ChanegeLog of Mesa, there are some bug-fixes of thread safe in Mesa-6.5. Can you test that? </pre> </blockquote> I'm using Mesa 7.0.1 with nvidia binary driver 169.09 and the problem still exists.<br> Yes, the main reason was rendering, but there is another problem too. FRP sensor calls dJointGetFeedback(). ODE is not thread safe and this call will generate segmentation faults some times. Also, this call should happen before next physics update since the feedback information will not be available then (I think!). <br> In fact, at first I decided to treat RenderControl as an exception (to call its EndCycle() function in the main thread and other EndCycle functions in their own thread), but because of the above problem I decided to call all of them in the main thread. I put it in my TODO list for future to consider a better implementation.<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap=""> 2). Synchronization by the barrier just make all the threads run one by one, it is not the real parallel. </pre> </blockquote> I don't think so. As far as I know, it is just another synchronization method. It doesn't affect thread execution model and they run in parallel. The only thing which it does is in it's wait function. According to boost docs:<br> <br> "When a barrier is created, it is initialized with a thread count N. The first N-1 calls to <code class="computeroutput">wait()</code> will all cause their threads to be blocked. The Nth call to <code class="computeroutput">wait()</code> will allow all of the waiting threads, including the Nth thread, to be placed in a ready state. The Nth call will also "reset" the barrier[...]. This functionality allows the same set of N threads to re-use a barrier object to synchronize their execution at multiple points during their execution."<br> <br> As all of the threads will be placed in a ready state, they should run in parallel. Please correct me if I'm wrong.<br> <br> Thanks again,<br> Hedayat<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">The original idea is make SimControlNodes run parallel, and PhysicsUpdate can also run at the same time. Since the scene tree can be used as cache. PrePhysicsUpdate copy data from scene tree to ODE, PosPhysicsUpdate copy data from ODE to scene tree. PhysicsUpdate only effects ODE internal data. ( To Joschka, Sorry, I do not finished the document work about single-thread and multi-thread. I want to make things clear firstly.) </pre> </blockquote> </body> </html> |
From: Yuan X. <xuy...@gm...> - 2008-03-08 04:57:12
|
Hi Hedayat, Thanks for your reply. > You should know better than me. When I was writing the code, my main focus > was on correctness. I was not sure about those functions and I didn't look > further to see if they are redundant. I decided to wait for people who know > it already! :). Yes, regarding to your description about > Pre/PostPhysicsUpdate() functions, it seems that they are redundant. So, if > updating GameControlServer is unnecessary too, Step() can be removed and > using mSceneServer->PhysicsUpdate(mSimStep) before call to > PostPhysicsUpdate() (with all required time calculations) should be enough. > So, please let me know if updating GameControlServer is necessary and I'll > remove redundant calls. I have not investigated it thoroughly, but it seems > that there is no need to update GameControlServer here. Am I right? GameControlServer should be updated after the PostPhysicsUpdate. So the loop of the main thread should be: mSceneServer->PrePhysicsUpdate mSceneServer->PhysicsUpdate mSceneServer->PostPhysicsUpdate GameControlServer->Update > I'm using Mesa 7.0.1 with nvidia binary driver 169.09 and the problem still > exists. > Yes, the main reason was rendering, but there is another problem too. FRP > sensor calls dJointGetFeedback(). ODE is not thread safe and this call will > generate segmentation faults some times. Also, this call should happen > before next physics update since the feedback information will not be > available then (I think!). Bad news, the OpenGL should be thread-safe. You can call dJointGetFeedback() in PostPhysicsUpdateInternal(), and cache the feedback information. It just likes cache the actions and realize them in PrePhysicsUpdateInternal(). > > I don't think so. As far as I know, it is just another synchronization > method. It doesn't affect thread execution model and they run in parallel. > The only thing which it does is in it's wait function. According to boost > docs: > [...] > > As all of the threads will be placed in a ready state, they should run in > parallel. Please correct me if I'm wrong. > Ah, I see! It is OK, and much simpler and clear than the old code, great job! But there may be a small problem in current code: The scene tree should not be accessed by the main thread and SimCtrThreads at the same time. (May be the problem will come after you remove the Step().) So when call the follow functions, the SimCtrThreads should be blocked. mSceneServer->PrePhysicsUpdate mSceneServer->PostPhysicsUpdate GameControlServer->Update So the lines in RunMultiThreaded() needs adjustment, I think it is easy for you. -- Best wishes! Xu Yuan School of Automation Southeast University, Nanjing, China mail: xuy...@gm... xy...@ya... web: http://xuyuan.cn.googlepages.com -------------------------------------------------- |
From: Hedayat V. <hed...@ai...> - 2008-03-08 17:42:04
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> <span>Hi Yuan,<br> <br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>"Yuan Xu" <a class="moz-txt-link-rfc2396E" href="mailto:xuy...@gm..."><xuy...@gm...></a></b></i> wrote on 03/08/2008 08:27:05 AM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">Hi Hedayat, Thanks for your reply. [...] GameControlServer should be updated after the PostPhysicsUpdate. So the loop of the main thread should be: mSceneServer->PrePhysicsUpdate mSceneServer->PhysicsUpdate mSceneServer->PostPhysicsUpdate GameControlServer->Update </pre> </blockquote> Done. Thanks! <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <blockquote type="cite"> <pre wrap=""> I'm using Mesa 7.0.1 with nvidia binary driver 169.09 and the problem still exists. Yes, the main reason was rendering, but there is another problem too. FRP sensor calls dJointGetFeedback(). ODE is not thread safe and this call will generate segmentation faults some times. Also, this call should happen before next physics update since the feedback information will not be available then (I think!). </pre> </blockquote> <pre wrap=""><!----> Bad news, the OpenGL should be thread-safe. </pre> </blockquote> Yes :(. But Mesa might be thread safe now. Since I've installed nvidia binary drivers, simspark uses nvidia OpenGL libraries instead of using Mesa. So, I can't say anything about mesa, but nvidia opengl is not thread safe.<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap=""> You can call dJointGetFeedback() in PostPhysicsUpdateInternal(), and cache the feedback information. It just likes cache the actions and realize them in PrePhysicsUpdateInternal(). </pre> </blockquote> <br> Good suggestion. I never liked creating feedback structs using new, and I've changed the code so the dJointFeedbacks are now stored inside ForceResistancePerceptor and no ODE calls are necessary anymore. It solved the problem.<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap=""><!----> Ah, I see! It is OK, and much simpler and clear than the old code, great job! </pre> </blockquote> Thanks! :)<br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">But there may be a small problem in current code: The scene tree should not be accessed by the main thread and SimCtrThreads at the same time. (May be the problem will come after you remove the Step().) So when call the follow functions, the SimCtrThreads should be blocked. mSceneServer->PrePhysicsUpdate mSceneServer->PostPhysicsUpdate GameControlServer->Update So the lines in RunMultiThreaded() needs adjustment, I think it is easy for you. </pre> </blockquote> I thought that I can call mSceneServer->PrePhysicsUpdate() in parallel :(. This is really bad! This function should be called after EndCycle() functions which run in parallel now. So, protecting mSceneServer->PrePhysicsUpdate() needs two more wait() calls. I wonder if this can be worse than calling EndCycle() and mSceneServer->PrePhysicsUpdate() functions in the main thread!<br> <br> The interesting (not much!) point is that I have not encountered any problems by calling mSceneServer->PrePhysicsUpdate() in parallel till now! Is there any conflicts between mSceneServer->PrePhysicsUpdate() and EndCycle() functions in the current code?! <br> <br> I've committed my changes. Any suggestions/comments are highly appreciated!<br> <br> Thanks,<br> Hedayat<br> </body> </html> |
From: Yuan X. <xuy...@gm...> - 2008-03-09 08:49:27
|
Hi Hedayat and all, > > Yes :(. But Mesa might be thread safe now. Since I've installed nvidia > binary drivers, simspark uses nvidia OpenGL libraries instead of using Mesa. > So, I can't say anything about mesa, but nvidia opengl is not thread safe. > The rcssserver3d-0.5.6 can run multi-threads in my computer, it does not need to call opengl in the main thread. but the cvs head code can't. It is very strange, so I decide to look into this problem. Because there are too many differences between cvs head and 0.5.6, I copy the barrier implementation to the rcssserver3d-0.5.6, then it works! while calling opengl in a new thread. I attached the testing server to this mail, could you test it in your system? So, the problem may caused by other changes relative to opengl, I tested and compared the of functions in class rendercontrol and rendernode, they should be OK. Thus, I think the problem may caused by the initialization of opengl, but the ruby scripts are changed a lot, I do not have time to trace them now. > > The interesting (not much!) point is that I have not encountered any > problems by calling mSceneServer->PrePhysicsUpdate() in > parallel till now! Is there any conflicts between > mSceneServer->PrePhysicsUpdate() and EndCycle() functions > in the current code?! > I am not sure, but they may conflict after future changing. I changes a little with your code.( See the package attached.) I hope it would be OK. There is another issue should be in mind is that the multi-threads increase the sense act latency, it makes agent behavior unlike in the singled-thread. I think this will make teams in trouble while changing from singled-thread to multi-threads. What do you think about it? -- Best wishes! Xu Yuan School of Automation Southeast University, Nanjing, China mail: xuy...@gm... xy...@ya... web: http://xuyuan.cn.googlepages.com -------------------------------------------------- |
From: Hedayat V. <hed...@ai...> - 2008-03-09 19:17:53
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> <span><br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>"Yuan Xu" <a class="moz-txt-link-rfc2396E" href="mailto:xuy...@gm..."><xuy...@gm...></a></b></i> wrote on 03/09/2008 12:19:22 PM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">Hi Hedayat and all, </pre> </blockquote> Hi!<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap=""> The rcssserver3d-0.5.6 can run multi-threads in my computer, it does not need to call opengl in the main thread. but the cvs head code can't. It is very strange, so I decide to look into this problem. Because there are too many differences between cvs head and 0.5.6, I copy the barrier implementation to the rcssserver3d-0.5.6, then it works! while calling opengl in a new thread. I attached the testing server to this mail, could you test it in your system? </pre> </blockquote> OK, I'll test it ASAP. Interesting!<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap=""> There is another issue should be in mind is that the multi-threads increase the sense act latency, it makes agent behavior unlike in the singled-thread. I think this will make teams in trouble while changing from singled-thread to multi-threads. What do you think about it? </pre> </blockquote> Sorry, would you please explain more about the reason of this increased latency? In both single threaded and multi-threaded modes StartCycle() is called immediately after EndCycle(). What increases the latency?<br> <br> And another question: it seems that it is enough to call SceneServer::Pre/PostPhysicsUpdate() functions once in a cycle. So, we can apply the same change in Step() for single threaded mode (to call PhysicsUpdate() function inside it's while loop instead of Update() function and calling Pre/PostPhysicsUpdate() functions (and GameControlServer::Update()) outside the while loop); right? It should benefit performance a little.<br> <br> Thanks a lot,<br> Hedayat<br> <br> </body> </html> |
From: Hedayat V. <hed...@ai...> - 2008-03-10 18:19:19
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html style="direction: ltr;"> <head> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> </head> <body style="direction: ltr;" bgcolor="#ffffff" text="#000000"> <p><span>Hi Yuan,</span></p> <p style="margin-bottom: 0cm; margin-top: 0pt;"><span></span></p> <span><br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>Hedayat Vatankhah <a class="moz-txt-link-rfc2396E" href="mailto:hed...@ai..."><hed...@ai...></a></b></i> wrote on 03/09/2008 10:45:48 PM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:47D...@ai..." type="cite"> <meta content="text/html;charset=UTF-8" http-equiv="Content-Type"> <span><br> <style type="text/css">blockquote {color: navy !important; background-color: RGB(245,245,245) !important; padding: 0 15 10 15 !important; margin: 15 0 0 0; border-left: #1010ff 2px solid;} blockquote blockquote {color: maroon !important; background-color: RGB(235,235,235) !important; border-left-color:maroon !important} blockquote blockquote blockquote {color: green !important; background-color: RGB(225,225,225) !important; border-left-color:teal !important} blockquote blockquote blockquote blockquote {color: purple !important; background-color: RGB(215,215,215) !important; border-left-color: purple !important} blockquote blockquote blockquote blockquote blockquote {color: teal !important; background-color: RGB(205,205,205) !important; border-left-color: green !important}</style><i><b>"Yuan Xu" <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:xuy...@gm..."><xuy...@gm...></a></b></i> wrote on 03/09/2008 12:19:22 PM:</span><br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap="">[...] I attached the testing server to this mail, could you test it in your system? </pre> </blockquote> OK, I'll test it ASAP. Interesting!<br> </blockquote> Unfortunately, it doesn't work on my system :(<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:47D...@ai..." type="cite"> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:18a...@ma..." type="cite"> <pre wrap=""> There is another issue should be in mind is that the multi-threads increase the sense act latency, it makes agent behavior unlike in the singled-thread. I think this will make teams in trouble while changing from singled-thread to multi-threads. What do you think about it? </pre> </blockquote> Sorry, would you please explain more about the reason of this increased latency? In both single threaded and multi-threaded modes StartCycle() is called immediately after EndCycle(). What increases the latency?<br> </blockquote> OK! I read the code and understood what you said. Yes, the variable latency is not desirable specially that it is not predictable :(<br> <br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:47D...@ai..." type="cite">And another question: it seems that it is enough to call SceneServer::Pre/PostPhysicsUpdate() functions once in a cycle. So, we can apply the same change in Step() for single threaded mode (to call PhysicsUpdate() function inside it's while loop instead of Update() function and calling Pre/PostPhysicsUpdate() functions (and GameControlServer::Update()) outside the while loop); right? It should benefit performance a little.<br> </blockquote> Another thing: PrePhysicsUpdate received a deltaTime parameter. But currently, it is called once per cycle in RunMultiThreaded. We should either remove the parameter (I'm not sure but it might be unused!) or put it inside the PhysicsUpdate while loop. Which one is better?<br> <br> Thanks,<br> Hedayat<br> <blockquote style="border-left: 2px solid rgb(16, 16, 255); color: navy; background-color: rgb(245, 245, 245); padding-left: 15px;" cite="mid:47D...@ai..." type="cite"> <br> Thanks a lot,<br> Hedayat<br> <br> </blockquote> </body> </html> |