Menu
▾
▴
Re: [Repast-interest] Parallelization of Repast Simulations
|
From: Upton, S. (S. (CIV) <sc...@np...> - 2012-05-29 13:20:47
|
Sam, If you want to parallelize your experiment, i.e., running your simulations 1000 times (or changing input parameters), I'd look into condor (http://research.cs.wisc.edu/condor/), especially if you're running your experiment on a single machine (personal Condor). If have several machines in a network, e.g., a lab, then condor is a good choice as well. If you have access to much bigger machines, e.g., Cray's with 1000's of processors, then they have their own scheduling software that you have to interface with, but it's definitely worth it, especially if your simulation takes some time to run. I have not done any work on parallelizing a serial simulation internally. steve From: Ronald Mintz [mailto:rj...@co...] Sent: Friday, May 25, 2012 5:17 PM To: rep...@li... Subject: Re: [Repast-interest] Parallelization of Repast Simulations Sam, I have been doing agent based simulations on different parameter sets using Repast 3 on a Mac with 8 processors. Method (2) below has worked very well with up to 8 simultaneous simulations, each on a separate parameter set, running in parallel. I found that the time to run 8 simultaneous simulations this way takes only 50% longer than the time to run a single simulation separately. There are two procedures that I have used. I have initially run up to 8 Eclipse processes simultaneously, with each Eclipse process running the same Java program to do the simulations, but on a different sequence of parameter sets. I have also written a Unix shell script to run each of the parallel simulations as a background process with semaphore-like controls to limit the number of simulations going on at one time to 8, so as not to overstrain my computer resources. The latter plan did not require explicitly dividing the set of simulations into 8 portions and programming each Eclipse process to execute one portion of the long sequence of simulations. It is just as easy to run up to 8 simulations in parallel as to run them as a single sequence as one process. The shell script maintains the running of 8 background processes, and starts a new process to run another simulation from the next parameter set whenever a previous simulation finishes. Each simulation is run from the Unix command line from the shell script without Eclipse. If one simulation crashes on its parameter set, the process continues with the next parameter set in parallel with the other simultaneously running simulations. Ron On May 25, 2012, at 6:25 AM, rep...@li... wrote: Send Repast-interest mailing list submissions to rep...@li... To subscribe or unsubscribe via the World Wide Web, visit https://lists.sourceforge.net/lists/listinfo/repast-interest or, via email, send a message with subject or body 'help' to rep...@li... You can reach the person managing the list at rep...@li... When replying, please edit your Subject line so it is more specific than "Re: Contents of Repast-interest digest..." Today's Topics: 1. Re: Parallelization with RepastHPC. (Dr Andrew J Cleary) 2. Freeze dried simulation with XML file (Mansoriya HAMIDOU) 3. Run many simulations consecutively (Mansoriya HAMIDOU) 4. Re: Parallelization with RepastHPC. (Murphy, John T.) From: Dr Andrew J Cleary <and...@gm...> Subject: Re: [Repast-interest] Parallelization with RepastHPC. Date: May 25, 2012 12:37:44 AM EDT To: rep...@li... If a single run of your simulation will fit into one process (enough memory, doesn't run for ridiculous amounts of time), method 2 is always better. On Thu, May 24, 2012 at 6:19 PM, Sam Seaver <sam...@gm...> wrote: Dear Repast, I'm now starting to consider how I would like run my simulation using parallel processes. It's my first foray into the world of parallelization so bear with me. My previously serialized simulation could be generalized as n agents performing n stochastic interactions in 3-dimensional space over 1,000 time-steps, after which statistics on the agents' state is calculated. The simulations would be performed 1,000 times, or more. As I understand it, there's two ways in which I can parallelize my simulations: 1) For a given number of processes, I can perform one simulation, where n agents are shared amongst all the processes, and their interactions are also shared amongst the processes. Since all of the agents are in the same 3-dimensional space, this seems it will involve a lot of non-local agents sharing their states amongst different processes. 2) Alternatively, I can run a given number of simulations at the same time, but each simulation occurs within one process, so every agent is local to its process, and this minimizes cross-process communication. I'm leaning towards the second method, but I'm not sure if one method is "better" than the other. Any thoughts? Thanks Sam -- Postdoctoral Fellow Mathematics and Computer Science Division Argonne National Laboratory 9700 S. Cass Avenue Argonne, IL 60439 http://www.linkedin.com/pub/sam-seaver/0/412/168 sam...@gm... (773) 796-7144 <tel:%28773%29%20796-7144> "We shall not cease from exploration And the end of all our exploring Will be to arrive where we started And know the place for the first time." --T. S. Eliot ---------------------------------------------------------------------------- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Repast-interest mailing list Rep...@li... https://lists.sourceforge.net/lists/listinfo/repast-interest -- "Show me a movement that doesn't hate somebody and I will join it at once." ~ Robert Anton Wilson "Authority should derive from the consent of the governed, not from the threat of force!" Barbie, in Toy Story 3. From: Mansoriya HAMIDOU <ham...@gm...> Subject: [Repast-interest] Freeze dried simulation with XML file Date: May 25, 2012 4:24:19 AM EDT To: rep...@li... Hi, I have a problem with the xml file loader: First I load the context from a context builder class. Context is loaded, with a 3D grid projection. Agents are added randomly (I have 3 agents types). Then I use "Freeze dry current data" to save the current context in an XML file. If I reset run and setect the xml file as data loader, I don't obtain the context saved: the number of agents is the same but the coordinates of agents are not the same. How can I obtain the same configuration: the same agents with the same coordinates ? I wanted to test loading data from the code , but I don't know how to do it, I tryed this but it doesn't work public class TerminalContext implements ContextBuilder<Agent> { @Override public Context build(Context<Agent> context) { // TODO Auto-generated method stub File file = new File("test.xml"); XMLContextBuilder xml = new XMLContextBuilder(file); xml.build(context); return context; } } the error was: ERROR [AWT-EventQueue-0] 2012-05-25 10:18:35,984 repast.simphony.visualization.engine.DisplayComponentControllerAction - Error while creating displays repast.simphony.visualization.engine.DisplayCreationException: Error while creating display. at repast.simphony.visualization.engine.DisplayCreator3D.createDisplay(DisplayC reator3D.java:101) at repast.simphony.visualization.engine.DisplayProducer.createDisplay(DisplayPr oducer.java:46) at repast.simphony.visualization.engine.DisplayComponentControllerAction.runIni tialize(DisplayComponentControllerAction.java:90) at repast.simphony.engine.controller.DefaultController$2.visit(DefaultControlle r.java:207) at repast.simphony.engine.controller.DefaultController$2.visit(DefaultControlle r.java:1) at repast.simphony.util.collections.NaryTree.preOrderTraverals(NaryTree.java:28 7) at repast.simphony.util.collections.NaryTree.preOrderTraverals(NaryTree.java:29 0) at repast.simphony.util.collections.NaryTree.preOrderTraverals(NaryTree.java:29 0) at repast.simphony.util.collections.NaryTree.preOrderTraversal(NaryTree.java:28 3) at repast.simphony.engine.controller.DefaultController.runInitialize(DefaultCon troller.java:205) at repast.simphony.engine.controller.DefaultController.runInitialize(DefaultCon troller.java:376) at repast.simphony.ui.RSApplication.initSim(RSApplication.java:159) at repast.simphony.ui.action.InitRun.actionPerformed(InitRun.java:17) at javax.swing.AbstractButton.fireActionPerformed(Unknown Source) at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source) at javax.swing.DefaultButtonModel.setPressed(Unknown Source) at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source) at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source) at java.awt.Component.processMouseEvent(Unknown Source) at javax.swing.JComponent.processMouseEvent(Unknown Source) at java.awt.Component.processEvent(Unknown Source) at java.awt.Container.processEvent(Unknown Source) at java.awt.Component.dispatchEventImpl(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Window.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.awt.EventQueue.access$000(Unknown Source) at java.awt.EventQueue$1.run(Unknown Source) at java.awt.EventQueue$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue$2.run(Unknown Source) at java.awt.EventQueue$2.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue.dispatchEvent(Unknown Source) at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.run(Unknown Source) Caused by: java.lang.RuntimeException: Projection 'Grid' not found. at repast.simphony.visualization.engine.AbstractDisplayCreator.createLayout(Abs tractDisplayCreator.java:51) at repast.simphony.visualization.engine.DisplayCreator3D.createDisplay(DisplayC reator3D.java:63) ... 49 more Thank you in advance. Mansoriya HAMIDOU Phd student University of Le Havre - France From: Mansoriya HAMIDOU <ham...@gm...> Subject: [Repast-interest] Run many simulations consecutively Date: May 25, 2012 4:31:15 AM EDT To: rep...@li... Hi, I want to run many simulations consecutivelyand obtais the average of "Tick Count". Is it possible ? My simlations always stop. I use Repast Simphony 2.0 regards Mansoriya HAMIDOU Phd student University of Le Havre - France From: "Murphy, John T." <jtm...@an...> Subject: Re: [Repast-interest] Parallelization with RepastHPC. Date: May 25, 2012 6:25:01 AM EDT To: "rep...@li..." <rep...@li...> I will add to this an academic comment that some patterns of agent interaction are not really amenable to parallelization via method 1. Some more information about the way that agents are selecting other agents with which to interact can clarify this for your specific simulation, but it would really be an exercise: if, as the first responder suggests, you can run your simulation on one process, method 2 is better. Best, John -- John T. Murphy Computational Postdoctoral Fellow Decision and Information Sciences and Argonne Leadership Computing Facility Argonne National Laboratory jtm...@an... From: Dr Andrew J Cleary <and...@gm... <mailto:and...@gm...%3cmailto:and...@gm...> <mailto:and...@gm...>> To: "rep...@li... <mailto:rep...@li...%3cmailto:repast-interest@lists .sourceforge.net%3e> <mailto:rep...@li...>" <rep...@li... <mailto:rep...@li...%3cmailto:repast-interest@lists .sourceforge.net> <mailto:rep...@li...>> Subject: Re: [Repast-interest] Parallelization with RepastHPC. If a single run of your simulation will fit into one process (enough memory, doesn't run for ridiculous amounts of time), method 2 is always better. On Thu, May 24, 2012 at 6:19 PM, Sam Seaver <sam...@gm... <mailto:sam...@gm...%3cmailto:sam...@gm...> <mailto:sam...@gm...>> wrote: Dear Repast, I'm now starting to consider how I would like run my simulation using parallel processes. It's my first foray into the world of parallelization so bear with me. My previously serialized simulation could be generalized as n agents performing n stochastic interactions in 3-dimensional space over 1,000 time-steps, after which statistics on the agents' state is calculated. The simulations would be performed 1,000 times, or more. As I understand it, there's two ways in which I can parallelize my simulations: 1) For a given number of processes, I can perform one simulation, where n agents are shared amongst all the processes, and their interactions are also shared amongst the processes. Since all of the agents are in the same 3-dimensional space, this seems it will involve a lot of non-local agents sharing their states amongst different processes. 2) Alternatively, I can run a given number of simulations at the same time, but each simulation occurs within one process, so every agent is local to its process, and this minimizes cross-process communication. I'm leaning towards the second method, but I'm not sure if one method is "better" than the other. Any thoughts? Thanks Sam -- Postdoctoral Fellow Mathematics and Computer Science Division Argonne National Laboratory 9700 S. Cass Avenue Argonne, IL 60439 http://www.linkedin.com/pub/sam-seaver/0/412/168 sam...@gm... <mailto:sam...@gm...%3cmailto:sam...@gm...> <mailto:sam...@gm...> (773) 796-7144<tel:%28773%29%20796-7144> "We shall not cease from exploration And the end of all our exploring Will be to arrive where we started And know the place for the first time." --T. S. Eliot ---------------------------------------------------------------------------- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Repast-interest mailing list Rep...@li... <mailto:Rep...@li...%3cmailto:Repast-interest@lists .sourceforge.net> <mailto:Rep...@li...> https://lists.sourceforge.net/lists/listinfo/repast-interest -- "Show me a movement that doesn't hate somebody and I will join it at once." ~ Robert Anton Wilson "Authority should derive from the consent of the governed, not from the threat of force!" Barbie, in Toy Story 3. ---------------------------------------------------------------------------- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___________________ ____________________________ Repast-interest mailing list Rep...@li... https://lists.sourceforge.net/lists/listinfo/repast-interest |