Re: [Pypar-developers] multicore machines with pypar
Brought to you by:
uniomni
From: Jordan A. <jc...@co...> - 2009-10-03 14:17:04
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type"> <title></title> </head> <body bgcolor="#ffffff" text="#000000"> mpirun does work with executables. For example I'm able to use it with cpi.c, which is included with MPICH. I don't know about timings.c (I don't seem to have that file)...<br> <br> mpirun cannot launch python itself, however, I can run Python normally.<br> <br> My mpirun doesn't have a -x option. I looked through the manual for mpich and couldn't find a similar option to what you describe.<br> <br> mpirun -np 4 C:Python25\bin\python.exe generates the same CreateProcess error.<br> <br> If you have any other suggestions I would be happy to hear them. Otherwise, this seems to be more of a mpich/python problem than a pypar problem, so I will followup with the mpich support team...<br> <br> Thank you for your time,<br> --Jordan<br> <br> <br> Ole Nielsen wrote: <blockquote cite="mid:b4a...@ma..." type="cite">This is a step forward. Can I just confirm:<br> mpirun works with executables (e.g. timings.c)<br> mpirun cannot launch python itself<br> however, you can run Python normally.<br> <br> Is ths correct?<br> <br> If so, I think the problem would relate to your system path (i.e. how it can find Python)and how mpirun interprets it. I seem to remember, that the -x option with MPIRUN does something to make sure every process is aware of environment variables such as PATH.<br> <br> So perhaps you could try something like<br> <br> mpirun -x PATH -np 4 python<br> <br> <br> or failing that, juse the full pathname to Python (from memory)<br> <br> mpirun -np 4 C:Python25\bin\python.exe <br> <br> <br> Let us know how you go<br> <br> Cheers<br> Ole<br> <div class="gmail_quote">On Sat, Oct 3, 2009 at 5:21 AM, Jordan Atlas <span dir="ltr"><<a moz-do-not-send="true" href="mailto:jc...@co...">jc...@co...</a>></span> wrote:<br> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div bgcolor="#ffffff" text="#000000">Ole,<br> <br> I should add that this program doesn't seem to be specific to pypar. Rather, I can't seem to run anything pythonic. For example, if I type 'mpirun -np 4 python' alone, it fails with the same error. I can, however, run parallel .exe files using mpirun.<br> <br> I just wanted to say this in case it provides a clue as to what I may be doing incorrectly. <div> <div class="h5"><br> <br> Thanks,<br> <br> --Jordan<br> <br> Ole Nielsen wrote: <blockquote type="cite">Could you try<br> <br> C:\Python25\Lib\site-packages\pypar>mpirun -np 4 python test_pypar.py<br> <br> and let me know if that gives the same error?<br> <br> Has anyone else on the list seen this?<br> <br> Cheers and thanks<br> Ole<br> <br> <br> <div class="gmail_quote">On Thu, Oct 1, 2009 at 2:49 AM, Jordan Atlas <span dir="ltr"><<a moz-do-not-send="true" href="mailto:jc...@co..." target="_blank">jc...@co...</a>></span> wrote:<br> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div bgcolor="#ffffff" text="#000000">Hi Ole,<br> <br> Thanks for your response. Following your advice, I was able to verify that I can get MPICH to run multiple processes that utilizes both of my machines' processors.<br> <br> Now I'm having trouble getting the pypar tests to run. If I type:<br> <br> C:\Python25\Lib\site-packages\pypar>mpirun -np 4 test_pypar.py<br> <br> Then I get the error:<br> <br> Unable to launch 'C:\Python25\Lib\site-packages\pypar\test_pypar.py', error 8: CreateProcess failed:<br> Not enough storage is available to process this command.<br> <br> I assume it is talking about "RAM" storage, but my machine has over 2GB of RAM free right now, so I don't understand the problem. Am I calling the program correctly?<br> <br> Note that if I run test_pypar without calling mpirun, I get the output:<br> <br> C:\Python25\Lib\site-packages\pypar>test_pypar.py<br> Pypar (version 2.1.0) initialised MPI OK with 1 processors<br> I am processor 0 of 1 on node jca33-305.<br> <br> Please let me know if my question is unclear.<br> <br> Thanks,<br> <font color="#888888"><br> --Jordan</font> <div> <div><br> <br> <br> Ole Nielsen wrote: <blockquote type="cite">Hi Jordan<br> <br> Great that you got pypar installed!<br> <br> As for the number of processors it is really a matter for the underlying MPI implementation.<br> If you run a C program, such as the include timing measurement, using MPI you would have the same issue.<br> <br> Depending on the MPI implementation there are different ways of telling how many CPUs you got.<br> With LAM MPI, there is a .machines file, with OPENMPI it is a matter for mpirun to specify.<br> <br> Typically, you run you parallel program on e.g. 8 processes using<br> <br> mpirun -np 8 <parallel_program><br> <br> <br> Hope this helps<br> <br> Cheers<br> Ole<br> <br> <br> PS - everything is better if you use Linux, though :-)<br> <br> <br> <div class="gmail_quote">On Thu, Sep 24, 2009 at 4:16 AM, Jordan Atlas <span dir="ltr"><<a moz-do-not-send="true" href="mailto:jc...@co..." target="_blank">jc...@co...</a>></span> wrote:<br> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div bgcolor="#ffffff" text="#000000">Hi Ole,<br> <br> After carefully following the readme instuctions and making a couple changes to the compile files, I was able to get pypar installed on my machine. However, it reports that my machine only has one processor, even though it has two.<br> <br> Is there anything in particular I need to do to make sure pypar is aware of multiple processors in windows? <br> <br> Thanks,<br> <br> --Jordan<br> <font color="#888888"><br> <div>-- <br> <b>Jordan Atlas</b> <br> Email: <a moz-do-not-send="true" href="mailto:jc...@co..." target="_blank">jc...@co...</a> <br> Chemical and Biomolecular Engineering <br> Cornell University </div> </font></div> </blockquote> </div> <br> </blockquote> <br> </div> </div> </div> </blockquote> </div> <br> </blockquote> <br> </div> </div> </div> </blockquote> </div> <br> </blockquote> <br> </body> </html> |