Re: [Pypar-developers] multicore machines with pypar
Brought to you by:
uniomni
Menu
▾
▴
Re: [Pypar-developers] multicore machines with pypar
|
From: Jordan A. <jc...@co...> - 2009-10-03 14:17:04
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
mpirun does work with executables. For example I'm able to use it with
cpi.c, which is included with MPICH. I don't know about timings.c (I
don't seem to have that file)...<br>
<br>
mpirun cannot launch python itself, however, I can run Python normally.<br>
<br>
My mpirun doesn't have a -x option. I looked through the manual for
mpich and couldn't find a similar option to what you describe.<br>
<br>
mpirun -np 4 C:Python25\bin\python.exe generates the same CreateProcess
error.<br>
<br>
If you have any other suggestions I would be happy to hear them.
Otherwise, this seems to be more of a mpich/python problem than a pypar
problem, so I will followup with the mpich support team...<br>
<br>
Thank you for your time,<br>
--Jordan<br>
<br>
<br>
Ole Nielsen wrote:
<blockquote
cite="mid:b4a...@ma..."
type="cite">This is a step forward. Can I just confirm:<br>
mpirun works with executables (e.g. timings.c)<br>
mpirun cannot launch python itself<br>
however, you can run Python normally.<br>
<br>
Is ths correct?<br>
<br>
If so, I think the problem would relate to your system path (i.e. how
it can find Python)and how mpirun interprets it. I seem to remember,
that the -x option with MPIRUN does something to make sure every
process is aware of environment variables such as PATH.<br>
<br>
So perhaps you could try something like<br>
<br>
mpirun -x PATH -np 4 python<br>
<br>
<br>
or failing that, juse the full pathname to Python (from memory)<br>
<br>
mpirun -np 4 C:Python25\bin\python.exe <br>
<br>
<br>
Let us know how you go<br>
<br>
Cheers<br>
Ole<br>
<div class="gmail_quote">On Sat, Oct 3, 2009 at 5:21 AM, Jordan Atlas
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:jc...@co...">jc...@co...</a>></span> wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">Ole,<br>
<br>
I should add that this program doesn't seem to be specific to
pypar. Rather, I can't seem to run anything pythonic. For example, if
I type 'mpirun -np 4 python' alone, it fails with the same error. I
can, however, run parallel .exe files using mpirun.<br>
<br>
I just wanted to say this in case it provides a clue as to what I
may be doing incorrectly.
<div>
<div class="h5"><br>
<br>
Thanks,<br>
<br>
--Jordan<br>
<br>
Ole Nielsen wrote:
<blockquote type="cite">Could you try<br>
<br>
C:\Python25\Lib\site-packages\pypar>mpirun -np 4 python test_pypar.py<br>
<br>
and let me know if that gives the same error?<br>
<br>
Has anyone else on the list seen this?<br>
<br>
Cheers and thanks<br>
Ole<br>
<br>
<br>
<div class="gmail_quote">On Thu, Oct 1, 2009 at 2:49 AM, Jordan
Atlas <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:jc...@co..." target="_blank">jc...@co...</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">Hi Ole,<br>
<br>
Thanks for your response. Following your advice, I was able to
verify that I can get MPICH to run multiple processes that utilizes
both of my machines' processors.<br>
<br>
Now I'm having trouble getting the pypar tests to run. If I type:<br>
<br>
C:\Python25\Lib\site-packages\pypar>mpirun -np 4 test_pypar.py<br>
<br>
Then I get the error:<br>
<br>
Unable to launch 'C:\Python25\Lib\site-packages\pypar\test_pypar.py',
error 8: CreateProcess failed:<br>
Not enough storage is available to process this command.<br>
<br>
I assume it is talking about "RAM" storage, but my machine has over 2GB
of RAM free right now, so I don't understand the problem. Am I calling
the program correctly?<br>
<br>
Note that if I run test_pypar without calling mpirun, I get the output:<br>
<br>
C:\Python25\Lib\site-packages\pypar>test_pypar.py<br>
Pypar (version 2.1.0) initialised MPI OK with 1 processors<br>
I am processor 0 of 1 on node jca33-305.<br>
<br>
Please let me know if my question is unclear.<br>
<br>
Thanks,<br>
<font color="#888888"><br>
--Jordan</font>
<div>
<div><br>
<br>
<br>
Ole Nielsen wrote:
<blockquote type="cite">Hi Jordan<br>
<br>
Great that you got pypar installed!<br>
<br>
As for the number of processors it is really a matter for the
underlying MPI implementation.<br>
If you run a C program, such as the include timing measurement, using
MPI you would have the same issue.<br>
<br>
Depending on the MPI implementation there are different ways of telling
how many CPUs you got.<br>
With LAM MPI, there is a .machines file, with OPENMPI it is a matter
for mpirun to specify.<br>
<br>
Typically, you run you parallel program on e.g. 8 processes using<br>
<br>
mpirun -np 8 <parallel_program><br>
<br>
<br>
Hope this helps<br>
<br>
Cheers<br>
Ole<br>
<br>
<br>
PS - everything is better if you use Linux, though :-)<br>
<br>
<br>
<div class="gmail_quote">On Thu, Sep 24, 2009 at 4:16 AM,
Jordan
Atlas <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:jc...@co..." target="_blank">jc...@co...</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">Hi Ole,<br>
<br>
After carefully following the readme instuctions and making a
couple changes to the compile files, I was able to get pypar installed
on my machine. However, it reports that my machine only has one
processor, even though it has two.<br>
<br>
Is there anything in particular I need to do to make sure pypar is
aware of multiple processors in windows? <br>
<br>
Thanks,<br>
<br>
--Jordan<br>
<font color="#888888"><br>
<div>-- <br>
<b>Jordan Atlas</b> <br>
Email: <a moz-do-not-send="true" href="mailto:jc...@co..."
target="_blank">jc...@co...</a> <br>
Chemical and Biomolecular Engineering <br>
Cornell University </div>
</font></div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</body>
</html>
|
