Summary from Jordan Atlas in regard to running Pypar on Windows using MPICH.
I you are right the first thing that must be resolved is for mpich to launch Python.
But many othyers on the mailing list have used MPICH so it seems strange.
I am in transit in Indonesia at the moment, so everything I say is from memory.
There is a c program that ships with pypar called eiter timings.c or network_timimngs.c or something like that. It is a pure C program that measures bandwidth using MPI. There is also a python version doing the same thing using pypart.
As for the option -x it might be -X (capital). This is the case in OPENMPI at least.
If is doesn;'t have gthat option, I think one question is how MPICH allows the environment variables to passed on to the MPI processes.
Cheers and please let us know if you find the problem.
PS Otherwise, switch to Linux and OpenMPI :-)
On Sun, Oct 4, 2009 at 1:16 AM, Jordan Atlas <firstname.lastname@example.org> wrote:
mpirun does work with executables. For example I'm able to use it with cpi.c, which is included with MPICH. I don't know about timings.c (I don't seem to have that file)...
mpirun cannot launch python itself, however, I can run Python normally.
My mpirun doesn't have a -x option. I looked through the manual for mpich and couldn't find a similar option to what you describe.
mpirun -np 4 C:Python25\bin\python.exe generates the same CreateProcess error.
If you have any other suggestions I would be happy to hear them. Otherwise, this seems to be more of a mpich/python problem than a pypar problem, so I will followup with the mpich support team...
Thank you for your time,
Ole Nielsen wrote:This is a step forward. Can I just confirm:
mpirun works with executables (e.g. timings.c)
mpirun cannot launch python itself
however, you can run Python normally.
Is ths correct?
If so, I think the problem would relate to your system path (i.e. how it can find Python)and how mpirun interprets it. I seem to remember, that the -x option with MPIRUN does something to make sure every process is aware of environment variables such as PATH.
So perhaps you could try something like
mpirun -x PATH -np 4 python
or failing that, juse the full pathname to Python (from memory)
mpirun -np 4 C:Python25\bin\python.exe
Let us know how you go
On Sat, Oct 3, 2009 at 5:21 AM, Jordan Atlas <email@example.com> wrote:
I should add that this program doesn't seem to be specific to pypar. Rather, I can't seem to run anything pythonic. For example, if I type 'mpirun -np 4 python' alone, it fails with the same error. I can, however, run parallel .exe files using mpirun.
I just wanted to say this in case it provides a clue as to what I may be doing incorrectly.
Ole Nielsen wrote:Could you try
C:\Python25\Lib\site-packages\pypar>mpirun -np 4 python test_pypar.py
and let me know if that gives the same error?
Has anyone else on the list seen this?
Cheers and thanks
On Thu, Oct 1, 2009 at 2:49 AM, Jordan Atlas <firstname.lastname@example.org> wrote:
Thanks for your response. Following your advice, I was able to verify that I can get MPICH to run multiple processes that utilizes both of my machines' processors.
Now I'm having trouble getting the pypar tests to run. If I type:
C:\Python25\Lib\site-packages\pypar>mpirun -np 4 test_pypar.py
Then I get the error:
Unable to launch 'C:\Python25\Lib\site-packages\pypar\test_pypar.py', error 8: CreateProcess failed:
Not enough storage is available to process this command.
I assume it is talking about "RAM" storage, but my machine has over 2GB of RAM free right now, so I don't understand the problem. Am I calling the program correctly?
Note that if I run test_pypar without calling mpirun, I get the output:
Pypar (version 2.1.0) initialised MPI OK with 1 processors
I am processor 0 of 1 on node jca33-305.
Please let me know if my question is unclear.
Ole Nielsen wrote:Hi Jordan
Great that you got pypar installed!
As for the number of processors it is really a matter for the underlying MPI implementation.
If you run a C program, such as the include timing measurement, using MPI you would have the same issue.
Depending on the MPI implementation there are different ways of telling how many CPUs you got.
With LAM MPI, there is a .machines file, with OPENMPI it is a matter for mpirun to specify.
Typically, you run you parallel program on e.g. 8 processes using
mpirun -np 8 <parallel_program>
Hope this helps
PS - everything is better if you use Linux, though :-)
On Thu, Sep 24, 2009 at 4:16 AM, Jordan Atlas <email@example.com> wrote:
After carefully following the readme instuctions and making a couple changes to the compile files, I was able to get pypar installed on my machine. However, it reports that my machine only has one processor, even though it has two.
Is there anything in particular I need to do to make sure pypar is aware of multiple processors in windows?
Chemical and Biomolecular Engineering