Summary from Jordan Atlas in regard to running Pypar on Windows using MPICH.


---------- Forwarded message ----------
From: Jordan Atlas <jca33@cornell.edu>
Date: Mon, Oct 5, 2009 at 12:24 AM
Subject: Re: multicore machines with pypar
To: Ole Nielsen <ole.moller.nielsen@gmail.com>


Hi Ole,

    Thank you for attending to my questions even though you are in transit!  I think I have found the answer to my problems...  Basically it's to pass in the path to python when running pypar based scripts.  You suggested it earlier, but I wasn't careful enough to make sure I was passing in the correct path (on windows there is no 'bin').  Sorry about that.

    There are some other details related to the email you sent most recently, which are below in case you are interested.  Also, the actual testing of pypar fails on a particular test so I'll probably email you again after I've investigated that a bit.

1)  I found network_timing.c (earlier I had just searched for "timings" with an s so I missed it). 

FYI, it took a while to get network_timing.exe to run properly.  It would crash every time I ran it with a stack overflow exception.  I was able to prevent this by changing the constant "MAXM" to 50,000 instead of 500,000.  Just letting you know in case this is unexpected.

I can now execute the network_timing example (in C) with this output:

MAXM = 50000, number of processors = 2
Measurements are repeated 10 times for reliability
I am process 0 on jca33-305.
I am process 1 on jca33-305.
Timing overhead is 0.000000 seconds

Run 1 of 10
Run 2 of 10
Run 3 of 10
Run 4 of 10
Run 5 of 10
Run 6 of 10
Run 7 of 10
Run 8 of 10
Run 9 of 10
Run 10 of 10
Bytes transferred   time (micro seconds)
                    min        avg        max
----------------------------------------------
         8            12         19         41
     40008            70         97        118
     80008            90        133        155
    120008            85        160        186
    160008           126        206        280
    200008           115        225        259
    240008           132        259        346
    280008           105        250        322
    320008           156        269        374
    360008           161        293        398

Linear regression on best timings (t = t_l + t_b * bytes):
  t_b = 0.000324
  t_l = 47.273603
  Estimated relative variance = 0.000000001

Estimated bandwith (1/t_b):  3082.216 Mb/s
Estimated latency:           12 micro s


2)  When I try "mpirun -np 2 network_timing.py", I get a CreateProcess error:

Unable to launch 'C:\Python25\Lib\site-packages\pypar\pypar_2.1.0_66\source\network_timing.py', erro

r 8: CreateProcess failed: Not enough storage is available to process this command.


This is a different error than I get when just trying to open python with mpirun (though fixing this one might just leave me with the old error). 

Thinking that this might also be related to MAXM, I tried changing the value of MAXM in network_timing.py, and it did NOT fix the error.

3) One question is how MPICH allows the environment variables to passed on to the MPI processes.

In the process of trying to figure out how to pass this information to MPICH, I tried to pass pythons path in on the command line as you suggested earlier.  This is when I realized I had used the wrong path earlier (it's C:\python25\python as opposed to C:\python25\bin\python).  Once I fixed this I was able to run the tests with 'mpirun -np 4 C:\python25\python test_pypar.py' (except for one failure towards the end).  Also, network_timings.py works as well!


Thanks!

--Jordan

Ole Nielsen wrote:
OK

I you are right the first thing that must be resolved is for mpich to launch Python.
But many othyers on the mailing list have used MPICH so it seems strange.

I am in transit in Indonesia at the moment, so everything I say is from memory.
There is a c program that ships with pypar called eiter timings.c or network_timimngs.c or something like that. It is a pure C program that measures bandwidth using MPI. There is also a python version doing the same thing using pypart.

As for the option -x it might be -X (capital). This is the case in OPENMPI at least.
If is doesn;'t have gthat option, I think one question is how MPICH allows the environment variables to passed on to the MPI processes.

Cheers and please let us know if you find the problem.
Ole

PS Otherwise, switch to Linux and OpenMPI :-) 

On Sun, Oct 4, 2009 at 1:16 AM, Jordan Atlas <jca33@cornell.edu> wrote:
mpirun does work with executables.  For example I'm able to use it with cpi.c, which is included with MPICH.  I don't know about timings.c (I don't seem to have that file)...

mpirun cannot launch python itself, however, I can run Python normally.

My mpirun doesn't have a -x option.  I looked through the manual for mpich and couldn't find a similar option to what you describe.

mpirun -np 4 C:Python25\bin\python.exe generates the same CreateProcess error.

If you have any other suggestions I would be happy to hear them.  Otherwise, this seems to be more of a mpich/python problem than a pypar problem, so I will followup with the mpich support team...

Thank you for your time,
--Jordan



Ole Nielsen wrote:
This is a step forward. Can I just confirm:
mpirun works with executables (e.g. timings.c)
mpirun cannot launch python itself
however, you can run Python normally.

Is ths correct?

If so, I think the problem would relate to your system path (i.e. how it can find Python)and how mpirun interprets it. I seem to remember, that the -x option with MPIRUN does something to make sure every process is aware of environment variables such as PATH.

So perhaps you could try something like

mpirun -x PATH -np 4 python


or failing that, juse the full pathname to Python (from memory)

mpirun -np 4 C:Python25\bin\python.exe  


Let us know how you go

Cheers
Ole
On Sat, Oct 3, 2009 at 5:21 AM, Jordan Atlas <jca33@cornell.edu> wrote:
Ole,

    I should add that this program doesn't seem to be specific to pypar.  Rather, I can't seem to run anything pythonic.  For example, if I type 'mpirun -np 4 python' alone, it fails with the same error.  I can, however, run parallel .exe files using mpirun.

    I just wanted to say this in case it provides a clue as to what I may be doing incorrectly.


Thanks,

--Jordan

Ole Nielsen wrote:
Could you try

C:\Python25\Lib\site-packages\pypar>mpirun -np 4 python test_pypar.py

and let me know if that gives the same error?

Has anyone else on the list seen this?

Cheers and thanks
Ole


On Thu, Oct 1, 2009 at 2:49 AM, Jordan Atlas <jca33@cornell.edu> wrote:
Hi Ole,

    Thanks for your response.  Following your advice, I was able to verify that I can get MPICH to run multiple processes that utilizes both of my machines' processors.

    Now I'm having trouble getting the pypar tests to run.  If I type:

C:\Python25\Lib\site-packages\pypar>mpirun -np 4 test_pypar.py

Then I get the error:

Unable to launch 'C:\Python25\Lib\site-packages\pypar\test_pypar.py', error 8: CreateProcess failed:
 Not enough storage is available to process this command.

I assume it is talking about "RAM" storage, but my machine has over 2GB of RAM free right now, so I don't understand the problem.  Am I calling the program correctly?

Note that if I run test_pypar without calling mpirun, I get the output:

C:\Python25\Lib\site-packages\pypar>test_pypar.py
Pypar (version 2.1.0) initialised MPI OK with 1 processors
I am processor 0 of 1 on node jca33-305.

    Please let me know if my question is unclear.

Thanks,

--Jordan



Ole Nielsen wrote:
Hi Jordan

Great that you got pypar installed!

As for the number of processors it is really a matter for the underlying MPI implementation.
If you run a C program, such as the include timing measurement, using MPI you would have the same issue.

Depending on the MPI implementation there are different ways of telling how many CPUs you got.
With LAM MPI, there is a .machines file, with OPENMPI it is a matter for mpirun to specify.

Typically, you run you parallel program on e.g. 8 processes using

mpirun -np 8 <parallel_program>


Hope this helps

Cheers
Ole


PS - everything is better if you use Linux, though :-)


On Thu, Sep 24, 2009 at 4:16 AM, Jordan Atlas <jca33@cornell.edu> wrote:
Hi Ole,

    After carefully following the readme instuctions and making a couple changes to the compile files, I was able to get pypar installed on my machine.  However, it reports that my machine only has one processor, even though it has two.

    Is there anything in particular I need to do to make sure pypar is aware of multiple processors in windows? 

Thanks,

--Jordan

--
Jordan Atlas
Email: jca33@cornell.edu
Chemical and Biomolecular Engineering
Cornell University