Re: [Pypar-developers] PyPar installation

[Romesh A. <rom...@gm...>]

Hi Ole,

That seems to fix compilation on the OS X machine- compiling gives

romesh: source > python compile_pypar_locally.py
mpicc -c mpiext.c
-I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6
-I/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy/core/include
-o mpiext.o -Wall
mpiext.c: In function ‘type_map’:
mpiext.c:127: warning: return makes integer from pointer without a cast
mpiext.c: In function ‘op_map’:
mpiext.c:168: warning: return makes integer from pointer without a cast
mpicc -bundle -flat_namespace -undefined suppress mpiext.o  -o mpiext.so
MPI initialised OK

However, test_pypar.py does not run correctly. An mpd session is opened on
the local machine with

romesh: source > mpdboot --totalnum=1 --ncpus=2

And then test_pypar.py is run using

romesh: source > mpirun -np 2 python test_pypar.py
Pypar (version 2.1.0) initialised MPI OK with 2 processors
I am processor 0 of 2 on node Romesh-MBP-3.local
I am processor 1 of 2 on node Romesh-MBP-3.local
Raw communication of numeric integer arrays OK
...
...
...
Raw reduce using pypar.BXOR OK
Traceback (most recent call last):
  File "test_pypar.py", line 953, in <module>
    assert status.error == 0
AssertionError
rank 0 in job 1  Romesh-MBP-3.local_49875   caused collective abort of all
ranks
  exit status of rank 0: return code 1


with the error originating at line 953, and status variable being set in
line 941

   941      B, status = pypar.receive(numproc-1, return_status = True)
   953      assert status.error == 0

Any ideas on what might be causing this?

We will try this on the Linux machine on Monday

Thanks for the quick reply!

Romesh

2009/12/19 Ole Nielsen <ole...@gm...>

> Hi Romesh
>
> When I build pypar on our 64 bit Intel cluster (Ubuntu 9.04) with OpenMPI,
> the compile string looks like this:
>
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC -I/usr/lib/openmpi/include
> -I/usr/lib/openmpi/include/openmpi
> -I/usr/lib/python2.6/dist-packages/numpy/core/include
> -I/usr/include/python2.6 -c mpiext.c -o build/temp.linux-x86_64-2.6/mpiext.o
> -fPIC
> gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
> build/temp.linux-x86_64-2.6/mpiext.o -L/usr/lib/openmpi/lib -lmpi -lopen-rte
> -lopen-pal -ldl -lnsl -lutil -lm -o
> build/lib.linux-x86_64-2.6/pypar/mpiext.so
>
> and all is well. The same is true on a 32 bit Ubuntu 9.10 system.
>
>
> Could I ask you to try to run
> python compile_pypar_locally.py
>
> That will compile the mpiext.so using mpicc directly (bypassing distutils)
> and the compile output should look something like this.
>
> nielso@alamba:~/sandpit/pypar/source$ python compile_pypar_locally.py
> mpicc -c mpiext.c -I/usr/include/python2.6
> -I/usr/lib/python2.6/dist-packages/numpy/core/include -o mpiext.o -Wall
> -fPIC
> mpicc -shared mpiext.o  -o mpiext.so -mcmodel=medium
> MPI initialised OK
>
>
> Then you should be able to run the unit tests
> mpirun -np 2 python test_pypar.py
>
> Please let us know how you go.
>
> Cheers
> Ole
>
>
>
>
> un...@in...
>
>
> On Fri, Dec 18, 2009 at 3:26 PM, Romesh Abeysuriya <rom...@gm...>wrote:
>
>> Hi Ole,
>>
>> We are trying to build PyPar on a couple of platforms, and are
>> encountering the same problem.
>>
>> The two architectures are:  64-bit intels (one Linux, the other OS X Snow
>> Leopard).  The common
>> error we encounter in both is associated with this message:
>>
>> gcc -pthread -shared build/temp.linux-x86_64-2.4/mpiext.o
>> -L/users/prl/rga112/mpich2-install-scucomp1/lib -lmpich -lopa -lpthread -lrt
>> -lopa' -o build/lib.linux-x86_64-2.4/pypar/mpiext.so
>> sh: -c: line 0: unexpected EOF while looking for matching `''
>> sh: -c: line 1: syntax error: unexpected end of file
>> _exec_command_posix failed (status=512)
>> /usr/bin/bash: -c: line 0: unexpected EOF while looking for matching `''
>> /usr/bin/bash: -c: line 1: syntax error: unexpected end of file
>> /usr/bin/bash: -c: line 0: unexpected EOF while looking for matching `''
>> /usr/bin/bash: -c: line 1: syntax error: unexpected end of file
>> error: Command "gcc -pthread -shared build/temp.linux-x86_64-2.4/mpiext.o
>> -L/users/prl/rga112/mpich2-install-scucomp1/lib -lmpich -lopa -lpthread -lrt
>> -lopa' -o build/lib.linux-x86_64-2.4/pypar/mpiext.so" failed with exit
>> status 2
>>
>> The main problem stems from this -lopa' element.  Grepping around your
>> source for "lopa" or "opa" didn't give us a clue how to fix this problem.
>>
>> Additionally, on Snow Leopard, problems arise because PyPar is trying to
>> create a Universal Binary, when only x86_64 MPICH libraries have been
>> compiled, so we are looking for a way to remove the extra -arch switches.
>>
>> We really would like to use PyPar in our project, so if you could please
>> advise us how to fix these problems, we'd be really grateful.
>>
>> For reference, here's some version information on our software stack we're
>> using to build PyPar:
>>
>> MPICH2-1.2.1
>> Python 2.6.1 (OS X)
>> Numpy 1.0.1 (current rpmforge repository version), latest SVN (OS X)
>> GCC 4.1.2 (Linux), 4.2.1 (OS X)
>> PyPar 2.1.0.66
>>
>>
>> Thanks and all the best...
>>
>> Romesh Abeysuriya
>> Jay Larson
>>
>>
>>
>