[Pypar-developers] Successful compilation on x86_64 cluster

[Ole N. <ole...@gm...>]

Hello Craig and tanks for your interest in pypar and the compilation
information.

Regarding Python finding its packages when invoked using mpirun I
think it is a matter of exporting the environment variable PYTHONPATH
using something like this

mpirun -x PYTHONPATH .....

I have made an alias taking care of that.

Please let me know if this addresses you problem.

Cheers
Ole

> From: Craig Finch <cf...@ie...>
> To: pyp...@li...
> Date: Fri, 17 Oct 2008 14:55:15 -0700 (PDT)
> Subject: Successful compilation on x86_64 cluster
> Here' s a short summary of what I did to compile pypar and install it in my home directory on our Intel Xeon 64 bit cluster with Intel compilers and MPI:
>
> Compile and install pypar:
> python setup.py config --compiler=intel install --home=~
>
> The mpiext library compiles and installs without error, but it can't be loaded by Python.  I compiled the binary manually with the following command:
>
> icc -fPIC -i-dynamic -D_EM64T_ -D_SMP_ -DUSE_HEADER_CACHING -DONE_SIDED -DMPID_USE_SEQUENCE_NUMBERS -D_SHMEM_COLL_ -DRDMA_CM -I/usr/include/python2.4 -I/home/cfinch/lib64/python/numpy/core/include -I/usr/include -O2 -I/usr/mpi/intel/mvapich2-0.9.8-15/include -L/usr/mpi/intel/mvapich2-0.9.8-15/lib -Wl,-rpath -Wl,/usr/mpi/intel/mvapich2-0.9.8-15/lib -lmpich -L/usr/lib64 -lrdmacm -libverbs -libumad -lpthread -lrt -I/usr/include/python2.4 -c mpiext.c -o mpiext.o
>
> I then linked it using mpicc:
> mpicc -shared mpiext.o -o mpiext.so -mcmodel=medium
>
> And copied it to
> cp mpiext.so ~/lib64/python/pypar/
>
> I don't claim to understand the options I had to use to get icc to compile the extension correctly.  I basically looked at the files setup.py and compile.py, and kludged together options until something worked.  It would be great if someone could explain how this process works and include it in the docs for future reference.
>
> I do have one question.  When I submit a pypar job using mpirun, python can't seem to find the Python packages (numpy, pypar, etc). that are installed in my home directory (~/lib64/python).  I verified that ~/lib64/python is in the Python path for the Python processes that are started by mpirun, but they still can't find my packages.  Any suggestions how to make them available?  I don't think our cluster admin has any experience with parallel Python.
>
>   Craig Finch
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
>
>
> ---------- Forwarded message ----------
> From: pyp...@li...
> To:
> Date:
> Subject: confirm 469f2ca2493a178d1ef6720486793ce4522b0f2f
> If you reply to this message, keeping the Subject: header intact,
> Mailman will discard the held message.  Do this if the message is
> spam.  If you reply to this message and include an Approved: header
> with the list password in it, the message will be approved for posting
> to the list.  The Approved: header can also appear in the first line
> of the body of the reply.
>