Re: [Plplot-devel] python extension linking, was RE: python branch now "merged"

[Alan W. I. <ir...@be...>]

On 2009-07-01 00:09-0500 Geoffrey Furnish wrote:

> [...]Now, what I find is that if I configure Python (today I tested 2.6.2, but I
> believe from prior experience that 2.5.x is the same in these regards) by
> just saying:
>
>     cd Python-2.6.2
>     configure --prefix=$PREFIX
>     make
>     make install
>
> Then what happens is that libpython2.6.a is installed in $PREFIX/lib, but
> libpython2.6.so is NOT installed.
>
> To get that installed, you have to do:
>
>    cd Python-2.6.2
>    make distclean     # Critical to avoid "configure hysteresis"
>    configure --prefix=$PREFIX --enable-shared
>    make
>    make install
>
> If you do that, then libpython2.6.so shows up in $PREFIX/lib.
>
> HOWEVER: PLplot's Python binding needs numpy.  And here comes the rub.  It
> turns out that Numpy's installation will *not work* if the python was built
> --enable-shared.  If you don't build --enable-shared, then you can go into
> the numpy distribution and run $PREFIX/bin/python setup.py install, and it
> goes right where you want it, under $PREFIX.  But if your installed
> $PREFIX/bin/python was configured with --enable-shared, then KABOOM, when you
> run numpy's installer, it tries to write into /usr/lib64/*, and ignores the
> PREFIX to which your python was configured.

One possibility is that the automake based build you seem to be using
for python may be outdated.  Surely, you can now build a new version of python
using an old version of python and the setup.py approach?  It's possible if
you do that, then numpy will work fine.

Another possibility is that in the _numpy_ setup.py, there was a simple
oversight where they hardcoded the system path rather than taking into
account the possibility that you are building and installing all
python-related items with your own PREFIX.

It's been years since I have debugged a setup.py file, but IIRC they are
usually pretty short so it may not be that difficult for you to debug this
issue. But if you cannot do that, then a numpy bug report is certainly
indicated.

> [...]The only solution to this unholy mess that I have been able to find, is to
> leave PYTHON_LIBRARIES off the link line for the PLplot python extension
> modules.  On Linux at least, I know that this omission is inconsequential.
> When you actually load the extension modules at run time, the symbols are
> filled in from the dynloading executor process (which in my case will turn
> out to be an embellished python with some application-specific python
> extension stuff compiled in).  And all is well.

This seems like a reasonable temporary workaround for the bug in numpy
installation, but the problem is we have already historically gone through
platform testing of excluding PYTHON_LIBRARIES, and it caused problems.
Thus, removing PYTHON_LIBRARIES now will always introduce a question mark
about the reliability of our python extension module builds.  (That concern
would be reduced by redoing the platform testing again, but even if it
worked now for the platforms we are able to test, it still leaves a question
mark for the platforms we are not able to test at this time.)

To avoid that question mark, I suggest the following compromise.

Put in a cmake option (EXCLUDE_PYTHON_LIBRARIES) which you can set for your
needs, and which everyone else can ignore.  We can then remove that
workaround once the fundamental issue with numpy is fixed.

Once I have the okay from you for this approach, I can implement it in about
5 minutes inside cmake/modules/python.cmake by setting PYTHON_LIBRARIES to
nothing if EXCLUDE_PYTHON_LIBRARIES is true.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________