Re: [Plplot-devel] This next release cycle should finish in early 2018

[Alan W. I. <ir...@be...>]

On 2017-08-26 14:36+0200 Ole Streicher wrote:

> On 26.08.2017 12:43, Alan W. Irwin wrote:
[...]
>> This simply means you need to install the python3-numpy package
>> (to be consistent with the above python3), and all should be well.
>
> Thanks With installing python3-numpy as well, it works. I didn't update
> the dependencies from plplot-12 (where we had  Python-2 bindings only).

Hi Ole:

I feel pretty strongly this discussion belongs on the PLplot development
list so I have put it back there again.

When you immediately posted that Python failure for plplot-5.13.0, I
was beginning to wonder if I would need to quickly release 5.13.1 to
fix whatever problem you had found.  So it is a big relief to me to
hear from you there is no plplot-5.13.0 issue in this regard and
installing the right version of the numpy package completely fixed the
issue.  :-)

>
> This opens however a new can of worms for me: If we are able to build
> Python-2 *and* Python-3 packages, it would be great to do so. Also, I'd
> like to build it for all supported Python minor versions (3.5 and 3.6
> for the moment, and Python 2.7) during the package build. Is there any
> way to configure or patch this? Supporting all Debian-supported versions
> is part of our Python policy.

That policy might need some adjustments, i.e., Python2 support should
be allowed, but I am pretty sure it should not be mandatory because
upstream Python developer support of Python2 is obviously not as active as
Python3 support, and that sometimes causes problems for Python2.

For an example of this, consider our examples/python/pytkdemo
interactive example that demonstrates use of our Tcl-based plframe
GUI under Python.  This is a complex beast with namespace issues
I have not yet been able to track down so to work around those I
do a double import of bindings/python/Plframe.py using

import Plframe
from Plframe import *

Such double imports to work around namespace issues are fairly common,
but from my python2 interactive testing experience this double import
would sometimes generate *.pyc corruption issues for the
bindings/python/Plframe.pyc file which should automatically be
generated by Python by that first import *only* if
bindings/python/Plframe.pyc does not exist or if
bindings/python/Plframe.py has been updated.

I consulted with a Python developer concerning this issue, and his
advice was to move to Python3 because there were known race conditions
that had been fixed in that latter case, but he wasn't sure if they
had been fixed for Python2, and he strongly implied that Python
developers tended to concentrate on Python3 for obvious reasons so
issues did slip through the cracks for Python2, and this corruption
issue was probably just one example of that.

So once (thanks to Hazen's efforts) PLplot was ready for Python3, I
started to use it almost exclusively to test it, and my tests showed
complete reliability and also no bindings/python/Plframe.pyc
corruption issue. So based on that demonstrated Python2 corruption
issue that Python3 solved, I updated our build system to make Python3
the preference if both Python3 and Python2 were available. And ever
since that change I have never run into that *.pyc corruption issue.
So my experience certainly jibes with what the Python developer said.

Now it could be that developer had some axe to grind, and he was
therefore too discouraging about the reliability of Python2.  So it is
possible that race condition for Python2 that seems to be the source
of this *.pyc corruption issue has been fixed in later Python2
versions than I currently have with Debian Jessie (2.7.9-2+deb8u1).
But my experience this Python developer's attitude toward Python2
should certainly give Debian developers some food for thought about
any mandatory Python2 requirements they may have in their Python
policy.

Those Debian policy issues aside, Plplot continues to work fine for
python2 aside from the relatively rare corruption issue for
bindings/python/Plframe.pyc which is straightforward (although
annoying) to work around by removing that corrupt file so that python
will regenerate it (typically in reliable fashion for quite a while).
Furthermore, this issue will likely go away once I figure out how to
move to a single name-spaced import of PLframe, and I suspect, in any
case, you likely don't bother to package the still somewhat
experimental examples/python/pytkdemo along with the special Plframe
module and Pltk_init extension module that are unique needs of that
example.

So I suggest if you still want to build a python2 package for PLplot,
that you do that in a separate build where both python2 and
python-numpy are available and you use the cmake option
-DFORCE_PYTHON2=ON.  In that case and assuming that Python2 and
python-numpy are installed, our build system will do the right thing
with regard to building the Python2 plplot module using consistent
Python2 library and numpy versions, and also use the correct install
location for the resulting Python2 plplot module.  Also, this extra
build does not need to cost your very much since you can use
(-DFORCE_PYTHON2=ON -DDEFAULT_NO_DEVICES=ON -DPLD_extqt=ON
-DDEFAULT_NO_BINDINGS=ON -DENABLE_python=ON -DENABLE_qt=ON
-DENABLE_pyqt4=ON) to, for example, configure a build and install of
just the minimum PLplot components you need to support both python and
pyqt4 under Python2.

Alternatively, you might be able to trace through the
-DFORCE_PYTHON2=ON logic in cmake/modules/python.cmake, and provide a
patch to simultaneously support both Python2 and Python3 builds. But
that is a lot of work with a lot of chance to introduce build-system
errors/inconsistencies so I think the above multiple build approach
is much better.

>
> One solution could be to write a standard "setup.py", which would allow
> to run the Debian Python build machinery here. This automatically builds
> the Python modules for all supported versions, and installs them in the
> preferred path. Would you think it is problematic to write one? (I am
> not asking that you do it yourself; I just want to know if it is worth
> the effort).

Let's face it, the setup.py approach requires you to implement a
python-based build system and build systems are always tricky.  And
that is especially true when you are trying to simultaneously support
more than one version, see comment above.  So I think you are talking
about a considerable amount of implementation work, maintenance, and
potential errors.  In the distant past we used a setup.py approach for
our Python components but eventually abandoned it once we figured out
everything we needed to do with our autotools-based build system at
that time and we continued without using the setup.py approach when we
implemented the CMake-based build system that replaced that
autotools-based build system a decade ago.

In sum, that decision about implementing a setup.py approach is up to
you, but my advice is to not bother with that and instead use our
existing CMake-based build system to do what you want using the above
multiple build approach.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________