[Plplot-devel] Plans for a last Python2 versus Python3 fix, Python2 deprecation by 5.16.0 and removal by 5.17.0

[Alan W. I. <Ala...@gm...>]

On 2019-08-06 13:18+0100 Phil Rosenberg wrote:

> Sorry, my last email wasn't sent to the list - see below.
>
> I think I have found the solution, by changing the shebang to python2
> rather than python.
>
> I'm going to commit this change as it is a change that only affects us
> developers. Alan - if you could test to check it works on your linux
> system, that would be great.

Hi Phil:

Sorry for my own delay in answering you which was caused by gmail
troubles (started by a bad recovery) that has finally been
straightened out yesterday (Thursday) after 5 days of no gmail access
for me.

Glad you found a solution for the scripts/convert_comment.py issue.
That's a good solution in the short term but unfortunately not in the
long term.  The problem is Python2 is near its end of life (see, for
example, the discussion in <https://lwn.net/Articles/756628/>).  And
in any case very likely due to anticipation by Python developers of
that end of life, my experience is Python2 is buggy and not as well
supported as Python3.  So in any case I planned to deprecate the
PLplot Python2 support soon and remove it completely fairly soon
after.  So I will need to generalize your above solution for the
Python3 case.

[The essential documentation for this general
solution](https://portingguide.readthedocs.io/en/latest/exceptions.html)
shows there are both "except" and "raise" differences in Python2
versus Python3 syntax for handling exceptions.  Furthermore, there are
a number of different places in our Python code where we use "except"
and "raise", and apparently they are all currently implemented in
Python2 syntax (as you discovered for the case of the raise command in
scripts/convert_comment.py).

My plan for dealing with this issue is to change all "except" and
"raise" Python commands to Python3 syntax only (this will include
scripts/convert_comment.py and therefore changing your shebang from
python2 to python3 and testing that change works here under python3)
and deprecation of Python2 (meaning users will have to go out of their
way to specify -DPL_DEPRECATED_PYTHON=ON in order to still access
Python2).  This planned deprecation in the current release cycle
should propagate to the forthcoming 5.16.0 release, and the further
plan is to remove Python2 completely early in the next release cycle
that will lead to 5.17.0.

If you or anyone else here has concerns about any aspect of the above
general plans (but especially the timing of the Python2 deprecation
and/or removal), please let me know soon.

Alan


>
> On Tue, 6 Aug 2019 at 12:50, Phil Rosenberg <p.d...@gm...> wrote:
>>
>> Hi Alan
>> The error message I gave is the complete error message, there is no
>> output of the line variable. This, combined with the fact that the
>> carat is pointing to the end of the line of python code, made me think
>> this was a syntax error in the python code itself.
>>
>> When I opened convert_comment.py in visual studio, the error
>> highlighting underlined all instances of raise RuntimeError. Hovering
>> the mouse over displayed the following error
>>
>> invalid syntax, only exception value is allowed in 3.x
>>
>> Googling this error lead me to
>> https://stackoverflow.com/questions/34463087/valid-syntax-in-both-python-2-x-and-3-x-for-raising-exception
>>
>> So I think, basically the syntax for raise has changed between 2.x and
>> 3.x and for whatever reason, on my system the script is being executed
>> using 3.x
>>
>> Any fix suggestions? I don't really know python at all.
>>
>> Phil
>>
>> On Wed, 31 Jul 2019 at 19:49, Alan W. Irwin <Ala...@gm...> wrote:
>>>
>>> On 2019-07-31 12:44+0100 Phil Rosenberg wrote:
>>>
>>>> Thanks Alan
>>>> I've just tried again with the style_source script, but I'm hitting
>>>> another problem. I now get the error:
>>>>
>>>> File "scripts/convert_comment.py", line 72
>>>>    raise RuntimeError, "Cannot interpret trailing character(s) after
>>>> */ for this line"
>>>>                      ^
>>>> SyntaxError: invalid syntax
>>>> ERROR: scripts/convert_comment.py failed for file plplot_config.h.in
>>>>
>>>>
>>>> Any suggestions? I have both python 2 and 3 installed.
>>>
>>> That error message comes from this logic in scripts/convert_comment.py
>>> which hasn't been changed (from git blame -w) since 2010.
>>>
>>>      # Note trailing "\n" has not (yet) been removed from line so
>>>      # that the next to last character is at position len(line) - 3.
>>>      if end_comment >=0 and end_comment !=  len(line) - 3:
>>>          if ifsingleline and start_comment >=0:
>>>              # Skip most further processing for a line with embedded
>>>              # comments outside of multiline blocks of comments.
>>>              start_comment = -1
>>>              end_comment = -1
>>>          else:
>>>              sys.stderr.write(line)
>>>              raise RuntimeError, "Cannot interpret trailing character(s) after */ for this line"
>>>
>>> So that error message should have included the results from sys.stderr.write(line)
>>> from the line in plplot_config.h.in that is stored in the Python "line" variable
>>> that appears to be causing this python logic problem.
>>>
>>> The usual interpretation of this error message is you have commentary
>>> in plplot_config.h.in which is not in legitimate form.  For example,
>>> you might have forgotten the trailing "*/" on a comment.  So I would
>>> test that possibility by attempting to build the plplot target before
>>> styling, which would necessarily attempt to compile the configured
>>> plplot_config.h.
>>>
>>> However, if that "easy" answer is not the correct one, please send the
>>> complete error message including the output of the Python "line" variable
>>> that is causing the issue.
>>>
>>> Of course, the above Python logic only works if there are no
>>> line-ending issues in Python, i.e., the Python "line" variable
>>> contains a string that is terminated simply by \n rather than
>>> \r\n. And note that by git default plplot_config.h.in will have \r\n
>>> line endings on MSYS2.  But the discussion in
>>> <https://stackoverflow.com/questions/10785131/line-endings-in-python>
>>> seems to imply that on Python automatically converts all \r and \r\n
>>> line endings for text files to \n.  Also, my impression is you have
>>> exercised the above scripts/convert_comment.py logic from 2010 with no
>>> issues in the past on Cygwin (where again, the checked out
>>> plplot_config.h.in should have \r\n line endings.) So I would only
>>> look at this potential line ending issue (by dumping out each raw byte
>>> of the above line) only as a last resort (i.e., only if the line that
>>> is causing this error compiles with no issues).
>>>
>>> Alan
>>> __________________________
>>> Alan W. Irwin
>>>
>>> Programming affiliations with the FreeEOS equation-of-state
>>> implementation for stellar interiors (freeeos.sf.net); the Time
>>> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
>>> software package (plplot.org); the libLASi project
>>> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
>>> and the Linux Brochure Project (lbproject.sf.net).
>>> __________________________
>>>
>>> Linux-powered Science
>>> __________________________
>

__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.org); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________