Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-15 07:11-0000 Arjen Markus wrote:

> Hi Alan,
>
>
>
>> -----Original Message-----
>> From: Alan W. Irwin [mailto:ir...@be...]
>> Sent: Friday, August 11, 2017 6:33 PM
>>
>> Once you perform (1), (2'), and (3), I should have two report tarballs to look at as
>> well as one ~62MB tarball collection of plot file results, and assuming those reveal
>> no more issues (other than the "missing pyqtconfig.py" issue we will be dealing with
>> after the release), then these should be the last noninteractive tests you need to
>> make on this platform before the release.
>>
> This is turning out to be very frustrating - a sort of spring procession of Echternach.
>

> While my installation of MinGW-w64/MSYS2 is clean (I did not mess
with it other than via pacman), I thought it would be a good idea to
do a parallel installation anyway - as a test for my automated
installation procedure and as future help to others.

I agree this was a really good idea.  It's just too bad that
you ran into so many initial troubles with it (as you
discuss below), but I am sure you will eventually figure
out all of those issues that you encountered.

>
> So:
>

> - The automated installation went fairly smooth, but CMake did not
acknowledge the MSYS or Unix generators that my original installation
did have.

> - I had to resort to the MinGW generator, but then you have to move
"sh.exe" out of the way.

I can see why you attempted to try "MinGW Makefiles" when "MSYS2
Makefiles" was temporarily not working, but now the latter is working
again I suggest you stick with "MSYS2 Makefiles" until you complete
your tests to avoid the sh.exe complications implicit in using "MinGW
Makefiles".

However, after the release if you ever want to pursue "MinGW
Makefiles" again, I would suggest the following slightly different
approach that does not mess with any files installed by pacman and 
which worked very well for me for the MinGW/MSYS case.

* Copy everything in your pristine (i.e., without sh.exe removed) /usr/bin to some other location called say
/usr/bin_without_sh.

* Remove sh.exe from /usr/bin_without_sh.  (Since this
is a different location then your pristine /usr/bin, this change does
not clobber that pristine location and therefore does not affect pacman
results in any way.)

* Replace /usr/bin on your PATH with /usr/bin_without_sh

This PATH change in at least the MinGW/MSYS platform case lead to good
comprehensive test results for the "MinGW Makefiles" generator so I
think that will be generally true in the MinGW-w64/MSYS2 platform case
as well.  However, you may also run into find difficulties, e.g., for
cmake/modules/findwxWidgets.cmake.  Therefore, debugging such issues
will likely take some time to sort out which is one other reason why I
have suggested you stick with "MSYS Makefiles" for now.

> - I had some trouble getting "git" to be available via the
MinGW-w64/MSYS2 shell - I later realised that was my fault. So I
corrected that this morning by expanding the PATH correcrly. Only to
discover that the git installation then brings in its own "sh.exe".

That makes sense.  The git package is from the Unix tools (msys2)
repository of MinGW-w64/MSYS2.  So any installation such as that
likely updates everything from the Unix tools side and therefore likely
figures out there is a sh.exe file missing from that side and
fixes it.

> - Yesterday I got stuck because CMake was invoking the MicroSoft
version of make - despite the fact that "which make" in the
MinGW-w64/MSYS2 shell reports "/usr/bin/make".

The "which" command is sometimes unreliable.  For example, suppose
your PATH was "fullpath_one:fullpath_two", i.e., anything in
fullpath_one is higher on the PATH than anything in fullpath_two. Now,
suppose there is an executable (let's call it make.exe) located in
fullpath_two but not in fullpath_one, then "which make" will (rightly)
tell you the location is fullpath_two/make.exe.  Now suppose you
subsequently install a completely different make.exe in fullpath_one.
Then "which make" will still (in error) report fullpath_two/make.exe!
This occurs even though that new version of make is higher on your PATH and
will get executed when you execute "make" even though the fullpathname
reported by "which make" is different!

To avoid this "which" unreliability (especially when doing installs!) I
suggest you use the "hash" builtin command available from bash.exe
instead which is more complicated to use then "which" but completely
reliable.  The complication is hash has to be called twice (once to
update the hash table to fill it with the correct full pathname
corresponding to current PATH and specified <command> and once to
print out that full pathname corresponding to <command>).  So always
invoke it like this,

hash <command>; hash -t <command>

e.g.,

irwin@raven> hash make; hash -t make
/usr/bin/make

So here are my guesses to explain the above peculiar result you
describe for cmake.  At the time when you invoked cmake, you may not
have set the correct PATH to start with or your PATH might have been
wrong because you were not in the correct shell.  Alternatively your
"which" result could have been unreliable or else made with a
different PATH or shell setting compared to what you had when you invoked
cmake.

> - I realised, however, that I could instruct the comprehensive test
script to explicitly use "/usr/bin/make". So I added that option to
the invocation.

Yes, fullpaths are always reliable, but so is "hash make; hash -t make"

> - To my surprise CMake did acknowledge the "MSYS generator" this
morning (*), but because I had moved "sh.exe" out of the way,
"/usr/bin/make" refused to do its job, as it could not find the
"sh.exe" executable. So CMake failed. Sigh.

[out of order]
> (*) MinGW-w64 seems to require that you restart the shell after
installing things if you want to take full advantage of the freshly
installed package. That may be an explanation for yesterday's failure
with CMake. A small mistake in my install script required me to
install CMake manually.

> Sorry to sound so grumpy, but it was rather frustrating as I already
said.

My sympathies.  Tiny details that are slightly wrong can drive you crazy.

This is why I agreed with you above that an automated install
procedure is absolutely the best way to go.  Debugging such an
automated process forces most of the tiny details to eventually be
correct.  Furthermore, using automated procedures makes it much easier
to reproduce previous results.

> The whole thing is requiring more of my attention than I can
actually spare for the next few days (I have to revise an article and
that is mostly in my spare time).  I will try to get bits and pieces of
the tests done inbetween all other stuff, but I am not going to
promise anything as far as time of delivery is concerned.

Thanks for being honest about all the other critical calls on your
time right now. That and your current inability to reproduce what you
did before make it pretty likely it is going to be quite a long time
from now before you can finish your testing. And, of course, that
conflicts with my strong desire to get out the release in a timely
manner now that I have made my last planned change to the source code.

So given this situation and my additional strong desire for PLplot to
be a fun spare-time hobby activity for you (just as it is for me)
rather than some horrible slog, it appears there are two general
courses open to us.

1. Release soon.  It is unlikely you will have complete testing
results for your three platforms any time soon so this option
virtually guarantees that we will release with incomplete testing on
your part.

2. Wait indefinitely until you do have some relaxed time to
   complete your testing, then release.

Let's make this tough decision between these two general choices on
Friday when you will presumably have a much better idea about how your article
revision is going and therefore how much time you will have for
PLplot testing in the near future.  Meanwhile, a day or two off from
testing may clear your mind and make it possible to do it again in
a way that reproduces what you have done before.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________