plplot-devel — Technical discussion about PLplot development

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Tuesday, August 08, 2017 11:46 AM
>
> So if you use make -j8 (or so) and ctest -j8 for your comprehensive test it will really
> go ~4 times faster because those -j options will utilize your four cores completely.
> But you have found in the past that the -j option gives unreliable results for both
> make and ctest for both the Cygwin and MinGW-w64/MSYS2 platforms.  So you
> have dropped these -j options with the result that you only use one of your cores,
> but at least you do get slow but reliable results that way.
>
I explicitly use the build command "make" because of that, but I still see four instances of make running, as well as six instances of bash. This is with message "make VERBOSE=1 test_noninteractive in the installed examples build tree" as the last visible text.

Not at all sure what this means, the information I get from the task manager does not reveal much about what these processes are actually doing (like in which directory etc.)

Regards,

Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-08 08:44-0000 Arjen Markus wrote:

> Hi Alan,
>
>
>
>> -----Original Message-----
>> From: Alan W. Irwin [mailto:ir...@be...]
>> Sent: Tuesday, August 08, 2017 1:59 AM
>>> As the full run takes half a day
>>
>> [...]
>>
>> The fairly equivalent test here (although I test substantially more components of
>> PLplot because Linux has allowed me to install virtually all of the PLplot soft
>> dependencies) takes ~2 hours here on a decade-old (but still fairly high end) two-
>> cpu box with each of those cpu's running at 2.4GHz.  So I would predict it would
>> take roughly 4 hours there on your one-cpu box if that single cpu also runs at
>> roughly 2.4GHz.  So out of curiosity does your "half a day" correspond to half a
>> working day, i.e., 4 hours or do you mean something much longer than that?
>>

> As the laptop I use has four cores (8 hyperthreads), I would expect it to be faster, rather than slower. However, the wall clock time it takes may be determined by the starting and stopping of the many processes, not so much by the running of each. That is my impression at least, not based on any kind of measurement.

So if you use make -j8 (or so) and ctest -j8 for your comprehensive
test it will really go ~4 times faster because those -j options will
utilize your four cores completely.  But you have found in the past
that the -j option gives unreliable results for both make and ctest
for both the Cygwin and MinGW-w64/MSYS2 platforms.  So you have
dropped these -j options with the result that you only use one of your
cores, but at least you do get slow but reliable results that way.

It just struck me that the other possibility is using all four cores
really heats up your computer, and if you haven't had a recent dust
cleanout that excess heat could cause hardware glitches.  So that
might have been the source of the previous unreliability you had with
-j options. So if your system guy was willing to blow out the dust on
your computer, then you might want to try -j8 again (for both make and
ctest).

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,

> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Tuesday, August 08, 2017 11:26 AM
>
> > I have Qt4 now in the installation, but not pyqt4 due to the lack of
> "SIP" for this platform. The information on SIP for MinGW-w64 is a trifle convoluted.
> That is for later.
>
> It's great that you now have Qt4 installed.  Furthermore, the name of the pyqt4
> package you need to install is mingw-w64-x86_64-python3-sip and its dependencies.


Thanks for digging that up. I will install it later - as soon as the test has finished.

>
> You did not mention diffutils.  What is the installation status of that package?
>
I forgot to mention that - yes, I installed it. And it comes up with an interesting report on Lua:

Comparison test using psc device

...

lua

  Missing examples            :

  Differing graphical output  :  14a 19 23

  Missing stdout              :

  Differing stdout            :  31

These differences pop up in various configurations. I have not seen them in the Cygwin results, so this is something specific to MinGW-w64/MSYS2.

Regards,

Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-08 07:30-0000 Arjen Markus wrote:

> No, half a working day :).

Good.  That is a relief there is not some huge PLplot timing disparity
between the Linux and MinGW-w64/MSYS2 platforms.

[...]
>> That installation script should include a well-maintained list of packages that are
>> needed for PLplot testing (and general full use of PLplot on this platform by users).
>> That list of packages should include the ones you currently have installed plus "git"
>> and "diffutils" (where the lack of prefix on these package names means you will be
>> installing from the "msys" repository of MinGW-w64/MSYS2), and either "mingw-
>> w64-x86_64-qt4" (preferably if your package installation allows that without conflicts
>> since Qt4 is a more reliable library than Qt5 right now) or "mingw-w64-x86_64-qt5".
>
> I checked the list of packages for "git", but there is nothing obvious that will bring git into the mix. According to Google hits, you should simply install git for Windows.
>

> I have Qt4 now in the installation, but not pyqt4 due to the lack of
"SIP" for this platform. The information on SIP for MinGW-w64 is a
trifle convoluted. That is for later.

It's great that you now have Qt4 installed.  Furthermore, the name of
the pyqt4 package you need to install is mingw-w64-x86_64-python3-sip
and its dependencies.  At least in Debian's case, the python3-sip
package and its dependencies are all you need, and I assume that is
also true of the mingw-w64-x86_64-python3-sip package which I found in
the list of MinGW-w64/MSYS2 packages in the "mingw64" repository given
at <http://repo.msys2.org/mingw/x86_64/>.  So since I found that
package you should probably install it immediately rather than trying
to remember this e-mail later, and that means even your noninteractive
testing would include a build test of our pyqt4 component.

However, assuming that pyqt4 builds in your noninteractive testing, if
you want to do run-time interactive testing of it later this week
(after I have made my final pre-5.13.0 change to wxwidgets), I
can show you how to limit your interactive comprehensive testing
to just our wxwidgets and pyqt4 components (as opposed to
the pure wxwidgets interactive test you ran last time).  But
if you want to continue to limit that interactive comprehensive
test to just wxwidgets before 5.13.0 is released that would be
fine as well.

You did not mention diffutils.  What is the installation status of that
package?

Both the diffutils and git packages are in in the list of
MinGW-w64/MSYS2 packages in the "msys2" repository given at 
<http://repo.msys2.org/msys/x86_64/>.  This repository package list is
quite different from the "mingw64" package list I used above, but
these two repositories are both part of the MinGW-w64/MSYS2 platform.

If only git and mingw-w64-x86_64-python3-sip are still missing from
your install, and the test is already running that's OK, and I would
just let it finish.  But if diffutils is also missing from your
install, that is a pretty serious deficiency (since that means your
comprehensive test misses the most important part of the test).  So
for that case, I would stop any test that is running, install all of
diffutils, git, and mingw-w64-x86_64-python3-sip, and start the test
again from scratch.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Tuesday, August 08, 2017 1:59 AM
> > As the full run takes half a day
>
> [...]
>
> The fairly equivalent test here (although I test substantially more components of
> PLplot because Linux has allowed me to install virtually all of the PLplot soft
> dependencies) takes ~2 hours here on a decade-old (but still fairly high end) two-
> cpu box with each of those cpu's running at 2.4GHz.  So I would predict it would
> take roughly 4 hours there on your one-cpu box if that single cpu also runs at
> roughly 2.4GHz.  So out of curiosity does your "half a day" correspond to half a
> working day, i.e., 4 hours or do you mean something much longer than that?
>
As the laptop I use has four cores (8 hyperthreads), I would expect it to be faster, rather than slower. However, the wall clock time it takes may be determined by the starting and stopping of the many processes, not so much by the running of each. That is my impression at least, not based on any kind of measurement.

Regards,

Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



See below.



> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Tuesday, August 08, 2017 1:59 AM
> To: Arjen Markus
> Cc: PLplot development list
> Subject: RE: [Plplot-devel] Planning for the release of 5.13.0
>
> On 2017-08-07 11:05-0000 Arjen Markus wrote:
>
> > This certainly fixed the "Bad Address" problem - see the attached
> tarball.
>
> Hi Arjen:
>
> I did my usual checks of this report, and I am thrilled that error is gone AND you
> were able to finish the script this time without any further showstopping errors.
> However, this test result is still incomplete, and your platform needs to be updated
> to address those issues, see below.
>
> [...]
> > As the full run takes half a day
>
> [...]
>
> The fairly equivalent test here (although I test substantially more components of
> PLplot because Linux has allowed me to install virtually all of the PLplot soft
> dependencies) takes ~2 hours here on a decade-old (but still fairly high end) two-
> cpu box with each of those cpu's running at 2.4GHz.  So I would predict it would
> take roughly 4 hours there on your one-cpu box if that single cpu also runs at
> roughly 2.4GHz.  So out of curiosity does your "half a day" correspond to half a
> working day, i.e., 4 hours or do you mean something much longer than that?
>

No, half a working day :).

> [out of order]
> > I noticed that the MinGW-w64/MSYS2 installation that I used
> does not have the diff or cmp command, so a comparison is missing.
> [...] I will add diff later (and check the other components that are not accepted right
> now).
>
> I am really pleased that you are planning to do get rid of this (and other, see below)
> platform deficiencies by making this important update of your MinGW-w64/MSYS2
> platform before you do any more PLplot testing on this platform.
>
> There is one important caveat you should keep in mind about this platform update.
> My judgement from listening to complaints for a long time on the MSYS2 list is a
> given snapshot of this platform is not always in a self-consistent or even working
> state from one day to the next.  So most days seem to be OK, but sometimes not.
> So to make yourself fairly immune to that uncertainty I suggest you attempt no
> additional package installs of any kind for your present platform.
> Instead, you should install a completely new version of
> MinGW-w64/MSYS2 from scratch using a unique installation prefix so you can go
> back to your current working but somewhat limited version of this platform if the
> new snapshot of MinGW-w64/MSYS2 does not work.
> Of course that new install should be automated as much as possible with a script to
> make it trivially easy for you to do further installs of MinGW-w64/MSYS2 from
> scratch as needed, i.e., one or two days later if that new version does not work for
> today's snapshot.
>
> That installation script should include a well-maintained list of packages that are
> needed for PLplot testing (and general full use of PLplot on this platform by users).
> That list of packages should include the ones you currently have installed plus "git"
> and "diffutils" (where the lack of prefix on these package names means you will be
> installing from the "msys" repository of MinGW-w64/MSYS2), and either "mingw-
> w64-x86_64-qt4" (preferably if your package installation allows that without conflicts
> since Qt4 is a more reliable library than Qt5 right now) or "mingw-w64-x86_64-qt5".

I checked the list of packages for "git", but there is nothing obvious that will bring git into the mix. According to Google hits, you should simply install git for Windows.

I have Qt4 now in the installation, but not pyqt4 due to the lack of "SIP" for this platform. The information on SIP for MinGW-w64 is a trifle convoluted. That is for later.
>
> All three package installs are important. The git package needs to be installed so
> that the comprehensive test script can figure out what git commit you are testing for
> PLplot.  The diffutils package needs to be installed to supply the all-important
> diff.exe executable (as you independently discovered above).  And qt4 (or qt5)
> needs to be installed so you can test on this platform our qt device driver which is
> potentially a source of many high-quality noninteractive devices on this platform (as
> well as one important high-quality interactive device, qtwidget, that you might wish
> to test post-release as a basis of comparison with other interactive devices on this
> platform).
>
> So I am very much looking forward to your noninteractive test report for your latest
> snapshot of MinGW-w64/MSYS2 with git.exe, diff.exe, and some version of the Qt
> suite of libraries all installed because that will finally make the level of your testing
> on this platform consistent with Greg Jung's level of testing two years ago.
> Furthermore, because Greg reported successful testing with both diff and Qt, I am
> fairly confident that the addition of diff and Qt by you will reveal no new issues.
> However, we will see about that since both
> MinGW-w64/MSYS2 and PLplot have changed a lot in the last couple of years.
>
Right. Running it today.

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-07 11:05-0000 Arjen Markus wrote:

> This certainly fixed the "Bad Address" problem - see the attached
tarball.

Hi Arjen:

I did my usual checks of this report, and I am thrilled that error
is gone AND you were able to finish the script this time without any
further showstopping errors.
However, this test result is still incomplete,
and your platform needs to be updated to address
those issues, see below.

[...]
> As the full run takes half a day

[...]

The fairly equivalent test here (although I test substantially more
components of PLplot because Linux has allowed me to install virtually
all of the PLplot soft dependencies) takes ~2 hours here on a
decade-old (but still fairly high end) two-cpu box with each of those
cpu's running at 2.4GHz.  So I would predict it would take roughly 4
hours there on your one-cpu box if that single cpu also runs at
roughly 2.4GHz.  So out of curiosity does your "half a day" correspond
to half a working day, i.e., 4 hours or do you mean something much
longer than that?

[out of order]
> I noticed that the MinGW-w64/MSYS2 installation that I used
does not have the diff or cmp command, so a comparison is missing.
[...] I will add diff later (and check the other components that
are not accepted right now).

I am really pleased that you are planning to do get rid of this (and
other, see below) platform deficiencies by making this important
update of your MinGW-w64/MSYS2 platform before you do any more PLplot
testing on this platform.

There is one important caveat you should keep in mind about this
platform update. My judgement from listening to complaints for a long
time on the MSYS2 list is a given snapshot of this platform is not
always in a self-consistent or even working state from one day to the
next.  So most days seem to be OK, but sometimes not.  So to make
yourself fairly immune to that uncertainty I suggest you attempt no
additional package installs of any kind for your present platform.
Instead, you should install a completely new version of
MinGW-w64/MSYS2 from scratch using a unique installation prefix so you
can go back to your current working but somewhat limited version of
this platform if the new snapshot of MinGW-w64/MSYS2 does not work.
Of course that new install should be automated as much as possible
with a script to make it trivially easy for you to do further installs
of MinGW-w64/MSYS2 from scratch as needed, i.e., one or two days later
if that new version does not work for today's snapshot.

That installation script should include a well-maintained
list of packages that are needed for PLplot testing (and general full
use of PLplot on this platform by users).  That list of packages
should include the ones you currently have installed plus "git" and
"diffutils" (where the lack of prefix on these package names means you
will be installing from the "msys" repository of MinGW-w64/MSYS2), and
either "mingw-w64-x86_64-qt4" (preferably if your package installation
allows that without conflicts since Qt4 is a more reliable library
than Qt5 right now) or "mingw-w64-x86_64-qt5".

All three package installs are important. The git package needs to be
installed so that the comprehensive test script can figure out what
git commit you are testing for PLplot.  The diffutils package needs to
be installed to supply the all-important diff.exe executable (as you
independently discovered above).  And qt4 (or qt5) needs to be
installed so you can test on this platform our qt device driver which
is potentially a source of many high-quality noninteractive devices on
this platform (as well as one important high-quality interactive
device, qtwidget, that you might wish to test post-release
as a basis of comparison with other interactive devices on this
platform).

So I am very much looking forward to your noninteractive test report
for your latest snapshot of MinGW-w64/MSYS2 with git.exe, diff.exe,
and some version of the Qt suite of libraries all installed because
that will finally make the level of your testing on this platform
consistent with Greg Jung's level of testing two years ago.
Furthermore, because Greg reported successful testing with both diff
and Qt, I am fairly confident that the addition of diff and Qt by you
will reveal no new issues. However, we will see about that since both
MinGW-w64/MSYS2 and PLplot have changed a lot in the last couple of
years.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



This certainly fixed the "Bad Address" problem - see the attached tarball. I noticed that the MinGW-w64/MSYS2 installation that I used does not have the diff or cmp command, so a comparison is missing. As the full run takes half a day, I will proceed now with the Cygwin version to get a quick impression of your most recent fix for both platforms. I will add diff later (and check the other components that are not accepted right now).

Regards,

Arjen

> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Monday, August 07, 2017 2:51 AM
> To: Arjen Markus
> Cc: PLplot development list
> Subject: Re: [Plplot-devel] Planning for the release of 5.13.0
>
> On 2017-08-04 00:51-0700 Alan W. Irwin wrote:
>
> > The current status is I am almost done with the implementation of that
> > fix [for the bad address issue], but I need to get some sleep before I
> > finish, test, and commit it (likely by late Friday my time or early Saturday your
> time).
>
> OK.  Finally (commit a5a0d25) got that fix done.
>


DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-04 00:51-0700 Alan W. Irwin wrote:

> The current status is I am almost done with the implementation of that
> fix [for the bad address issue], but I need to get some sleep before I finish, test, and commit it
> (likely by late Friday my time or early Saturday your time).

OK.  Finally (commit a5a0d25) got that fix done.

I just now realized (sigh) that commit message needed the following
change in the "Tested by" section:

scripts/comprehensive_test.sh --prefix "../comprehensive_test_disposeable blank" --do_submit_dashboard yes --do_test_interactive no --do_test_noninteractive no --do_test_install_tree no --do_test_traditional_install_tree no

should be replaced by 
scripts/comprehensive_test.sh --prefix "../comprehensive_test_disposeable blank" --do_submit_dashboard yes --do_test_interactive no

That is, the only constraint on the breadth of the comprehensive test
that actually occurred was the interactive part of comprehensive
testing was dropped.  So this should be very similar to your own
further non-interactive testing.

I am looking forward to your report for MinGW-w64/MSYS2 confirming
this issue has been fixed by the above change (and hopefully revealing
there are no further noninteractive issues on that platform).

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-03 11:56-0700 Alan W. Irwin wrote:

> But assuming (now that both PATH and PKG_CONFIG_PATH are set properly
> and you are using an unambiguous make command of the correct form for
> MinGW-w64/MSYS2) you can replicate by hand this way, can you replicate
> in even a simpler way by running the exact same gcc command that the
> make command does?
>

Hi Arjen:

I saw your further note concerning working on this replication
question over the weekend.  However, there has been a new development
(see below) that will likely lead to the solution of the "Bad address"
issue without you needing to figure out how to replicate the issue by
hand.  So pursuing this replication question further is up to you.  On
the one hand, it will be educational for you, but on the other hand,
it is likely not necessary to solve the original "Bad address" issue.

Here is that new development.  I did a further google search for the
Bad address issue and found (see
<https://sourceforge.net/p/msys2/mailman/message/34476535/>) that Greg
Jung had already covered this ground.  Furthermore, following a
suggestion from the MSYS2 mailing list he discovered the fix which was
to drop the rpath option from the gcc command!

So that is great, but there were no actual details about how he
implemented that, and for unknown reasons a definitive version of that
fix never got into our source tree.  So further analysis follows.

It is clear from

shared/noninteractive/output_tree/traditional_make_noninteractive.out

in your report that gcc does at least accept the options

-Wl,-rpath
-Wl,"D:/plplot-svn/comprehensive_test_disposeable/shared/noninteractive/install_tree/lib"

but the corresponding options from

nondynamic/noninteractive/output_tree/traditional_make_noninteractive.out

for the non-working nondynamic case are

-Wl,-rpath
-Wl,"D:/plplot-svn/comprehensive_test_disposeable/nondynamic/noninteractive/install_tree/lib:/mingw64/lib"

That is, it is extremely likely that last colon delimiter is causing
the "gcc.exe: Bad address" issue.  I can think of many ways to fix
that, but I don't think we should bother, and instead we should simply
follow what Greg did above and what CMake does for our CMake-based
build of the installed examples which is there is no mention of rpath
for all Windows platforms. (You can verify that last statement for
yourself by searching for "rpath" in all the *.out files other than
the "traditional_make_noninteractive.out" files in your recent Cygwin
and MinGW-w64/MSYS2 reports.)

The current status is I am almost done with the implementation of that
fix, but I need to get some sleep before I finish, test, and commit it
(likely by late Friday my time or early Saturday your time).  So
hopefully this fix will give you a chance to complete (including the
diff and Qt install updates) the noninteractive comprehensive testing
of MinGW-w64/MSYS2 this weekend.  And that would be a big step toward
our mutual goal of releasing a well-tested 5.13.0 soon.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



Thanks for this analysis and the suggested steps for further investigation of this puzzle. I hope to have time this weekend to investigate this.



Regards,



Arjen



> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Thursday, August 03, 2017 8:56 PM
> To: Arjen Markus
> Cc: PLplot development list
> Subject: RE: [Plplot-devel] Planning for the release of 5.13.0
>
> On 2017-08-02 19:36-0000 Arjen Markus wrote:
>
> >> -----Original Message-----
> >> From: Alan W. Irwin [mailto:ir...@be...]
> >> Sent: Tuesday, August 01, 2017 9:57 AM
> >>
> > ...
> >> 1.  If you get good results in the Cygwin case, but not the
> >> MinGW-w64/MSYS2 case, then that is pretty good evidence of a bug in
> >> the
> >> MinGW-w64/MSYS2 "make" package.
> >>
> >> 2. If both don't work, then that means long command lines work in
> >> neither, and perhaps we have to write off this PLplot issue for the
> >> MinGW-w64/MSYS2 case to longer path names/longer command lines in
> >> that case compared to the Cygwin case.  But I consider this result to
> >> be unlikely because according to
> >> <https://stackoverflow.com/questions/19354870/bash-command-line-
> >> and-input-limit>
> >> the Cygwin bash line limit is 32000 bytes, i.e., well above the 2001
> >> bytes generated above.
> >>
> >> 3. But if both work, then you will have to dig deeper.
> >>
> >>
> > There is but a single bash.exe on the system and the makefile you sent works
> perfectly on both Cygwin and MinGW.
>
> Hi Arjen:
>
> Actually, you must have at least two different bash.exe executables installed on
> your computer, one from Cygwin and one from MinGW-w64/MSYS2. So I think you
> meant the following, but just to be absolutely clear, can you double check that only
> the Cygwin-installed version version of bash.exe was on your PATH for the Cygwin
> experiment and only the MinGW-w64/MSYS2-installed version of bash.exe was on
> your PATH for the MinGW-w64/MSYS2 experiment?  Each are likely installed in the
> "/usr/bin/bash.exe" location on both platforms, but those are two very different
> places so, e.g., "ls -l /usr/bin/bash.exe" will show different dates and lengths for
> those two different files when run from Cygwin and MinGW-w64/MSYS2.  Of
> course, the best thing to do in each case for your tests is to run bash.exe using its
> full path name starting with a drive letter which is unambigous.  And as I said before,
> you should be doing the same for make.exe (and cmake.exe and
> ctest.exe) when your invocation script runs the comprehensive test script.
>
> > Your example did inspire me to try "gcc -o x x.c
> -DLONG=01234.....89" instead of "echo".
>
> > That, however, works perfectly
> as well. There must be something much more specific that is causing this. For the
> moment I have run out of easy-to-test ideas though.
>
> Such experiments as the one you tried above (and the further one you tried in a
> later post) are extremely useful.
>
> So assuming you really are using two different versions of bash for this test, then
> you have confirmed there are no line-length issues for make for either Cygwin or
> MinGW-w64/MSYS2 which is a big step forward.
>
> Stepping back a bit from these details, it seems to me the fundamental issue here
> is you have a comprehensive test failure that you have not yet been able to
> reproduce by hand.  So let's go back and see exactly why you cannot (so far)
> replicate by hand. Furthermore, to take advantage of the results accumulated on
> disk by your last comprehensive test with -DNON_TRANSITIVE=ON, let's stick
> with those.
> (Those results were identical to your previous ones without that option other than
> linking commands were much shorter.)
>
> The key here is to attempt to reproduce the bad script result by following all the
> essential steps that script does.  For example, from the report you sent of the -
> DNON_TRANSITIVE=ON case just prior to the error there was this messaged
> emitted by the script:
>
> 'Prepend "/d/plplot-
> svn/comprehensive_test_disposeable/nondynamic/noninteractive/install_tree/bin" to
> the original PATH
>
> That's really important since it sets the exact (nondynamic) versions of the PLplot
> libraries that are being used for this part of the test you are trying to replicate.
>
> That original PATH (according to that report) is
>
> PATH="/d/cmake3.4.3/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/Syst
> em32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPo
> werShell/v1.0/"
>
> So for this attempt at hand replication, do make sure to set PATH as
> follows:
>
> export \
> PATH="/d/plplot-
> svn/comprehensive_test_disposeable/nondynamic/noninteractive/install_tree/bin:/d/
> cmake3.4.3/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Wi
> ndows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1
> .0/"
>
> Also, (again following the report) you should set the environment variable
>
> export \
> PKG_CONFIG_PATH="/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig"
>
> Then (still following what the script does)
>
> cd \
> /d/plplot-
> svn/comprehensive_test_disposeable/nondynamic/noninteractive/install_tree/share/
> plplot5.12.0/examples/c
>
> make x00c.exe
>
> Does that make command (or much better yet its fullpath equivalent that starts with
> a drive letter) replicate the "Bad address" error?
>
> If not, then it appears the most likely explanation is I have forgotten something
> essential that the script does, and I will have to dig deeper.
>
> But assuming (now that both PATH and PKG_CONFIG_PATH are set properly and
> you are using an unambiguous make command of the correct form for
> MinGW-w64/MSYS2) you can replicate by hand this way, can you replicate in even
> a simpler way by running the exact same gcc command that the make command
> does?
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state implementation for
> stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net);
> PLplot scientific plotting software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux
> Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-02 19:36-0000 Arjen Markus wrote:

>> -----Original Message-----
>> From: Alan W. Irwin [mailto:ir...@be...]
>> Sent: Tuesday, August 01, 2017 9:57 AM
>>
> ...
>> 1.  If you get good results in the Cygwin case, but not the
>> MinGW-w64/MSYS2 case, then that is pretty good evidence of a bug in the
>> MinGW-w64/MSYS2 "make" package.
>>
>> 2. If both don't work, then that means long command lines work in neither, and
>> perhaps we have to write off this PLplot issue for the
>> MinGW-w64/MSYS2 case to longer path names/longer command lines in that case
>> compared to the Cygwin case.  But I consider this result to be unlikely because
>> according to <https://stackoverflow.com/questions/19354870/bash-command-line-
>> and-input-limit>
>> the Cygwin bash line limit is 32000 bytes, i.e., well above the 2001 bytes generated
>> above.
>>
>> 3. But if both work, then you will have to dig deeper.
>>
>>
> There is but a single bash.exe on the system and the makefile you sent works perfectly on both Cygwin and MinGW.

Hi Arjen:

Actually, you must have at least two different bash.exe executables
installed on your computer, one from Cygwin and one from
MinGW-w64/MSYS2. So I think you meant the following, but just to be
absolutely clear, can you double check that only the Cygwin-installed
version version of bash.exe was on your PATH for the Cygwin experiment
and only the MinGW-w64/MSYS2-installed version of bash.exe was on your
PATH for the MinGW-w64/MSYS2 experiment?  Each are likely installed in
the "/usr/bin/bash.exe" location on both platforms, but those are two
very different places so, e.g., "ls -l /usr/bin/bash.exe" will show
different dates and lengths for those two different files when run
from Cygwin and MinGW-w64/MSYS2.  Of course, the best thing to do in
each case for your tests is to run bash.exe using its full path name
starting with a drive letter which is unambigous.  And as I said
before, you should be doing the same for make.exe (and cmake.exe and
ctest.exe) when your invocation script runs the comprehensive test
script.

> Your example did inspire me to try "gcc -o x x.c
-DLONG=01234.....89" instead of "echo".

> That, however, works perfectly
as well. There must be something much more specific that is causing
this. For the moment I have run out of easy-to-test ideas though.

Such experiments as the one you tried above (and the further one
you tried in a later post) are extremely useful.

So assuming you really are using two different versions
of bash for this test, then you have confirmed there are no line-length
issues for make for either Cygwin or MinGW-w64/MSYS2 which is a big
step forward.

Stepping back a bit from these details, it seems to me the fundamental
issue here is you have a comprehensive test failure that you have not
yet been able to reproduce by hand.  So let's go back and see exactly
why you cannot (so far) replicate by hand. Furthermore, to take
advantage of the results accumulated on disk by your last
comprehensive test with -DNON_TRANSITIVE=ON, let's stick with those.
(Those results were identical to your previous ones without that
option other than linking commands were much shorter.)

The key here is to attempt to reproduce the bad script result by
following all the essential steps that script does.  For example, from
the report you sent of the -DNON_TRANSITIVE=ON case just prior to the
error there was this messaged emitted by the script:

'Prepend "/d/plplot-svn/comprehensive_test_disposeable/nondynamic/noninteractive/install_tree/bin" to the original PATH

That's really important since it sets the exact (nondynamic) versions of the PLplot
libraries that are being used for this part of the test you are trying
to replicate.

That original PATH (according to that report) is

PATH="/d/cmake3.4.3/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/"

So for this attempt at hand replication, do make sure to set PATH as
follows:

export \
PATH="/d/plplot-svn/comprehensive_test_disposeable/nondynamic/noninteractive/install_tree/bin:/d/cmake3.4.3/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/"

Also, (again following the report) you should set the environment
variable

export \
PKG_CONFIG_PATH="/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig"

Then (still following what the script does)

cd \
/d/plplot-svn/comprehensive_test_disposeable/nondynamic/noninteractive/install_tree/share/plplot5.12.0/examples/c

make x00c.exe

Does that make command (or much better yet its fullpath equivalent
that starts with a drive letter) replicate the "Bad address" error?

If not, then it appears the most likely explanation is I have
forgotten something essential that the script does, and I will have to
dig deeper.

But assuming (now that both PATH and PKG_CONFIG_PATH are set properly
and you are using an unambiguous make command of the correct form for
MinGW-w64/MSYS2) you can replicate by hand this way, can you replicate
in even a simpler way by running the exact same gcc command that the
make command does?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



I thought that perhaps it was due to the number of arguments, but even that does not seem to be the case:



all:

            gcc -o x x.c -DX0-0 -DX1=1 ... -DX199=199



will produce an executable x.exe without any problems.



Regards,



Arjen

From: Arjen Markus [mailto:Arj...@de...]
Sent: Wednesday, August 02, 2017 9:37 PM
To: Alan W. Irwin
Cc: PLplot development list
Subject: Re: [Plplot-devel] Planning for the release of 5.13.0


Hi Alan,



See below - we are no closer to a solution, sigh.



Regards,



Arjen


Your example did inspire me to try "gcc -o x x.c -DLONG=01234.....89" instead of "echo". That, however, works perfectly as well. There must be something much more specific that is causing this. For the moment I have run out of easy-to-test ideas though.

Regards,

Arjen
DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



See below - we are no closer to a solution, sigh.



Regards,



Arjen

> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Tuesday, August 01, 2017 9:57 AM
>
...
> 1.  If you get good results in the Cygwin case, but not the
> MinGW-w64/MSYS2 case, then that is pretty good evidence of a bug in the
> MinGW-w64/MSYS2 "make" package.
>
> 2. If both don't work, then that means long command lines work in neither, and
> perhaps we have to write off this PLplot issue for the
> MinGW-w64/MSYS2 case to longer path names/longer command lines in that case
> compared to the Cygwin case.  But I consider this result to be unlikely because
> according to <https://stackoverflow.com/questions/19354870/bash-command-line-
> and-input-limit>
> the Cygwin bash line limit is 32000 bytes, i.e., well above the 2001 bytes generated
> above.
>
> 3. But if both work, then you will have to dig deeper.
>
>
There is but a single bash.exe on the system and the makefile you sent works perfectly on both Cygwin and MinGW.

Your example did inspire me to try "gcc -o x x.c -DLONG=01234.....89" instead of "echo". That, however, works perfectly as well. There must be something much more specific that is causing this. For the moment I have run out of easy-to-test ideas though.

Regards,

Arjen


DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-08-02 07:07-0000 Arjen Markus wrote:

[...]

>> -----Original Message-----
>> From: Alan W. Irwin [mailto:ir...@be...]
>> Sent: Tuesday, August 01, 2017 9:57 AM
> ...

> Using that option [-DNON_TRANSITIVE=ON], the final command is
shorter, but still 1200+ characters. And, more importantly, the result
is the same: a Bad Address.

The -DNON_TRANSITIVE=ON option removes lots of the -L and -l options,
but still leaves lots of -I options (which we treat in a transitive
way only) so we are still stuck with that pretty long (1200+ bytes)
line length.

It's an extremely interesting result, however, that
-DNON_TRANSITIVE=ON appears to work (got just as far into the test
before finding this Bad Address error) as well as
-DNON_TRANSITIVE=OFF.  So once you get completely through the comprehensive
script without issues on MinGW-w64/MSYS2 with -DNON_TRANSITIVE=ON, I
will make -DNON_TRANSITIVE=ON the default on that platform, and the
same with Cygwin.

By the way, when carefully checking the above results, it appears you
do not have either git.exe or diff.exe installed on MinGW-w64/MSYS2.
The git.exe application is needed to help the comprehensive test
script figure out what PLplot commit you are testing (and might also
be generally useful to you).  The diff.exe application is of
more fundamental concern since it is what is used to determine
the important PostScript difference report.

I am not completely sure whether you should install the
"mingw-w64-x86_64" version or the "msys" version of these packages,
but since they are used primarily in a bash environment, I suggest you
try installing the msys versions first to see how they work.  In that
case, the names of the relevant packages are "git" and "diffutils".  I
also mentioned before you were missing the Qt package.  I would try
installing the mingw-w64-x86_64 version of qt4 first since Qt4 is a
higher quality library than Qt5 right now, but if that leads to
conflicts, try the mingw-w64-x86_64 version of qt5 instead.

>> Meanwhile, assuming the fundamental issue is command-line length, I just checked,
>> and all Makefile*.in files in our source tree contain the following line:
>>
>> SHELL = @SH_EXECUTABLE@
>>
>> and for your MinGW-w64/MSYS2 case
>>
>> # Check for all POSIX tool commands that have been found from the MSYS2 side
>> irwin@raven> find . -name CMakeCache.txt |xargs grep 'usr/bin'
>> ./shared/noninteractive/build_tree/CMakeCache.txt:CMAKE_MAKE_PROGRAM:FI
>> LEPATH=D:/mingw64/usr/bin/make.exe
> ...
>>
>> But I did notice that one possible issue is you simply used "make" for your
>> comprehensive test.
>>
>> What does "which make" give you for that platform?  Does it refer to that above
>> D:/mingw64/usr/bin/make.exe POSIX version or something else?
>>
> 'which make' gives the answer "/usr/bin/make", but that it is the very same file as
> "D:/mingw64/usr/bin/make.exe". I further checked: there is only a single make.exe in the installation.

OK

>
> ...
>>
>> But assuming that is not the issue, my working hypothesis is this command-line
>> length limitation is a MinGW-w64/MSYS2 bug since it does not appear to be an
>> issue for Cygwin's version of the POSIX make command.
>>
>> To investigate that hypothesis further, please take a look at the extremely simple
>> attached Makefile test case I have created.
>>
> I did not have time yet to perform this test. Hopefully today sometime.

I look forward to the results from those simple tests which should be
quite informative.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



See below.



Regards,



Arjen

> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Tuesday, August 01, 2017 9:57 AM
...
>
> One slightly problematic issue I noted for this platform was you put
> /d/cmake3.4.3/bin first on your PATH.  It appears that your script invocation used
>
> --cmake_command "/mingw64/bin/cmake"
>
> and your resulting CMakeCache.txt files found that same correct version, but
> nevertheless, to remove this source of uncertainty I would highly recommend
> removing this path (/d/cmake3.4.3/bin) to an incorrect (and severely dated) version
> of CMake from the PATH in your launch script for this platform.
>
I checked that: I indeed set it explicitly in the script to run the tests. I will remove it, but that version of Cmake would not be accepted anyway - PLplot after all forces a minimum version 3.6.2.

>
> Thanks for that extremely useful extra bit of research.  So it appears bash is not
> complaining about those 1800+ characters, but make is.
>
>
> So indeed, the non-working nondynamic case has a much longer command line
> (than the working shared case (because in the nondynamic case, the driver object
> code is in the shared library so _all_ driver library dependencies have to be
> mentioned (for the default **transitive** linking that occurs for the MinGW-
> w64/MSYS2 platform).
>
Using that option, the final command is shorter, but still 1200+ characters. And, more importantly, the result is the same: a Bad Address.

> So for this case as well, you might want to try -DNON_TRANSITIVE=ON.
> ...
>
> Meanwhile, assuming the fundamental issue is command-line length, I just checked,
> and all Makefile*.in files in our source tree contain the following line:
>
> SHELL = @SH_EXECUTABLE@
>
> and for your MinGW-w64/MSYS2 case
>
> # Check for all POSIX tool commands that have been found from the MSYS2 side
> irwin@raven> find . -name CMakeCache.txt |xargs grep 'usr/bin'
> ./shared/noninteractive/build_tree/CMakeCache.txt:CMAKE_MAKE_PROGRAM:FI
> LEPATH=D:/mingw64/usr/bin/make.exe
...
>
> But I did notice that one possible issue is you simply used "make" for your
> comprehensive test.
>
> What does "which make" give you for that platform?  Does it refer to that above
> D:/mingw64/usr/bin/make.exe POSIX version or something else?
>
'which make' gives the answer "/usr/bin/make", but that it is the very same file as
"D:/mingw64/usr/bin/make.exe". I further checked: there is only a single make.exe in the installation.

...
>
> But assuming that is not the issue, my working hypothesis is this command-line
> length limitation is a MinGW-w64/MSYS2 bug since it does not appear to be an
> issue for Cygwin's version of the POSIX make command.
>
> To investigate that hypothesis further, please take a look at the extremely simple
> attached Makefile test case I have created.
>
I did not have time yet to perform this test. Hopefully today sometime.

Attached is the result with -DNON_TRANSITIVE=ON
Regards,

Arjen
DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0 --Cygwin Ada issue

[Alan W. I. <ir...@be...>]

>> -----Original Message-----
>> From: Alan W. Irwin [mailto:ir...@be...]
>> Sent: Tuesday, August 01, 2017 5:02 AM
>> To: Arjen Markus
>> Cc: PLplot development list
>> Subject: Planning for the release of 5.13.0 --Cygwin Ada issue
>>
>> On 2017-07-31 07:13-0000 Arjen Markus wrote:
>>
>>> Here are some results from my most recent, i.e. yesterday's, testing session.
>>
>> Hi Arjen:
[...]
>> It turns out this is a long-standing issue, see your post from more than a year ago
>>
>> 2016-03-17 Arjen Markus    (21K) RE: comprehensive testing results on Cygwin -
>> Ada build errors
>>
>
> Forgot all about that. The script I use for cdash reports takes care of the extra information needed for Ada, so I thought it worked fine.
>
>> However, I think I have a lot better understanding of the issue now, then I did back
>> then.
>>
> ...
>
>>
>> So to test that possibility, please use the -DNON_TRANSITIVE=ON cmake option
>> for this test to see if (a) you get further with Ada, and (b) if that option causes
>> issues for other parts of our comprehensive tests for the Cygwin platform.  And
>> depending on what combination of (a) and (b) you obtain, I will take it from there.
>>

> Very much the same thing: link errors concerning doubly defined
symbols.

Thanks for that tarball report.  I double-checked with it
that the offending command line was very much shortened by the
above option, i.e., it is now

/usr/bin/gnatmake.exe  -Wl,--enable-auto-import "-aI/cygdrive/d/plplot-svn/plplot-git/examples/ada" "-aI/cygdrive/d/plplot-svn/plplot-git/bindings/ada" "-aL/cygdrive/d/plplot-svn/comprehensive_test_disposeable/shared/noninteractive/build_tree/bindings/ada/CMakeFiles/plplotada.dir" xtraditional13a.adb -cargs  -largs ../../dll/libplplotada.dll.a ../../dll/libplplot.dll.a

with only the two directly needed libraries mentioned by the -cargs
-largs combination of options.

So the implementation of this idea was perfect, but (as you said) the
(bad) result is very much the same, sigh.

So I am going to make the official decision now to not investigate
this issue again until after the release.  Which is good because
it simplifies our pre-release lives considerably. :-)

So document your launch script for Cygwin with a datestamped comment
about why you are disabling Ada, and you are done for now
with this Ada issue on Cygwin.

However, after the release you should build and comprehensively test a
good simple test case (i.e., see cmake/test_ada/README for
instructions about how to do that) on Cygwin, and then I would plan to
use your results for that as the basis of a bug report to Cygwin to
see if the Ada packager for that platform can figure out why your
build of that simple test case is (presumably) going wrong.

I have written up this testing idea in my post-release notes so there
should be no need to remind me about this idea then.  :-)

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-07-31 07:13-0000 Arjen Markus wrote:

> Here are some results from my most recent, i.e. yesterday's, testing session.

Hi Arjen:

Here is my promised detailed response to your recent noninteractive
comprehensive test report for MinGW-w64/MSYS2.


>> II. The noninteractive comprehensive test for MinGW-w64/MSYS2
[...]

One slightly problematic issue I noted for this platform
was you put
/d/cmake3.4.3/bin first on your PATH.  It appears
that your script invocation used

--cmake_command "/mingw64/bin/cmake"

and your resulting CMakeCache.txt files found that same correct
version, but nevertheless, to remove this source of uncertainty I
would highly recommend removing this path (/d/cmake3.4.3/bin) to an
incorrect (and severely dated) version of CMake from the PATH in your
launch script for this platform.

>
>>
>> * ERROR: Traditional make test_noninteractive failed in the installed examples tree
>> (for the nondynamic case)
>>
>> A google search for the term <"gcc.exe: Bad address"> didn't appear to find
>> anything relevant.  (There was one issue in Cygwin for 2012 involving commands
>> separated with a semicolon, but we don't use a semicolon in the present case.)
>>
> I tried the following:
>

> - Run the build command explicitly (it is 1800+ characters long!) -
this succeeded without any complaint and the resulting program did its
job.

>
> - Run make so that the program would be built. That failed with the "Bad address" message.
>

> My current conclusion is that the length of the command is causing a
problem in make. It may just go over some internal edge. There is not
much PLplot-ish about this, except for the length of the directory
names, it would seem.

Thanks for that extremely useful extra bit of research.  So it appears
bash is not complaining about those 1800+ characters, but
make is.

Here is some more research on the issue:

irwin@raven> grep x00c.exe nondynamic/noninteractive/output_tree/traditional_make_noninteractive.out | grep -v 'Error 126' | wc
       1      78    1882

irwin@raven> grep x00c.exe shared/noninteractive/output_tree/traditional_make_noninteractive.out | wc
       1      32     682

That last grep stanza on that first command was to eliminate the error
message from the count.

So indeed, the non-working nondynamic case has a much longer command
line (than the working shared case (because in the nondynamic case, the
driver object code is in the shared library so _all_ driver library
dependencies have to be mentioned (for the default **transitive** linking
that occurs for the MinGW-w64/MSYS2 platform).

So for this case as well, you might want to try -DNON_TRANSITIVE=ON.
That might work around this issue (there will be a much shorter
command line, if the command-line length really is the issue here),
but I have no clue whether non-transitive linking is as well-supported
by the GNU build tools on MinGW-w64/MSYS2 (and Cygwin for that matter)
as it is on Linux so I wait for your reports concerning
-DNON_TRANSITIVE=ON for both the MinGW-w64/MSYS2 and Cygwin
platforms with a great deal of interest.

Meanwhile, assuming the fundamental issue is command-line length,
I just checked, and all Makefile*.in files in our source tree
contain the following line:

SHELL = @SH_EXECUTABLE@

and for your MinGW-w64/MSYS2 case

# Check for all POSIX tool commands that have been found from the MSYS2 side
irwin@raven> find . -name CMakeCache.txt |xargs grep 'usr/bin'
./shared/noninteractive/build_tree/CMakeCache.txt:CMAKE_MAKE_PROGRAM:FILEPATH=D:/mingw64/usr/bin/make.exe
./shared/noninteractive/build_tree/CMakeCache.txt:ENV_FOR_ONSGMLS:FILEPATH=D:/mingw64/usr/bin/env.exe
./shared/noninteractive/build_tree/CMakeCache.txt:SED_EXECUTABLE:FILEPATH=D:/mingw64/usr/bin/sed.exe
./shared/noninteractive/build_tree/CMakeCache.txt:SH_EXECUTABLE:FILEPATH=D:/mingw64/usr/bin/bash.exe
./shared/noninteractive/build_tree/CMakeCache.txt:TAIL_EXECUTABLE:FILEPATH=D:/mingw64/usr/bin/tail.exe
./shared/noninteractive/install_build_tree/CMakeCache.txt:CMAKE_MAKE_PROGRAM:FILEPATH=D:/mingw64/usr/bin/make.exe
./nondynamic/noninteractive/build_tree/CMakeCache.txt:CMAKE_MAKE_PROGRAM:FILEPATH=D:/mingw64/usr/bin/make.exe
./nondynamic/noninteractive/build_tree/CMakeCache.txt:ENV_FOR_ONSGMLS:FILEPATH=D:/mingw64/usr/bin/env.exe
./nondynamic/noninteractive/build_tree/CMakeCache.txt:SED_EXECUTABLE:FILEPATH=D:/mingw64/usr/bin/sed.exe
./nondynamic/noninteractive/build_tree/CMakeCache.txt:SH_EXECUTABLE:FILEPATH=D:/mingw64/usr/bin/bash.exe
./nondynamic/noninteractive/build_tree/CMakeCache.txt:TAIL_EXECUTABLE:FILEPATH=D:/mingw64/usr/bin/tail.exe
./nondynamic/noninteractive/install_build_tree/CMakeCache.txt:CMAKE_MAKE_PROGRAM:FILEPATH=D:/mingw64/usr/bin/make.exe

So all should be well there, i.e., CMake has found the POSIX version
of the make command rather than the incorrect "mingw" native windows
version of the make command which presumably would have serious
command-line length issues if used for a traditional build (i.e., not
a cmake-based build where all sorts of measures are typically deployed
by CMake to beat Windows command-line limitations for both the POSIX
make case and the native Windows make case).

But I did notice that one possible issue is you simply used "make" for your comprehensive test.

What does "which make" give you for that platform?  Does it refer to
that above D:/mingw64/usr/bin/make.exe POSIX version or something else?

(To avoid this question in future, your invocation scripts for all
platforms should specify the exact make command that is being
used.), e.g.,

--build_command D:/mingw64/usr/bin/make.exe

for MinGW-w64/MSYS2.

But assuming that is not the issue, my working hypothesis is this
command-line length limitation is a MinGW-w64/MSYS2 bug since it does
not appear to be an issue for Cygwin's version of the POSIX make
command.

To investigate that hypothesis further, please
take a look at the extremely simple attached Makefile test case I have created.

On Linux the result (from the otherwise empty directory where this Makefile
is created) is

software@raven> make |wc
       1       1    2001

i.e., there is one line with no blanks so there is also one
blank-delimited word generated by that "echo" command inside that
Makefile, and that one line (and word) consists of 2001 bytes.

What happens if you run that test with SHELL suitably modified to the
Cygwin location for bash.exe from the same Cygwin environment you use
to test PLplot?

What happens if you run that test with SHELL changed to
D:/mingw64/usr/bin/bash.exe, see above from the same MinGW-w64/MSYS2
environment you use to test PLplot?

1.  If you get good results in the Cygwin case, but not the
MinGW-w64/MSYS2 case, then that is pretty good evidence of a bug in
the MinGW-w64/MSYS2 "make" package.

2. If both don't work, then that means long command lines work in
neither, and perhaps we have to write off this PLplot issue for the
MinGW-w64/MSYS2 case to longer path names/longer command lines in
that case compared to the Cygwin case.  But I consider this
result to be unlikely because according to
<https://stackoverflow.com/questions/19354870/bash-command-line-and-input-limit>
the Cygwin bash line limit is 32000 bytes, i.e., well above the 2001
bytes generated above.

3. But if both work, then you will have to dig deeper.

For example, you would need to double-check you could execute that
"echo" command in the Makefile without issues from bash.exe, and that
version of bash.exe was identical to what is specified for the SHELL
variable in the

10 relevant install-tree locations, i.e.,

irwin@raven> grep 'Makefile$' comprehensive_test_listing.out |grep install_tree
-rw-r--r-- 1 markus DIRECTORY+Group(513)    3432 Jul 30 12:09 ./nondynamic/noninteractive/install_tree/share/plplot5.12.0/examples/c/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)    4145 Jul 30 12:09 ./nondynamic/noninteractive/install_tree/share/plplot5.12.0/examples/c++/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)    2952 Jul 30 12:09 ./nondynamic/noninteractive/install_tree/share/plplot5.12.0/examples/fortran/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)    8742 Jul 30 12:09 ./nondynamic/noninteractive/install_tree/share/plplot5.12.0/examples/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)    2615 Jul 30 12:09 ./nondynamic/noninteractive/install_tree/share/plplot5.12.0/examples/tk/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)    3411 Jul 30 11:08 ./shared/noninteractive/install_tree/share/plplot5.12.0/examples/c/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)    4124 Jul 30 11:08 ./shared/noninteractive/install_tree/share/plplot5.12.0/examples/c++/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)    2927 Jul 30 11:08 ./shared/noninteractive/install_tree/share/plplot5.12.0/examples/fortran/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)  8738 Jul 30 11:08 ./shared/noninteractive/install_tree/share/plplot5.12.0/examples/Makefile
-rw-r--r-- 1 markus DIRECTORY+Group(513)  2577 Jul 30 11:08 ./shared/noninteractive/install_tree/share/plplot5.12.0/examples/tk/Makefile

So I have described three possible debugging trails to follow with the
attached simple test case, but my bet is you will find 1. to be true
which should be easy to deal with (e.g., with a bug report to the
MinGW-w64/MSYS2 which I can advise you about) assuming we have also
figured out a PLplot workaround such as -DNON_TRANSITIVE=ON for that
MinGW-w64/MSYS2 bug

Good luck, and I look forward to your next report for the
noninteractive MinGW-w64/MSYS2 case as well as the results of
the above simple tests for the attached Makefile

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0 --Cygwin Ada issue

[Arjen M. <Arj...@de...>]

Hi Alan,



See below in context.

Regards,

Arjen

> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Tuesday, August 01, 2017 5:02 AM
> To: Arjen Markus
> Cc: PLplot development list
> Subject: Planning for the release of 5.13.0 --Cygwin Ada issue
>
> On 2017-07-31 07:13-0000 Arjen Markus wrote:
>
> > Here are some results from my most recent, i.e. yesterday's, testing session.
>
> Hi Arjen:
>
> I first want to emphasize that I am well aware I am asking for a lot of testing from
> you, and thanks in advance for anything you can do!
>

:) It was surprising that it took so much time - I have most of this automated by a collection of ad hoc scripts, but some nursing is needed, especially if things do not work out as before.

> It turns out this is a long-standing issue, see your post from more than a year ago
>
> 2016-03-17 Arjen Markus    (21K) RE: comprehensive testing results on Cygwin -
> Ada build errors
>

Forgot all about that. The script I use for cdash reports takes care of the extra information needed for Ada, so I thought it worked fine.

> However, I think I have a lot better understanding of the issue now, then I did back
> then.
>
...

>
> So to test that possibility, please use the -DNON_TRANSITIVE=ON cmake option
> for this test to see if (a) you get further with Ada, and (b) if that option causes
> issues for other parts of our comprehensive tests for the Cygwin platform.  And
> depending on what combination of (a) and (b) you obtain, I will take it from there.
>
Very much the same thing: link errors concerning doubly defined symbols. There do not seem to be - at first sight - other issues, but they might come later in the build process.

Regards,

Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Planning for the release of 5.13.0 non-Ada Cygwin issues

[Alan W. I. <ir...@be...>]

Hi Arjen:

Further to your non-Ada report for Cygwin, I have analyzed it now for
any issues as follows:

0. comprehensive.sh.out indicates the script ran without any
showstopper issues.  i.e., no glitches, which is good!  Because of that
previous glitch I can see why you stuck to testing commit c8739dd0a
from 2017-07-16, but please now update to the latest version of that
master branch for all further testing not only to test my latest
commit, but also all the other commits that have been made since that
date.

1.
# Check for any warning regressions in the cmake output.
less shared/noninteractive/output_tree/cmake.out

That file is what I used to determine the present disabled component
regressions (compared to December results) that I remarked on previously
in this thread.  But you should look further in
that file for the specific reasons for those regressions that you
should address.

2. 
# Check for some non-standard warnings:
grep -i warning */*/output_tree/*.out |\
grep -vE 'cmake.out|ctest.out' |\
grep -vE 'PLPLOT WARNING|PRIVATE|deprecated|Resource leak|2 problems|Some graphical or stdout'
shared/noninteractive/output_tree/make_noninteractive.out:/cygdrive/d/plplot-svn/plplot-git/bindings/tk/Pltk_Init.c:40:12:
warning: ‘Matrix_Init’ redeclared without dllimport attribute:
previous dllimport ignored [-Wattributes]

I looked further into this issue, and I have
commited (38ac923) what I believe is the solution.

However, as emphasized by that commit message this change does
need testing on Windows platforms so I look forward
to your next noninteractive Cygwin comprehensive report
to demonstrate I have really fixed this issue (and not
introduced some other issue).


3.
# Check for all errors (other than the usual mention of test.error when
# cleaning traditional builds):
grep -i error */*/output_tree/*.out |grep -v 'traditional.*test.error'
nondynamic/noninteractive/output_tree/cmake.out:-- WARNING: Octave-4
has been found which is likely to lead to build errors for PLplot.
shared/noninteractive/output_tree/cmake.out:-- WARNING: Octave-4 has
been found which is likely to lead to build errors for PLplot.

The above cmake output is the usual warning about Octave-4 which has
been discussed for modern Linux distributions recently here.  My
(post-release) plan is to debug the Octave-4 case myself (with
appropriately patched swig if necessary) so that the example issues
that only occur for the UTF-8 examples are straightened out.

Until that issue is fixed, the above warning message is valid
and should be retained.

4.
# Check for any PostScript or Text differences between all non-C
# languages and the corresponding C results.
grep -B1 -A3 "Missing examples" */*/output_tree/*.out |less

These results were perfect.

In sum, this is a promising but still preliminary result for Cygwin
that is free of (hardware?) glitches, and assuming you deal with the
disabled component regressions and my fix for the above warning
actually works, we should be back to where we were in December.  And
we might even do better than that if my new idea for working with the
Ada issue works, and/or if you have time to try interactive testing of
wxwidgets alone for Cygwin.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0 non-Ada Cygwin issues

[Alan W. I. <ir...@be...>]

On 2017-07-31 07:13-0000 Arjen Markus wrote:

> Here are some results from my most recent, i.e. yesterday's, testing session.

Hi Arjen:

>> I. The noninteractive comprehensive test for Cygwin.
>>
>> * Find issues
>>
>> My records indicate you last ran a comprehensive test for Cygwin on 2016-12-15.
>> The options for that test were the same as the present ones, and that script was a
>> complete success.
>
> ...
>
>>
>> So the changes from December are the wxwidgets device driver is not present and
>> there is also a severe regression in enabled bindings.
>>
>> Is this exactly the same Cygwin system and launch script that you used before
>> when testing just before the release of PLplot-5.12.0?  If so, you should be able to
>> replicate those good December results by switching to PLplot-5.12.0 (e.g., by
>> running
>>

> It appears that when you add a package to the Cygwin installation,
an update follows for all the packages that are already present. This
led to me having to adjust the location where the Ada stuff is to be
found (change in version). I also had to add a few Python packages
(transition from Python 2 to Python 3).

[...]
> I have not had a look at the reason
the wxWidgets component was rejected. That is an issue for the coming
days.

To be clear, the issue is not limited to just wxwidgets.

Here is the status of disabled components of PLplot for the
current Cygwin test:

ENABLE_ada:             OFF
ENABLE_d:               OFF
ENABLE_java:            OFF
ENABLE_ocaml:           OFF
ENABLE_octave:          OFF
ENABLE_pdl:             OFF
ENABLE_pyqt4:           OFF
ENABLE_pyqt5:           OFF
ENABLE_itcl:            OFF
ENABLE_itk:             OFF
ENABLE_wxwidgets:       OFF

Of these,

ENABLE_pyqt4:           OFF
ENABLE_itcl:            OFF
ENABLE_itk:             OFF
ENABLE_wxwidgets:       OFF

are regressions compared to your successful December test. Likely
these regressions occur for the same reason (a system update for
Cygwin) as in the wxwidgets case.  Please deal with all these
regressions (not just the wxwidgets one) for your next Cygwin test.

By the way, your comprehensive _interactive_ tests of wxwidgets on
Cygwin are not a high priority for us because they can be expected to be slow
(assuming Cygwin has built their wxwidgets package against slow Cygwin
X rather than fast native Windows graphics).  Nevertheless, after you
have completed similar testing on MinGW-w64/MSYS2 and MSVC for fast
native Windows graphics, the corresponding Cygwin test is still a
"would be nice" for this release if you have the time after completing
the other two.  The reason for my interest in the Cygwin case is it
_should_ give the same result as on other POSIX systems such as Linux,
but it "would be nice" to confirm that.

>
>>
>> * Build error in the CMake-based build tree for the installed examples
>>
> ...
>>
>> make[3]: *** [c/CMakeFiles/x04c.dir/build.make:104: c/x04c.exe] Error 1
>>
>> but no accompanying diagnostic error message from gcc itself.
>>
>
> I have not seen that in the tests from yesterday.

Glad there don't appear to be any glitches in any of your
present reports (at least for now, see below).

> Which is an indication of something. Something to watch out for.

I absolutely agree.  As I emphasized to you off-list earlier today,
such inconsistent results (i.e., "glitches") are the hallmark of
hardware issues such as overheating or certain of your laptop's memory
modules beginning to go bad.  But it should be straightforward to
clean out dust (to reduce overheating typically by a substantial
amount since dust blocks airflow and insulates chips making it more
difficult for that reduced air flow to cool them) and also run a
memory checker (such as memtest86+
<https://en.wikipedia.org/wiki/Memtest86>) to detect any memory
modules that need to be replaced.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] Planning for the release of 5.13.0 --Cygwin Ada issue

[Alan W. I. <ir...@be...>]

On 2017-07-31 07:13-0000 Arjen Markus wrote:

> Here are some results from my most recent, i.e. yesterday's, testing session.

Hi Arjen:

I first want to emphasize that I am well aware I am asking for a lot
of testing from you, and thanks in advance for anything you can do!

Of course, the concern here is 40 per cent of our downloaders (as
measured by SourceForge from the 5.12.0 release date until today) use
some form of Windows, and ideally you and other here with access to
Windows platforms would comprehensively test on MinGW-w64/MSYS2, MSVC,
and Cygwin to reduce the number of bugs that our Windows users find
post-release, and also to identify the limitations (which components
have to be dropped) on each of those platforms.

Of course, this is all in the ideal case, and if testing from you over
the next few weeks is going to be severely constrained by other calls
on your time, then please do as much as you can **but also
communicate your time constraints to me**.  Then depending on what you
say, the progress I am making on some remaining irritating Linux
issues (how superscripts/subscripts and linebreak "\n" commands are
handled for wxwidgets and other devices), and in the interest of
getting out our fixes and new features in a timely manner, I will
likely decide to go ahead and release PLplot-5.13.0 without as much
Windows testing as we would like. (And likely not as much
superscripts/subscripts and linebreak fixing as I would like.)

More below in response to your Ada Cygwin report.  I will wait until
later to respond to your other two reports (with appropriatly modified
subject lines) to keep the size of each e-mail reasonable.

>> -----Original Message-----
>> From: Alan W. Irwin [mailto:ir...@be...]
>> Sent: Wednesday, July 19, 2017 1:43 AM
>> To: Arjen Markus
>> Cc: PLplot development list
>> Subject: RE: [Plplot-devel] Planning for the release of 5.13.0
>>
>> I. The noninteractive comprehensive test for Cygwin.
>>
>> * Find issues
>>
>> My records indicate you last ran a comprehensive test for Cygwin on 2016-12-15.
>> The options for that test were the same as the present ones, and that script was a
>> complete success.
>
> ...
>
>>
>> So the changes from December are the wxwidgets device driver is not present and
>> there is also a severe regression in enabled bindings.
>>
>> Is this exactly the same Cygwin system and launch script that you used before
>> when testing just before the release of PLplot-5.12.0?  If so, you should be able to
>> replicate those good December results by switching to PLplot-5.12.0 (e.g., by
>> running
>>

> It appears that when you add a package to the Cygwin installation,
an update follows for all the packages that are already present. This
led to me having to adjust the location where the Ada stuff is to be
found (change in version). I also had to add a few Python packages
(transition from Python 2 to Python 3).

That sounds like good package management on Cygwin's part to make sure
your system remains self-consistent.  (In contrast you have recently
run into inconsistent system problems for MinGW-w64/MSYS2, and those
are no fun at all.)

> Unfortunately, the new Ada packages result in a problem with
multiply defined symbols - see the corresponding tarball. So I had to
remove that from the test (by not expanding/explicitly setting the
CMAKE_LIBRARY_PATH environment variable).

It turns out this is a long-standing issue, see your post from
more than a year ago

2016-03-17 Arjen Markus    (21K) RE: comprehensive testing results on Cygwin - Ada build errors

However, I think I have a lot better understanding of the issue now,
then I did back then.

The command that leads to the build failure is

/usr/bin/gnatmake.exe  -Wl,--enable-auto-import "-aI/cygdrive/d/plplot-svn/plplot-git/examples/ada" "-aI/cygdrive/d/plplot-svn/plplot-git/bindings/ada" "-aL/cygdrive/d/plplot-svn/comprehensive_test_disposeable/shared/noninteractive/build_tree/bindings/ada/CMakeFiles/plplotada.dir" xtraditional13a.adb -cargs  -largs ../../dll/libplplotada.dll.a ../../dll/libplplot.dll.a /usr/lib/libltdl.dll.a /usr/lib/libdl.a /usr/lib/libshp.a /usr/lib/libfreetype.dll.a ../../dll/libcsirocsa.dll.a ../../dll/libcsironn.dll.a /usr/lib/libqhull.dll.a ../../dll/libqsastime.dll.a

The equivalent command (which works without issues) on my Debian Jessie platform is

/usr/bin/gnatmake  "-aI/home/software/plplot/HEAD/plplot.git/examples/ada" "-aI/home/software/plplot/HEAD/plplot.git/bindings/ada" "-aL/home/software/plplot/HEAD/build_dir/bindings/ada/CMakeFiles/plplotada.dir" xtraditional13a.adb -cargs  -largs ../../bindings/ada/libplplotada.so.2.1.0 ../../src/libplplot.so.14.0.0 -Wl,-rpath,/home/software/plplot/HEAD/build_dir/bindings/ada:/home/software/plplot/HEAD/build_dir/src

The Cygwin version of that command seems right to me for that platform
except for possibly one issue which is the huge overlinking that
occurs for that case, e.g., all the libraries mentioned by the -cargs
-largs option.  For Cygwin those libraries are both direct
(libplplotada, and libplplot) in our build tree) dependencies of the
Ada example build and also indirect (libcsirocsa, libcsironn, and
libqsastime.dll that PLplot builds for itself, and the system
libraries libltdl, libdl, libshp, libfreetype, and libqhull)
dependencies of the Ada example build (i.e., direct and indirect
dependencies of libplplot but not direct dependencies of the Ada
example build).  The (working) Linux version of the above command
points to just the direct dependencies of the Ada example build.  In
other words, (by default, see cmake/modules/plplot.cmake) our build
system by design uses non-transitive linking (i.e., only direct
dependencies) for POSIX platforms such as Linux, but transitive
linking (where every library is linked in that is mentioned indirectly
or directly) for non-POSIX platforms such as MinGW-w64/MSYS2, MSVC,
**AND Cygwin**.

But Cygwin prides itself on being mostly POSIX compatible. So I am
beginning to wonder if our default build-system rule is a mistake for
Cygwin that doesn't matter ordinarily, but gnatmake is perhaps
sensitive to this gross overlinking issue?

So to test that possibility, please use the -DNON_TRANSITIVE=ON cmake
option for this test to see if (a) you get further with Ada, and (b)
if that option causes issues for other parts of our comprehensive
tests for the Cygwin platform.  And depending on what combination of (a)
and (b) you obtain, I will take it from there.

Best wishes, and more later concerning your other two reports.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



Here are some results from my most recent, i.e. yesterday's, testing session.



The three tarballs have names that ought to be self-explanatory.

> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Wednesday, July 19, 2017 1:43 AM
> To: Arjen Markus
> Cc: PLplot development list
> Subject: RE: [Plplot-devel] Planning for the release of 5.13.0
>
> On 2017-07-18 12:36-0000 Arjen Markus wrote:
>
> > I attach three tarballs, resulting from the comprehensive test
...

>
> But after your holiday you should look in detail at the results of those analyses
> below.
>
> I. The noninteractive comprehensive test for Cygwin.
>
> * Find issues
>
> My records indicate you last ran a comprehensive test for Cygwin on 2016-12-15.
> The options for that test were the same as the present ones, and that script was a
> complete success.

...

>
> So the changes from December are the wxwidgets device driver is not present and
> there is also a severe regression in enabled bindings.
>
> Is this exactly the same Cygwin system and launch script that you used before
> when testing just before the release of PLplot-5.12.0?  If so, you should be able to
> replicate those good December results by switching to PLplot-5.12.0 (e.g., by
> running
>
It appears that when you add a package to the Cygwin installation, an update follows for all the packages that are already present. This led to me having to adjust the location where the Ada stuff is to be found (change in version). I also had to add a few Python packages (transition from Python 2 to Python 3).

Unfortunately, the new Ada packages result in a problem with multiply defined symbols - see the corresponding tarball. So I had to remove that from the test (by not expanding/explicitly setting the CMAKE_LIBRARY_PATH environment variable).

As this took quite some time, I have not had a look at the reason the wxWidgets component was rejected. That is an issue for the coming days.

>
> * Build error in the CMake-based build tree for the installed examples
>
...
>
> make[3]: *** [c/CMakeFiles/x04c.dir/build.make:104: c/x04c.exe] Error 1
>
> but no accompanying diagnostic error message from gcc itself.
>

I have not seen that in the tests from yesterday. Which is an indication of something. Something to watch out for.

>
> II. The noninteractive comprehensive test for MinGW-w64/MSYS2
>
> * Find issues:
>
> DRIVERS_LIST: cairo;mem;ntk;null;pdf;ps;svg;wingcc;wxwidgets;xfig
>
>
> ENABLE_ada:             OFF
> ENABLE_d:               OFF
> ENABLE_java:            OFF
> ENABLE_ocaml:           OFF
> ENABLE_octave:          OFF
> ENABLE_pdl:             OFF
> ENABLE_qt:              OFF
> ENABLE_pyqt4:           OFF
> ENABLE_pyqt5:           OFF
> ENABLE_itcl:            OFF
> ENABLE_itk:             OFF
>
> The only good historical comparison we have to these results is Greg's (see
> <https://sourceforge.net/p/plplot/wiki/Testing_PLplot/#Testing%20Reports>).
>
> where ada, d, java, ocaml, octave, pyqt4, itcl, and itk were not present in the tests
> for various reasons.  So the only real regression above from those historical results
> was qt was disabled (because it was not found).  So the above is a quite
> encouraging result but you are not quite there yet.
>
> To resolve that last included component regression, what Qt-related packages (if
> any) have you installed?
>
> If the answer is none, I would preserve your current MinGW-w64/MSYS2 snapshot
> (since a new snapshot might not work at all), modify your automated script to install
> that platform from scratch to include Qt
> (Qt4 is preferred but if packaging conflicts insist you need to use Qt5, then use that
> instead) and run that script to install a new snapshot with its own unique install
> prefix.
>
> Then repeat this test on that new snapshot platform with
> DCMAKE_CXX_FLAGS=-fabi-version=8 adjusted to a new number if that is
> needed).

I have not got Qt installed yet - was concentrating on wxWidgets :). Yet another one to take care of.

>
> * ERROR: Traditional make test_noninteractive failed in the installed examples tree
> (for the nondynamic case)
>
> A google search for the term <"gcc.exe: Bad address"> didn't appear to find
> anything relevant.  (There was one issue in Cygwin for 2012 involving commands
> separated with a semicolon, but we don't use a semicolon in the present case.)
>
I tried the following:

- Run the build command explicitly (it is 1800+ characters long!) - this succeeded without any complaint and the resulting program did its job.

- Run make so that the program would be built. That failed with the "Bad address" message.

My current conclusion is that the length of the command is causing a problem in make. It may just go over some internal edge. There is not much PLplot-ish about this, except for the length of the directory names, it would seem.

>
> III. The interactive comprehensive test for MinGW-w64/MSYS2 that is limited just to
> wxwidgets - and beyond ...
>
All to be looked into still.



DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

[Plplot-devel] Extensive testing of PLplot completed successfully for CMake-3.9.0

[Alan W. I. <ir...@be...>]

I recently (commit 79ed7a) worked around a CMake Java language issue
by modifying the official UseSWIG.cmake file.  As part of the testing
for that change, I took the opportunity to comprehensively test PLplot
using CMake-3.9.0 (see the commit message for the details).  I found
no issues with that test so others should feel reasonably confident
about using CMake-3.9.0 to build and test PLplot as well.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Install location logic substantially updated

[Alan W. I. <ir...@be...>]

On 2017-07-22 01:59-0700 Alan W. Irwin wrote:

[...]
> The above commit was rather intrusive so I gave it a complete
> noninteractive and interactive comprehensive test which passed with
> only minor dashboard submission issues (see the commit message for
> commit 62e07d4 for the details) which I have since fixed.
>
> However, our comprehensive tests do not test the
>
> make DESTDIR=<location of staging area> install
>
> trick used by packagers.  So I would appreciate you doing that
> and checking to see if this series of commits has introduced any
> problems.

To Ole and Orion:

I have just tried testing the use of DESTDIR as above by hand. As
expected, all files were installed in the staging area rather than the
designated install location.  I then moved that staging area to the
designated install location and built the test_noninteractive and
test_interactive targets by hand for that location for both our
CMake-based build system for the installed examples, and our
traditional (Makefile + pkg-config) build system for the installed
examples.  All seemed well for these 4 tests.  So that is a reassuring
result concerning our install location relocatability.

However, I still encourage both of you to attempt to package our git
master branch since that not only tests the above capability, but also
tests the latest PLplot in other useful ways.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________