plplot-devel — Technical discussion about PLplot development

Re: [Plplot-devel] [Bug] Plmap omits lines at the left hand side of the map

[Phil R. <p.d...@gm...>]

Hi Mark
Thanks for the reminder. I will definitely put as much effort as I can
into getting this sorted pre release.

Phil

On 24 July 2017 at 13:27, Mark de Wever <m.d...@da...> wrote:
> Hi Phil,
>
> I saw the release of 5.13 is being planned. I wondered whether it is
> possible to fix this issue before the release.
>
> Regards,
> Mark de Wever
>
>
> On 18.11.16 18:30, p.d...@gm... wrote:
>>
>> Hi Mark
>> Thanks for the report. I'll have a look at your example over the weekend
>> and see if I can fix the issue.
>>
>> Phil
>>
>> Sent from my Windows 10 phone
>>
>> From: Mark de Wever
>> Sent: 18 November 2016 11:06
>> To: plplot-dev
>> Subject: [Plplot-devel] [Bug] Plmap omits lines at the left hand side of
>> themap
>>
>> Hi,
>>
>> Recently I ran into an issue with the plplot 5.11.1 on Windows. The
>> plmap code seems to omit lines entirely when a part of the line is not
>> visible. This only occurs when the line is not visible on the left hand
>> side of the plot. When a part is not visible on the right hand side it
>> is properly shown.
>>
>> At my Debian Stable system I use the system's 5.10.0 version. There the
>> issue doesn't occur. (I noticed the plmap code has been rewritten
>> between 5.10.0 and 5.11.0.) I also tested the bug on Debian Stable with
>> 5.11.1 and a recent master [d71e48], both have the issue.
>>
>> Attached a modified example 19 where the bug is shown. The first plot
>> shows the entire coast of Ireland. The second plot should only omit a
>> small part of the coast, but instead the entire coast has been removed.
>> Only the internal border of Ireland remains visible. This seems to
>> happen with all coast lines; I picked Ireland since it shows the bug
>> nicely.
>>
>>
>> Regards,
>> Mark de Wever
>>
>> PS: It seems the old deprecated plmap code no longer shows a map (at
>> least on Windows). I didn't investigate further, I just installed the
>> shapelib library.
>>
>> PPS: There also seems to be another issue with a filled polygon map, but
>> I'm still investigating. If it really is an issue, I'll report it.
>>
>>
>

Re: [Plplot-devel] [Bug] Plmap omits lines at the left hand side of the map

[Mark de W. <m.d...@da...>]

Hi Phil,

I saw the release of 5.13 is being planned. I wondered whether it is 
possible to fix this issue before the release.

Regards,
Mark de Wever

On 18.11.16 18:30, p.d...@gm... wrote:
> Hi Mark
> Thanks for the report. I'll have a look at your example over the weekend and see if I can fix the issue.
> 
> Phil
> 
> Sent from my Windows 10 phone
> 
> From: Mark de Wever
> Sent: 18 November 2016 11:06
> To: plplot-dev
> Subject: [Plplot-devel] [Bug] Plmap omits lines at the left hand side of themap
> 
> Hi,
> 
> Recently I ran into an issue with the plplot 5.11.1 on Windows. The
> plmap code seems to omit lines entirely when a part of the line is not
> visible. This only occurs when the line is not visible on the left hand
> side of the plot. When a part is not visible on the right hand side it
> is properly shown.
> 
> At my Debian Stable system I use the system's 5.10.0 version. There the
> issue doesn't occur. (I noticed the plmap code has been rewritten
> between 5.10.0 and 5.11.0.) I also tested the bug on Debian Stable with
> 5.11.1 and a recent master [d71e48], both have the issue.
> 
> Attached a modified example 19 where the bug is shown. The first plot
> shows the entire coast of Ireland. The second plot should only omit a
> small part of the coast, but instead the entire coast has been removed.
> Only the internal border of Ireland remains visible. This seems to
> happen with all coast lines; I picked Ireland since it shows the bug nicely.
> 
> 
> Regards,
> Mark de Wever
> 
> PS: It seems the old deprecated plmap code no longer shows a map (at
> least on Windows). I didn't investigate further, I just installed the
> shapelib library.
> 
> PPS: There also seems to be another issue with a filled polygon map, but
> I'm still investigating. If it really is an issue, I'll report it.
> 
> 

[Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

We have made lots of progress (see our git log) since the release of
5.12.0 so I am looking forward to releasing 5.13.0 in the near future
(i.e., in early or late August depending on how much time Arjen has for
PLplot testing in the remainder of this month, and how soon we can
debug the issues he has found with his preliminary Windows platform
testing.

I am now declaring a soft code freeze for this release (which means
large documentation upgrades are still fine, but no actual code
changes should be merged to the master branch that will be the basis
of this release other than bug fixes).  Right now, this code freeze is
quite soft, but as we approach closer to the release it will harden
almost completely (i.e., only the simplest and most trivial bug fixes
will be accepted).  The point of this rapidly hardening policy is
those of us that test need some largely quiet time with regard to
actual code changes for a week or so just before the release to
finalize our testing results.

N.B. this code freeze does not preclude me continuing to work with Phil on
his private topic branch concerning implementing and using C exceptions
to handle all our C errors.

The projects that we need to finish before this release are the
following with all three of these release-critical in my view.

* Completely debug the Windows variant of the 3-semaphores approach to
   IPC between -dev wxwidgets and wxPLViewer.

   The status of this project is Arjen and I have gone through a number
   of iterations of code changes and Linux comprehensive wxwidgets tests on my part and
   similar comprehensive tests of wxwidgets on Windows platforms on
   Arjen's part.  My comprehensive test on Linux was fine (except
   for subscript rendering issues, see below) and
   Arjen also got a promising preliminary result (confined to just
   the shared library build tree) on MinGW-w64/MSYS2 that he needs
   to extend to the nondynamic, static, and install tree cases.

   In addition Arjen's tests with the MSVC compiler discovered an
   uninitialized variable in the superfix/subfix rendering code for
   wxwidgets.  I am the last developer to have touched that complicated
   code so it is up to me to fix it.  But the reason I touched it
   before was it was not rendering correctly at all.  The superscript
   rendering was a lot better after I modified that code to fix some
   obvious issues AND to introduce some semiempirical corrections to
   the vertical spacing for superscripts, but there was still an
   egregious subscript rendering issue (subscripts were being rendered
   exactly like superscripts).  I expect the uninitialized variable
   explains why I was having so much trouble debugging that code, but I
   am virtually positive there is more work to do there than that.
   Although I have not been able to work on this recently because of
   other interesting PLplot developments, I do plan to dig deep into
   that code and finally make it work properly starting tomorrow.

   Of course, such bug fixes are allowed and positively encouraged
   during the soft part of the code freeze so if anyone else wants to
   deal with a bug that has been bothering you (other than the
   well-known plfill bug where there are some deep issues I want to
   deal with myself, and now likely post-release again), I strongly
   encourage you to do that now.

* Comprehensive tests on the Linux and non-interactive comprehensive
   tests on MinGW-w64/MSYS2, MSVC, Cygwin, and
   Mac OS X platforms with fixes for all obvious issues discovered with
   these tests (or if the fix is not obvious or is complex, dropped
   components of PLplot for the platform in question).

   The status of this project is as follows:

   The comprehensive test success that I got last month on Linux has
   now recently been confirmed again (actually for a case where once
   again the prefixes for the source, build, and install trees all
   contained a space).  That Debian Jessie platform has all possible
   PLplot soft dependencies installed (albeit for rather old versions
   of software).

   As a result essentially all Linux-relevant components of PLplot
   were tested, and the tests were done for both noninteractive and
   interactive components of PLplot.  Thus, we are in good shape on
   Linux outside of the PLplot soft dependency version issues brought
   up recently by Ole such as Octave 4 which only works for pure ascii
   examples versus Octave-3 on Debian Jessie that works for both ascii
   and non-ascii UTF-8 examples.  (This octave-4 non-ascii UTF-8 issue
   is something I plan to deal with post-release once I have had a
   chance to install Debian stable = Stretch, testing, and unstable
   following Ole's suggestion.)

   Arjen had completed a successful comprehensive test of all relevant
   noninteractive components of Cygwin much earlier in this release
   cycle so I thought we were in good noninteractive shape on
   that platform, but for some reason that same test is now erroring out
   so that issue needs a lot further investigation by Arjen
   to discover the cause of that peculiar noninteractive regression on
   Cygwin.  For example, it has some symptoms that are consistent
   with it being a hardware failure, but there is no proof of that
   until he gets the classic inconsistent results for exactly the same test
   that are typically associated with some sort of hardware failure
   (e.g., errors in the memory hardware).

   Furthermore, Arjen's first test of an almost fully loaded (except
   for Qt) MinGW-w64/MSYS2 platform on the same box errored out so
   this is a regression against the good Greg Jung results we had for a
   fully loaded earlier version of this platform ~2 years ago. So this
   MinGW-w64/MSYS2 platform regression needs a lot further
   investigation by Arjen as well.

   My understanding is Arjen also plans a comprehensive noninteractive MSVC
   (+ Unix tools on PATH) test with one PLplot component removed
   that did not work the last time he tried this test on that platform.

   If some volunteer steps forward to do a comprehensive test of
   the latest PLplot on Mac OS X, that would be nice, but I don't view
   that as release critical.

* The typedef project

   The status of that project is my old plan (explained in the release
   notes for 5.12.0) to change most PLplot generic pointers to have the
   const attribute is a bad way to create const correctness because
   that imposes a long-term burden on users of these argument types to
   never pass back information using the generic pointers in their
   callbacks. So I have already (a couple of weeks ago) backed away
   from that plan and have gone back to what we had in 5.11.1 which was
   all our generic pointers in our API have the type PLPointer which is
   typedefed to void *.  The generic pointers PL_NC_GENERIC_POINTER and
   PL_GENERIC_POINTER (both of which were introduced in 5.12.0 by
   typedefing them to void *, but there _were_ big plans to change
   PL_GENERIC_POINTER to const void * and use that const type for the
   majority of our generic pointer API) will continue to be typedefed
   as void *.  However, they are now officially deprecated in favor of
   the PLPointer alternative which PLplot users have been using for
   years. So currently PL_NC_GENERIC_POINTER and PL_GENERIC_POINTER are
   typedef'd in include/plplot.h, but they have both been replaced by
   PLpointer everywhere in our code (other than that file) and also in
   our documentation.

   So the current result of the above work is the set of typedefs we use in
   our DocBook documentation is consistent with the non-deprecated
   typedefs specified in include/plplot.h.  So that is an excellent
   start!

   However, there is obviously lots more to do here such as to
   propagate the use of our non-deprecated typedefs (many first defined
   in 5.12.0) to all our C and C++ code in our bindings (e.g., our C++
   binding itself as well as the swig-generated Python, Java, Octave,
   and Lua bindings that are implemented using swig-generated C and C++
   source code) and our drivers and examples that are written in either
   C or C++.  Finally I would also like to make all our C and C++ code
   const correct.  However, I have run out of time in this release
   cycle so I will put off these typedef propagation efforts and
   updating our code to improve const correctness until after the
   release.  For the same reason, I plan to put off working
   on the plfill bug and OCaml PostScript difference issues
   until after this release as well.

So where we stand is the first topic above is relatively close to
being done, and the last topic is done.  However, the second topic
will have a large degree of uncertainty with regard to when it will be
resolved until Arjen's further tests determine the source of those
peculiar noninteractive regressions on those two Windows platforms.

So we are clearly getting much closer to the release because we have
made a lot of progress in the last few weeks and we are now at the
stage where I feel comfortable about declaring the soft freeze.  Also,
I am pretty sure the release will be in August, but I cannot be more
definite than that right now until Arjen figures out the causes of
those two testing regressions.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Plplot-devel mailing list
Plp...@li...
https://lists.sourceforge.net/lists/listinfo/plplot-devel

[Plplot-devel] PLplot Debian package -- spelling issues fixed

[Alan W. I. <ir...@be...>]

On 2017-07-15 11:49+0200 Ole Streicher wrote:

> And, finally, there are a few spelling errors found by lintian:
> directorys -> directories
> Continous -> Continuous
> argment -> argument
> argments -> arguments

The "directories" spelling issues had already been independently
corrected in this release cycle, but the "continuous" and "argument"
spelling issues were splattered across 22 files in our source tree!
So thanks for reporting those, and I have now fixed them in an
automated way (commit 5fc9108).

I have now reviewed all your posts, and I believe this e-mail
concludes my specific response to topics you have raised so far
(although the C exception handling work that spun off from our
discussion obviously continues due to Phil's efforts to finish his
proof of concept with some input and lots of well-deserved
encouragement from me).

I have very much appreciated your input and help (especially with the
nn and csa licensing issues), and I look forward to more input from
you once you have had a chance to package the git master branch of
PLplot that is soon going to lead to the release of 5.13.0.

Best wishes,

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. I. <ir...@be...>]

Hi Phil:

On 2017-07-23 22:00+0100 Phil Rosenberg wrote:

> On 23 July 2017 at 12:27, Alan W. Irwin <ir...@be...> wrote:
>> I suggested in my big e-mail to you early this week (Mon, 17 Jul 2017
>> 14:59:00 -0700 (PDT)) concerning your second iteration a method of doing
>> this using a list of plstream heap pointers.  Please respond to that
>> suggestion.
>
> When you said a list I'm not sure I understood. Do you mean a linked
> list rather than an array?

No, I was using list in a generic sense so it you store that
information (and also access it) in an array form that would be fine
with me. So basically the idea is whatever you are doing now in
plfreeall for handling plstream heap pointers correctly should also be
done in plmalloc, plcalloc, plrealloc, and plfree so we can 
replace all calls of malloc, calloc, realloc, and free in our
code with calls to the pl* variants of those.

>
>>
>> Also please respond to my suggestion (with appropriate code changes
>> while I handle the CMake part) in that e-mail concerning
>>
>> #define PL_IMPLEMENT_EXCEPTION_HANDLING and
>> #ifdef PL_IMPLEMENT_EXCEPTION_HANDLING
>>

Please go back to look at that original e-mail for this sub-topic, but
to summarize what I meant is please implement this preprocessor
instruction idea for iteration 4, and I will
subsequently implement a CMake option to control whether
PL_IMPLEMENT_EXCEPTION_HANDLING is #defined or not.


>> On 2017-07-22 22:47+0100 Phil Rosenberg wrote:
>>
>>> [in that same e-mail] You asked why every API call required a PLTRY block.
>>> You are correct
>>> that actually they don't. Every API call that may result in a call to
>>> plexit needs a PLTRY block.
>>
>>
>> I initially didn't understand this explanation, but after _much_ too
>> long a time considering various scenarios to explain my point of view
>> the light finally dawned. Exception handling does not allow you to
>> transfer control to arbitrary places in the code (a stupid assumption
>> I was making). Instead, it merely allows you to unwind the call graph
>> from the plexit call (in this case that is the only current use of
>> PLTHROW in our source code) to return control to a routine potentially
>> far up the call stack from there such as _any_ of the public API
>> routines (as you said) that have a possible call to plexit somewhere
>> lower down on their call graph.
>
> Yes correct, we just unwind the stack to a certain point in the call
> sequence (or whatever terminology you choose as you mentioned in your
> ps and pps).  This is good, because the user application is still sat
> waiting for our API function to return. We don't want to jump to some
> specific execution point, we need that function to return and for the
> user to find out somehow that there has been an error.

Thanks for that confirmation.  It is a big relief to me to finally move
beyond my previous misconception.

>
>>
>> So sorry for my previous noise on this sub-topic, but
>> now we are finally on the same page in this regard, I have some
>> further questions and comments about your implementation of
>> exception handling and how you are using that exception handling
>> to deal with bad errors.
>>
>> * If you miss putting a PLTRY and PLCATCH block in one of those public
>>   API cases where a potential call to plexit is somewhere lower on the
>>   call graph, then what does the current code do?  Does it simply
>>   crash (i.e., is that the potential cause of the serious issues I
>>   have reported to you in my previous post) or do you handle that
>>   internal error case by, e.g., emitting an "internal error: unhandled
>>   exception" message followed by an exit?
>
> I think that currently something very odd. The value of
> pls->currentjumpbuffer used in PLTHROW would be either out of date or
> NULL, so probably some sort of crash. This is why it is important that
> every API entry function includes a PLTRY, PLCATCH, PLENDTRY sequence.
> Of course we could set things up to check for this scenario, but then
> what would we do? Call exit()? I had been trying to think if there was
> a C method to check at compile time, but I cannot think of one (there
> would be ways in C++ I think). But the API changes so infrequently
> that I think just ensuring all API functions include the PLTRY,
> PLCATCH, PLENDTRY code is probably okay - open to suggestions though.

Assuming we develop this proof-of-concept into the real deal and merge
this topic to master (and so far I haven't seen any issues in this
proof-of-concept that would preclude reaching that eventual result),
there will be lots of functions where we need to add PLTRY and
PLCATCH blocks.

So given that situation where unhandled exceptions may be a fact of
life for us for some time (and possibly any time in the future when
new topics with e.g., additional externally accessible API are merged
to master) I think we should handle this situation with a specific
error message and calling exit rather than crashing with, e.g., a
segfault.  That conclusion is also based on my impression that is
exactly how C++ deals with the unhandled exception issue. Furthermore,
I think this is fundamentally a more satisfactory result than a crash
whenever an unhandled exception occurs (especially if we copy this
approach for our other non-plplot C libraries and our C device drivers,
see further discussion below of this possibility).

I mentioned "lots of functions" above.  Note those will not just be
limited to the official "c_*" API functions.  Instead, our
long-standing visibility work has determined all functions that are
externally accessed in our tests, and we have effectively labelled all
of those with the PLDLLIMPEXP visibility macro that is #defined in
include/pldll.h.in.  So that means identification of all the functions
that need PLTRY/PLCATCH blocks has already been done for us.  However,
there are a maximum of 326 of them spread over 6 files as revealed by

software@raven> grep -l 'PLDLLIMPEXP  *' include/*.h* |grep -v pldll
include/drivers.h
include/ltdl_win32.h
include/pdf.h
include/plplot.h
include/plplotP.h
include/plstrm.h

software@raven> grep 'PLDLLIMPEXP  *' include/*.h* |grep -v pldll |wc -l
326

(Note, I have included drivers.h and pdf.h in the above list because
in some configurations all driver code is absorbed into the core
PLplot library.)

I emphasize you don't have to be too concerned about inserting many
more PLTRY/PLCATCH blocks for your proof-of-concept. But assuming the
time comes in the relative near future when your C exception handling
proof-of-concept has matured sufficiently so you at least want to
merge it to the master branch as an option (see discussion above of
the PL_IMPLEMENT_EXCEPTION_HANDLING macro) for PLplot users, then
potentially there is a lot of editing work here with the prospect of
dealing with typographical bugs in that work.

Thus, would it be possible to deal with this situation automatically
with appropriate macro to generate the code (consisting of a PLTRY
block containing an appropriate call to the original renamed function,
and empty PLCATCH block and all visibility issues taken care of)?

However, if you don't think this automated approach is possible or it
is too much work or too error prone, then you can count on me to help
you with the required manual editing work for those 300+ functions
when the time comes.

>> * I also suggest you should implement a PLplot library state variable
>>   (let's call it plplot_library_status) that is initialized to 0 (good
>>   status) when the library is initialized but set to 1 (bad status) by
>>   plexit. Then follow up by implementing a c_plstatus() public API
>>   that returns the value of library_status and which emits a message
>>   such as ("library had bad error so it had to be shut down.  To use
>>   it again you need to initialize the library by calling one of the
>>   plinit family of routines".) Then an external application could call
>>   c_plstatus to check on the status of the library after any call to
>>   our public API and act accordingly.
>>
>>   Or it could be sloppy about that (i.e., our standard C examples
>>   might only call plstatus in one example to make sure it works). In
>>   which case after the library is shutdown (see above) by an internal
>>   call to plexit there would be a blizzard of further plabort (due to
>>   plsc->level being 0) and plexit messages after the first one because
>>   of that library shutdown.  But there would be no call to the dreaded
>>   exit routine so there is at least the chance that the external
>>   application will survive that sloppiness.  And if plexit does not
>>   call plend, I think the chances of an external application
>>   encountering a showstopper error are much worse.
>
> Hmm, this is quite interesting actually. As I said above I would like
> to not totally kill the PLStream - a good example could be that maybe
> plexit gets called due to a bad path for a shapefile for a map. This
> maybe could be fixed if the calling program checks for an error state.
> Of course people are lazy :-) so it might not get checked. In my
> initial thinking, the map plot would fail but all other plotting would
> carry on. maybe it would be better to set the PLStream to an error
> state and have all further API calls fail. Maybe the user could clear
> the error state to permit plotting to continue. I welcome thoughts on
> this.
>
> In terms of an error flag that you suggested. I am thinking perhaps
> all calls to plerr result in an error message being logged in the
> PLStream, plus when we reach the top level PLTRY/PLCATCH/PLENDTRY we
> call a user provided error callback. Then we have an API function to
> get the list of errors, another to clear the list of errors and one to
> check if there are any errors (which would just just check if the
> number of error messages in the log is greater than 0). So it is like
> your flag, but with some extra detail.

I have now thought a bit more about this issue with propagating
library error conditions to other C libraries and executables.
Because of the #includes that are normally used by all such libraries
that call the core PLplot library, these "other libraries and
executables" have access to the same PLTRY, PLCATCH, and PLTHROW
macros that the core PLplot library does.  So I am thinking along
the lines of all instances of PLplot calls occurring in our
standard examples, e.g.,

pladv(0);

being replaced by

pladv(0);
if(plstatus())
   PLTHROW(NULL);

The point is, this is a bit cleaner than

pladv(0);
if(plstatus())
{
   fprintf(stderr "There was an error in one of the PLplot library calls\n");
   exit(1);
}

So I hope that first form using PLTHROW is possible in the standard
examples.

In any case, no matter how you decide to propagate errors across C
library boundaries, I think our ultimate goal should be to use
exception handling (with at least a PLTRY and PLCATCH block in each of
all our C libraries and standard C executables and interface our C
exception handling with native language exception handling or C++
device drivers, bindings, and standard examples, and for bindings and
examples for all non-C++ languages (e.g., Ada, D, Python, OCaml) which
support native exception handling.  I do realize I am speculating
quite a bit here, but I hope you take it in the intended spirit which
is to generate discussion about the absolute best way to propagate library
errors across library boundaries considering these future needs.

To summarize, I think to help finish this proof-of-concept you need to

* fix the bugs I found for iteration 3,

* implement the "exit" idea for the case of unhandled exceptions to
   avoid crashing in that case,

* implement the idea concerning the PL_IMPLEMENT_EXCEPTION_HANDLING macro,

* finalize how you are going to propagate errors across library
   boundaries, and

* demonstrate a completely unlazy version of one of the C examples
   which itself has a PLTRY block and PLCATCH block in its main routine
   and PLTHROWs associated with each PLplot call executed if plstatus()
   (or whatever you decide to replace it with) indicates a library error
   condition.

And I need to

* Test all your remaining iterations until this proof of concept is
   finished including devising some additional tests with a local patch
   (obviously not to be committed) to generate lots of calls to plexit through many different potential
   call stacks to test propagation of errors for most PLplot commands
   of most examples including the non-lazy one and all the lazy ones.

* Implement a commit that removes trailing blanks if you don't do
   that yourself.

* Implement a commit that adds a CMake option to control whether
   the PL_IMPLEMENT_EXCEPTION_HANDLING C macro is #defined or not.

Once all tests are passed with the finished proof-of-concept then we
can safely merge this to master (with PL_IMPLEMENT_EXCEPTION_HANDLING
defaulting to undefined), and then afterward expand to full PLTRY
coverage (automatically or manually implemented) for all externally
visible PLplot functions.  And at some point introduce completely
consistent error handling for all our memory allocation calls
whether they correspond to a plstream pointer or not.

Once we are completely happy with the exception handling in libplplot
and our C examples, we should turn the default option ON so
PL_IMPLEMENT_EXCEPTION_HANDLING is #defined by default for most of our
users, and as time permits extend exception handling to our other C
libraries, our C and C++ device drivers and our bindings and examples
for all supported languages that implement their own native exception
handling.

So it is going to be quite a lot of work to get from here to there.
But this is a lofty goal (to eliminate exit completely from our entire
code base).  And I think the results as we gradually eliminate the
exit call from our entire code base (except for internal errors due to
exceptions which are not handled by accident). will be well worth it.
Also, as we phase in each component of this work, that phase is not
and all or nothing proposition since each component can be protected
by its own non-default CMake option (just like I am proposing with the
PL_IMPLEMENT_EXCEPTION_HANDLING macro) for the proof-of-concept once
it is finished.  So the work can be considerably spread out over a
long time and remain on our master branch during all of that time.


> Thanks for the thoughts and comments so far. They are very welcome.

And the same goes for your thoughts and comments as well!

Furthermore, since the lack of error handling has long been one of the
most important fundamental PLplot library issues, it is great to see
all your C exception-handler ideas fleshed out and implemented in your
current proof-of-concept rather than us having a rather speculative
discussion with absolutely no proof-of-concept code to help keep the
discussion focussed.

Best wishes,

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Phil R. <p.d...@gm...>]

On 23 July 2017 at 12:27, Alan W. Irwin <ir...@be...> wrote:
> Hi Phil:
>
> Here is further discussion of what you have said as opposed to the
> error report I sent you in my previous post concerning your 3rd
> iteration.
>
> Do keep in mind the possibility for the future that the memory
> allocation part of your commits are worthwhile in their own right and
> could be finished and landed on master much sooner than the rest of
> your commits that are concerned with implementation of C exception
> handling and use of that for handling our errors.

I'm not sure how useful they really are without exception handling.
Without such handling there is no location in the code where the array
of pointers to the allocated memory is reviewed and in addition the
best response to a failed allocation may not be a call to plexit (but
I guess mostly it would be).

>
> I suggested in my big e-mail to you early this week (Mon, 17 Jul 2017
> 14:59:00 -0700 (PDT)) concerning your second iteration a method of doing
> this using a list of plstream heap pointers.  Please respond to that
> suggestion.

When you said a list I'm not sure I understood. Do you mean a linked
list rather than an array?

>
> Also please respond to my suggestion (with appropriate code changes
> while I handle the CMake part) in that e-mail concerning
>
> #define PL_IMPLEMENT_EXCEPTION_HANDLING and
> #ifdef PL_IMPLEMENT_EXCEPTION_HANDLING
>
> On 2017-07-22 22:47+0100 Phil Rosenberg wrote:
>
>> [in that same e-mail] You asked why every API call required a PLTRY block.
>> You are correct
>> that actually they don't. Every API call that may result in a call to
>> plexit needs a PLTRY block.
>
>
> I initially didn't understand this explanation, but after _much_ too
> long a time considering various scenarios to explain my point of view
> the light finally dawned. Exception handling does not allow you to
> transfer control to arbitrary places in the code (a stupid assumption
> I was making). Instead, it merely allows you to unwind the call graph
> from the plexit call (in this case that is the only current use of
> PLTHROW in our source code) to return control to a routine potentially
> far up the call stack from there such as _any_ of the public API
> routines (as you said) that have a possible call to plexit somewhere
> lower down on their call graph.

Yes correct, we just unwind the stack to a certain point in the call
sequence (or whatever terminology you choose as you mentioned in your
ps and pps). This is good, because the user application is still sat
waiting for our API function to return. We don't want to jump to some
specific execution point, we need that function to return and for the
user to find out somehow that there has been an error.

>
> So sorry for my previous noise on this sub-topic, but
> now we are finally on the same page in this regard, I have some
> further questions and comments about your implementation of
> exception handling and how you are using that exception handling
> to deal with bad errors.
>
> * If you miss putting a PLTRY and PLCATCH block in one of those public
>   API cases where a potential call to plexit is somewhere lower on the
>   call graph, then what does the current code do?  Does it simply
>   crash (i.e., is that the potential cause of the serious issues I
>   have reported to you in my previous post) or do you handle that
>   internal error case by, e.g., emitting an "internal error: unhandled
>   exception" message followed by an exit?

I think that currently something very odd. The value of
pls->currentjumpbuffer used in PLTHROW would be either out of date or
NULL, so probably some sort of crash. This is why it is important that
every API entry function includes a PLTRY, PLCATCH, PLENDTRY sequence.
Of course we could set things up to check for this scenario, but then
what would we do? Call exit()? I had been trying to think if there was
a C method to check at compile time, but I cannot think of one (there
would be ways in C++ I think). But the API changes so infrequently
that I think just ensuring all API functions include the PLTRY,
PLCATCH, PLENDTRY code is probably okay - open to suggestions though.

>
> * What happens when an external application calls a "first" public API
>   which has another "second" public API lower down in its call graph
>   and even further down on the call graph of that second API occurs a
>   potential call to plexit? Then if plexit is called that will PLTHROW
>   control to the catch block of that second API rather than the first.
>   So for this particular call graph case for the first API this second
>   API needs to PLTHROW control to that first PLCATCH block rather than
>   simply returning.  And, of course, since that second API is
>   sometimes called directly from external applications and libraries
>   without anything higher in the call graph with a PLCATCH block, it
>   cannot PLTHROW all the time without sometimes running
>   into the internal error discussed above.  So how are you going to figure
> out
>   when to PLTHROW in a public API PLCATCH block and when not to PLTHROW?
>   Or is that case already automatically taken care of by your present code
>   (e.g., by each PLTRY macro incrementing a library status flag
>   so that the PLCATCH macro can check that flag to see if there
>   is a PLCATCH block higher in the call graph that it should PLTHROW
>   to)?

I had considered this ;-) We store a stack of jump buffers which
increases in size by one every time we enter a new PLTRY block. Also
the memory management marks this point with a NULL in the list of
pointers to allocated memory. A PLTHROW unwinds to the first
PLTRY/PLCATCH/PLENDTRY block it finds. At this point memory back to
the previous NULL is released and the top jump buffer is removed from
the stack. If there are any more jump buffers in the stack then part
of the PLENDTRY code does another PLTHROW with the new top jump buffer
of the stack. This repeats until there are no jump buffers left and we
will be at the API function initially called by the user. At least
that is the plan - I am yet to do the testing :-)

>
> * Shouldn't your current plexit call plend (like the master branch
>   version does) to shut down the library?  See further discussion
>   below concerning this suggestion.
My first thought is no - I would like the state of Plplot to be as it
was before the API call was made - although I have just realised that
this will not be the case, because the PLStreams are on the heap, not
the stack, they won't be restored by the stack unwind. I will need to
do this manually and probably do something like delay all memory frees
until PLENDTRY so I can restore the pointers.
>
> * I also suggest you should implement a PLplot library state variable
>   (let's call it plplot_library_status) that is initialized to 0 (good
>   status) when the library is initialized but set to 1 (bad status) by
>   plexit. Then follow up by implementing a c_plstatus() public API
>   that returns the value of library_status and which emits a message
>   such as ("library had bad error so it had to be shut down.  To use
>   it again you need to initialize the library by calling one of the
>   plinit family of routines".) Then an external application could call
>   c_plstatus to check on the status of the library after any call to
>   our public API and act accordingly.
>
>   Or it could be sloppy about that (i.e., our standard C examples
>   might only call plstatus in one example to make sure it works). In
>   which case after the library is shutdown (see above) by an internal
>   call to plexit there would be a blizzard of further plabort (due to
>   plsc->level being 0) and plexit messages after the first one because
>   of that library shutdown.  But there would be no call to the dreaded
>   exit routine so there is at least the chance that the external
>   application will survive that sloppiness.  And if plexit does not
>   call plend, I think the chances of an external application
>   encountering a showstopper error are much worse.

Hmm, this is quite interesting actually. As I said above I would like
to not totally kill the PLStream - a good example could be that maybe
plexit gets called due to a bad path for a shapefile for a map. This
maybe could be fixed if the calling program checks for an error state.
Of course people are lazy :-) so it might not get checked. In my
initial thinking, the map plot would fail but all other plotting would
carry on. maybe it would be better to set the PLStream to an error
state and have all further API calls fail. Maybe the user could clear
the error state to permit plotting to continue. I welcome thoughts on
this.

In terms of an error flag that you suggested. I am thinking perhaps
all calls to plerr result in an error message being logged in the
PLStream, plus when we reach the top level PLTRY/PLCATCH/PLENDTRY we
call a user provided error callback. Then we have an API function to
get the list of errors, another to clear the list of errors and one to
check if there are any errors (which would just just check if the
number of error messages in the log is greater than 0). So it is like
your flag, but with some extra detail.
>
>> However, if we just have a policy that
>> every API call must have a PLTRY block then it makes it easy to ensure
>> nothing gets missed. But perhaps this is overkill. A number of
>> functions in plcore.c, particularly the "get" type ones are very
>> simple and would never call plexit. I'm happy to take advice on where
>> the balance should lie?
>>
>> One other issue at the moment. I currently output a warning when
>> plfree is called on a pointer that is not being managed by the
>> PLStream. At present this will include all plfree calls made in
>> plend/plend1 and can include memory allocated by the contour routines.
>> The reason for this is due to the following chain of events
>> 1) These bits of memory get allocated by plalloc in an API call
>> somewhere (e.g cmap0).
>> 2) On exit from the API call plfreeall gets called
>> 3) plfreeall spots the allocated memory that wasn't freed and checks
>> all the pointers that belong to the PLStream
>> 4) It finds that the pointers belong to PLStream and need to not be
>> freed. It therefore stops tracking them.
>> 5) In plend1 plfree is called on the pointer. plfree cannot see the
>> pointer in the list of tracked pointers so issues a warning.
>>
>> There are a few options for dealing with this, but I haven't decided
>> which to use. For now my apologies for the list of warnings.
>>
>
> I am frankly out of gas, and this is already a pretty long e-mail
> so I will leave it to you to decide on the above topics unless
> someone else here wants to comment.
>
> Anyhow, after I get some sleep, I am looking forward to your further
> comments on (a) my error report for interation 3, and (b) what I have
> had the energy to discuss above.

Thanks for the thoughts and comments so far. They are very welcome.

>
> Alan
>
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. I. <ir...@be...>]

On 2017-07-23 06:00-0700 Alan W. Irwin wrote:

P.P.S.  As I am just discovering, one of the huge problems in this
field is terminology. For example, the wikipedia article
<https://en.wikipedia.org/wiki/Call_stack> refers immediately to 6
synomyms (ouch!) for call stack with one of those being just plain
"stack" which is the terminology used by the Linux man pages for
setjmp and longjmp.

"Call chain" is not one of those wikipedia-approved synonyms, but I
believe I must have picked up that terminology from another article I
read (where it meant the same thing).  Therefore, since the principal
terminology in wikipedia is "call stack", that is much preferable to
the "call chain" terminology I used below.  But nevertheless, I
thought I did pretty well below considering I wrote that at
6 o'clock in the morning after a long night of analysis without sleep.  :-)

Alan

> P.S. I just reviewed the wikipedia article on call graphs, and it
> appears I was misusing that term and should have used call chain
> instead where a call graph for a given library consists of all the
> potential call chains in the library with each one starting at one of
> the public API's.
>
> So my understanding now is the PLTHROW macro in plexit unwinds the
> particular call chain that called plexit until it reaches the first
> routine higher in that call chain where the
> the call was made in a PLTRY block and transfer control to the code in the
> corresponding PLCATCH block.
>
> @Phil:
>
> Do you agree with this explanation of PLTHROW, PLTRY, and PLCATCH or
> does it need further correction?
>
> If that explanation is substantially correct, that one scenario I was
> discussing could be described as two possible call chains in the call
> graph with one starting at the first public API traversing through the
> second public API and eventually traversing to plexit, and the second
> starting at the second public API and eventually traversing to plexit
> as well.  So the question was does the code need enhancement to deal
> with those two different possible call chains where in one case the
> PLCATCH block of the second API needs to PLTHROW to unwind the call
> chain to the first API PLTRY and transfer control to the corresponding
> PLCATCH block, and in the other case the second API catch block needs
> simply to return to the calling routine in the external application.
>
> And similarly this change in terminology needs to be applied to my
> just-previous discussion of other topics related to PLTHROW, PLTRY,
> and PLCATCH.
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Plplot-devel mailing list
> Plp...@li...
> https://lists.sourceforge.net/lists/listinfo/plplot-devel
>

__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. I. <ir...@be...>]

P.S. I just reviewed the wikipedia article on call graphs, and it
appears I was misusing that term and should have used call chain
instead where a call graph for a given library consists of all the
potential call chains in the library with each one starting at one of
the public API's.

So my understanding now is the PLTHROW macro in plexit unwinds the
particular call chain that called plexit until it reaches the first
routine higher in that call chain where the
the call was made in a PLTRY block and transfer control to the code in the
corresponding PLCATCH block.

@Phil:

Do you agree with this explanation of PLTHROW, PLTRY, and PLCATCH or
does it need further correction?

If that explanation is substantially correct, that one scenario I was
discussing could be described as two possible call chains in the call
graph with one starting at the first public API traversing through the
second public API and eventually traversing to plexit, and the second
starting at the second public API and eventually traversing to plexit
as well.  So the question was does the code need enhancement to deal
with those two different possible call chains where in one case the
PLCATCH block of the second API needs to PLTHROW to unwind the call
chain to the first API PLTRY and transfer control to the corresponding
PLCATCH block, and in the other case the second API catch block needs
simply to return to the calling routine in the external application.

And similarly this change in terminology needs to be applied to my
just-previous discussion of other topics related to PLTHROW, PLTRY,
and PLCATCH.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] Third proof of concept for the "exception handling" topic -- Further discussion

[Alan W. I. <ir...@be...>]

Hi Phil:

Here is further discussion of what you have said as opposed to the
error report I sent you in my previous post concerning your 3rd
iteration.

Do keep in mind the possibility for the future that the memory
allocation part of your commits are worthwhile in their own right and
could be finished and landed on master much sooner than the rest of
your commits that are concerned with implementation of C exception
handling and use of that for handling our errors.

I suggested in my big e-mail to you early this week (Mon, 17 Jul 2017
14:59:00 -0700 (PDT)) concerning your second iteration a method of doing
this using a list of plstream heap pointers.  Please respond to that
suggestion.

Also please respond to my suggestion (with appropriate code changes
while I handle the CMake part) in that e-mail concerning

#define PL_IMPLEMENT_EXCEPTION_HANDLING and
#ifdef PL_IMPLEMENT_EXCEPTION_HANDLING

On 2017-07-22 22:47+0100 Phil Rosenberg wrote:

> [in that same e-mail] You asked why every API call required a PLTRY block. You are correct
> that actually they don't. Every API call that may result in a call to
> plexit needs a PLTRY block.

I initially didn't understand this explanation, but after _much_ too
long a time considering various scenarios to explain my point of view
the light finally dawned. Exception handling does not allow you to
transfer control to arbitrary places in the code (a stupid assumption
I was making). Instead, it merely allows you to unwind the call graph
from the plexit call (in this case that is the only current use of
PLTHROW in our source code) to return control to a routine potentially
far up the call stack from there such as _any_ of the public API
routines (as you said) that have a possible call to plexit somewhere
lower down on their call graph.

So sorry for my previous noise on this sub-topic, but
now we are finally on the same page in this regard, I have some
further questions and comments about your implementation of
exception handling and how you are using that exception handling
to deal with bad errors.

* If you miss putting a PLTRY and PLCATCH block in one of those public
   API cases where a potential call to plexit is somewhere lower on the
   call graph, then what does the current code do?  Does it simply
   crash (i.e., is that the potential cause of the serious issues I
   have reported to you in my previous post) or do you handle that
   internal error case by, e.g., emitting an "internal error: unhandled
   exception" message followed by an exit?

* What happens when an external application calls a "first" public API
   which has another "second" public API lower down in its call graph
   and even further down on the call graph of that second API occurs a
   potential call to plexit? Then if plexit is called that will PLTHROW
   control to the catch block of that second API rather than the first.
   So for this particular call graph case for the first API this second
   API needs to PLTHROW control to that first PLCATCH block rather than
   simply returning.  And, of course, since that second API is
   sometimes called directly from external applications and libraries
   without anything higher in the call graph with a PLCATCH block, it
   cannot PLTHROW all the time without sometimes running
   into the internal error discussed above.  So how are you going to figure out
   when to PLTHROW in a public API PLCATCH block and when not to PLTHROW?
   Or is that case already automatically taken care of by your present code
   (e.g., by each PLTRY macro incrementing a library status flag
   so that the PLCATCH macro can check that flag to see if there
   is a PLCATCH block higher in the call graph that it should PLTHROW
   to)?

* Shouldn't your current plexit call plend (like the master branch
   version does) to shut down the library?  See further discussion
   below concerning this suggestion.

* I also suggest you should implement a PLplot library state variable
   (let's call it plplot_library_status) that is initialized to 0 (good
   status) when the library is initialized but set to 1 (bad status) by
   plexit. Then follow up by implementing a c_plstatus() public API
   that returns the value of library_status and which emits a message
   such as ("library had bad error so it had to be shut down.  To use
   it again you need to initialize the library by calling one of the
   plinit family of routines".) Then an external application could call
   c_plstatus to check on the status of the library after any call to
   our public API and act accordingly.

   Or it could be sloppy about that (i.e., our standard C examples
   might only call plstatus in one example to make sure it works). In
   which case after the library is shutdown (see above) by an internal
   call to plexit there would be a blizzard of further plabort (due to
   plsc->level being 0) and plexit messages after the first one because
   of that library shutdown.  But there would be no call to the dreaded
   exit routine so there is at least the chance that the external
   application will survive that sloppiness.  And if plexit does not
   call plend, I think the chances of an external application
   encountering a showstopper error are much worse.

> However, if we just have a policy that
> every API call must have a PLTRY block then it makes it easy to ensure
> nothing gets missed. But perhaps this is overkill. A number of
> functions in plcore.c, particularly the "get" type ones are very
> simple and would never call plexit. I'm happy to take advice on where
> the balance should lie?
>
> One other issue at the moment. I currently output a warning when
> plfree is called on a pointer that is not being managed by the
> PLStream. At present this will include all plfree calls made in
> plend/plend1 and can include memory allocated by the contour routines.
> The reason for this is due to the following chain of events
> 1) These bits of memory get allocated by plalloc in an API call
> somewhere (e.g cmap0).
> 2) On exit from the API call plfreeall gets called
> 3) plfreeall spots the allocated memory that wasn't freed and checks
> all the pointers that belong to the PLStream
> 4) It finds that the pointers belong to PLStream and need to not be
> freed. It therefore stops tracking them.
> 5) In plend1 plfree is called on the pointer. plfree cannot see the
> pointer in the list of tracked pointers so issues a warning.
>
> There are a few options for dealing with this, but I haven't decided
> which to use. For now my apologies for the list of warnings.
>

I am frankly out of gas, and this is already a pretty long e-mail
so I will leave it to you to decide on the above topics unless
someone else here wants to comment.

Anyhow, after I get some sleep, I am looking forward to your further
comments on (a) my error report for interation 3, and (b) what I have
had the energy to discuss above.

Alan

__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] Third proof of concept for the "exception handling" topic

[Alan W. I. <ir...@be...>]

On 2017-07-22 22:47+0100 Phil Rosenberg wrote:

> Attached is a new patch series in a tarball. I don't see a huge need
> to keep two separate branches any more. In fact I just tried to
> disentangle my latest few commits into two branches using git rebase
> -i and endured a whole world of pain.

Hi Phil:

Thanks very much for this third version of your exception handler
private topic branch.  It does sound like you prefer to keep your
development environment simpler (at least for now) with just one
branch so I will take that approach as well.

After unpacking the tarball here in

~irwin/Phil.Rosenberg/20170722/exception_handler/

I did the following:

# Create up-to-date master branch
git checkout master
git fetch
git merge --ff-only origin/master
# create new private topic branch from that branch
git checkout -b exception_handler3
# Apply _all_ your commits to that branch
cat ~irwin/Phil.Rosenberg/20170722/exception_handler/*.patch |git am

The result here of that last command was the following:

Applying: Added functions that will record memory allocations in the PLStream
Applying: Part way through plmalloc implementation
Applying: Removed plinitializememorylist and plfreeall calls
Applying: Fix issue with growing memorylist
Applying: Added missing ;
Applying: Added Memory allocation function declarations to plplotP.h
Applying: Added some further memory management functions and check for PLStream memory
/home/software/plplot/HEAD/plplot.git/.git/rebase-apply/patch:104: trailing whitespace.
//Cause a pointer to no longer be managed by plplot, 
warning: 1 line adds whitespace errors.
Applying: Fixed some problems with memory management and performed style update.
/home/software/plplot/HEAD/plplot.git/.git/rebase-apply/patch:98: trailing whitespace.

warning: 1 line adds whitespace errors.
Applying: moved memory management functions from plstrm.h to plplotP.h removing duplicate definitions.
Applying: Removed c_usplparseopts function, which was pollution from another branch
Applying: Removed use of plmalloc and plfree from plcont.c. This was pollution from another branch.
Applying: Added Exception handling macros/functions/variables
Applying: Add some test code for exception handling
Applying: Fixed PLENDTRY/PLTHROW bug
Applying: Remove plexit calls from advancejumpbuffer and plinitialisememorylist
Applying: Added some comments around PLTRY, PLCATCH and PLENDTRY
Applying: Free memory from jumpbuffer stack in plend1
Applying: Added PLTRY blocks to all the api functions in core.c
Applying: Added use of plmalloc to the contour code.
Applying: Restyle only

So all your commits applied cleanly on this fresh and up-to-date topic branch
aside from some extremely minor trailing whitespace issues which should
be easy for me to fix (but I won't do that yet) after the fact
by running the (Unix-tools-only) scripts/remove_trailing_whitespace.sh.

I tested that branch by setting the environment variable
CFLAGS='-g' (to generate line-number information in valgrind
reports), making a fresh configuration of PLplot with
the -DVALGRIND_ALL_TESTS=ON cmake option, and running

make VERBOSE=1 test_c_psc >& test_c_psc.out

Unlike my original good benchmark with this test, the result for this
branch was not clean, i.e. test_c_psc.out (collected in the attached
tarball) ends with an actual valgrind error for example 14.  For
historical reasons examples 14, 17, and 29 are done last so because of
this error for example 14, examples 17 and 29 were not executed, but
all the rest of our standard examples from 00 to 33 (except for 32
which we historically do not test because it has not been propagated
to languages other than C so there is no interesting basis of
comparison) were run and returned success (a 0 exit code). However,
the commands

software@raven> grep -L "0 errors from 0 contexts" examples/valgrind.x??c.psc.log
examples/valgrind.x11c.psc.log
examples/valgrind.x14c.psc.log
examples/valgrind.x21c.psc.log
software@raven> grep -L "no leaks are possible" examples/valgrind.x??c.psc.log
examples/valgrind.x14c.psc.log

demostrate there are imperfect valgrind results for examples 11, 14,
and 21 and actually looking at those three files (also collected in
the attached tarball) show those errors are quite serious in nature,
e.g., the initial trouble for all three is an invalid write which is
normally quite bad.  Furthermore, for example 14 that initial error
compounded so badly that valgrind itself exited with an error message
(and non-zero exit code).  I hope these collected valgrind reports
with exact source code line numbers will help you debug all these issues
so we can get back to perfect valgrind reports for the next iteration.

This is already long enough with an attachment for you to respond to
as well so I will place my further response to your additional remarks in a
separate e-mail.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Install location logic substantially updated

[Alan W. I. <ir...@be...>]

On 2017-07-22 15:07+0200 Ole Streicher wrote:

> It may be however be useful if you could create a "release candicate"
> (or beta) tarball, and give some time to test it. I could create a
> Debian package out of that and upload to our "experimental" branch.
> While you do probably not want to wait until it builds on all platforms
> (it has to go through review of our ftp-masters), I can do some tests
> locally and report back (but please, give some time; it is holiday
> season :-) )

Hi Ole:

I doubt I will bother with a release candidate.  But we all make
a pretty strong effort to keep the git master branch in good shape,
and the current tip of the master branch (i.e.,

4c90970 Build system: Quiet cmake developer warning

) is especially well tested.  So I suggest
you use that commit or later for your ongoing testing of
the development version of PLplot.

Of course, that commit is not a formal release candidate.  For
example, the version numbers (library SONAMEs as well as overall
version) have not been upgraded yet, and that will only happen some
time after our source code freeze and the associated review of the git
log to see what changes have been made since 5.12.0.  But the point is
this well-tested version is much closer to 5.13.0 than 5.12.0 so if
you fix all the issues for this development version (with me attending
to some of them upstream like the install location issues I just
addressed) you will likely have very little left to do on the Debian
packaging front when 5.13.0 is officially released.

Probably more important still, this is a relatively good time for me
to make upstream changes based on your experience with the developer
version of PLplot, but my willingness to implement such changes is
going to rapidly decrease (i.e., I will likely put you off until
post-release unless you have found something absolutely release
critical) as we move closer to the release.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Install location logic substantially updated

[Alan W. I. <ir...@be...>]

On 2017-07-22 11:42+0100 p.d...@gm... wrote:

> Just to check Alan, the mention of grep and all th linux centric
paths listed made me wonder how (if at all) this affects Windows
builds?

Hi Phil:

The find and grep commands were simply used to discover (hopefully)
all of the install location variables we potentially use in our source
tree for all platforms.  So you should have no platform concerns about
how I made that list of variables.

However, I do emphasize this is an intrusive change that needs testing
on all platforms to make sure I did not introduce any
platform-dependent problems by accident.

To start that testing process I did such testing for my combination of
Windows native CMake (version 3.8.2 downloaded in
binary form directly from KitWare), Wine-staging-2.10, and ancient (4.7.2)
version of MinGW/MSYS for the "MSYS Makefiles" generator.

For the cmake command-line options

-DCMAKE_INSTALL_PREFIX:PATH=/z/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging
-DFORTRAN_MOD_DIR:PATH='Z:\whateveR\dire ctorY'

(the single quotes are to protect the backslashes and space in that
string from interpretation or processing by bash.exe) the selected
cmake ouput results are
as follows:

[...]
name does not start with "CMAKE_INSTALL_".

CMAKE_INSTALL_PREFIX: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging
CMAKE_INSTALL_EXEC_PREFIX: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging
CMAKE_INSTALL_BINDIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/bin
CMAKE_INSTALL_DATADIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share
CMAKE_INSTALL_LIBDIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/lib
CMAKE_INSTALL_INCLUDEDIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/include
CMAKE_INSTALL_INFODIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share/info
CMAKE_INSTALL_MANDIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share/man
CMAKE_INSTALL_PKG_CONFIG_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/lib/pkgconfig
DATA_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share/plplot5.12.0
LIB_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/lib
INCLUDE_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/include/plplot
BIN_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/bin
DRV_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/lib/plplot5.12.0/drivers
DOC_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share/doc/plplot
INFO_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share/info
MAN_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share/man
PKG_CONFIG_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/lib/pkgconfig
FORTRAN_MOD_DIR: Z:/whateveR/dire ctorY
CMake Warning at cmake/modules/summary.cmake:44 (message):
   FORTRAN_MOD_DIR = Z:/whateveR/dire ctorY has prefix inconsistency with
   CMAKE_INSTALL_PREFIX =
   z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging
Call Stack (most recent call first):
   CMakeLists.txt:277 (summary)

ADA_INCLUDE_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/share/ada/adainclude/plplotada
ADA_LIB_DIR: z:/home/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/lib/ada/adalib/plplotada

Other important CMake variables:
[...]

Note, the above list of variables and values is shorter than what I
published for the Linux case because this platform provides few soft
dependencies for PLplot so many of the optional install-location
variables are not set.

The above output values of install-location variables have no specific
path transformations done by the CMake logic in our build system.
Therefore, those output results reflect whatever path transformations
are applied _by default_ by CMake to 
"-D<variable_name>:PATH=<value>" user-input values, e.g.,
CMAKE_INSTALL_PREFIX and FORTRAN_MOD_DIR in the above case.

So you can from this example, that this Windows native version of
CMake converts all input :PATH value formats to the native CMake
format for :PATH values. That transformation leaves case the same as
input (which is good because some Windows platforms provide
case-sensitive filenames), converts the slash form of drive letters to
the colon form (e.g., CMAKE_INSTALL_PREFIX) and converts backwards
slashes to forward slashes (e.g., FORTRAN_MOD_DIR).  And further
experiments showed that if the input value of a :PATH variable was in
the native CMake format for :PATH values, no transformation was done
to that value at all.  Note the values of these variables that are not
specified by the user are generally derived by appending sensible
defaults (in forward slash form) to :PATH prefixes (e.g.,
CMAKE_INSTALL_PREFIX) higher in the list that have been specified by
the user.  So the net result is all these internal CMake values are in
native CMake format for :PATH values.  As a result, we appear to have
the ideal case that if a Windows platform user wants to modify any of
the above results he can do so using exactly the same native CMake
:PATH variable format as is output above, but users are
also allowed to use other input formats for :PATH variables.

If there is a prefix inconsistency the user is warned of that (as in
FORTRAN_MOD_DIR above) but our build system continues on despite that
inconsistency, i.e., the files installed using the particular
install-location variable that uses an inconsistent prefix will be
installed in that inconsistent location.  And I have confirmed the
user gets this warning even if it is only a case-sensitive prefix
inconsistency, e.g., the upper-case "E" in /homE below

-DFORTRAN_MOD_DIR:PATH=z:/homE/wine/wine_staging/build_results/install-2.10_jessie_wine_staging/whatever

I contemplated making the prefix comparison with
${CMAKE_INSTALL_PREFIX} in a case insensitive way to avoid this kind
of warning but I thought better of it after my google search told me
there are windows platforms out there where a registry change can make
filenames (and I assume directory and drive-letter names) case
sensitive. However, as a result of the case-sensitive comparison in
the present logic, if a user specifies a prefix for the value of one
of the install-location variables that differs only in case from
${CMAKE_INSTALL_PREFIX}, they will get the warning which will be
spurious.  However, detecting whether Windows is case sensitive or not is
likely a nightmare, there is no harm from this spurious warning for
case-insensitive windows, and it is easy enough for the user to quiet
this warning by specifying exactly (including case) consistent
install prefixes.

@Everybody:

So I have pretty much come to the end of the tests I can apply for
this change both on Linux and Windows, and I am satisfied with those
results.  However, I repeat it has been an intrusive change so the
Windows users here should try some equivalent experiments to what I
tried above for their favorite generator just to make sure for that
generator that all input :PATH variables are translated to internal
format for any of such variables that are output in the cmake summary
(and in CMakeCache.txt), and that internal format is also suitable for
:PATH variable input.

Of course, much more extensive testing than such limited experiments
would be extremely useful as well leading up to our release so Arjen's
efforts to make on-going runs of scripts/comprehensive_test.sh for at
least the noninteractive case for Cygwin, MinGW-w64/MSYS2, and MSVC (+
the Unix tools on PATH trick that Arjen got working recently) are most
appreciated.  Of course, it is important for others lurking on this
list to participate in such important tests as well to make sure all
your own variants of Windows and Linux platforms are covered by such
testing (and to insure at least one variant of Mac OS X is covered by
our testing).  So please contact me if you are interested in adding
yourself to the comprehensive testing team for PLplot, and we can take
it from there.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] First proof-of-concept versions of PLplot "memory allocation" and "exception handling" topics

[Phil R. <p.d...@gm...>]

Hi Alan
I have sorted the issue with example 8. It turned out I had put some
plmalloc calls in there and forgotten about them.

Attached is a new patch series in a tarball. I don't see a huge need
to keep two separate branches any more. In fact I just tried to
disentangle my latest few commits into two branches using git rebase
-i and endured a whole world of pain. So I won't be returning to the
separate memory allocation functions branch any more. In the attached
patch series, patches 1-11 just have memory allocation functions.
After applying just these patches you should have source that builds
and runs, with the new functions defined but not used. patches 12 and
onward introduce the PLTRY, PLCATCH and PLENDTRY blocks and add them
into the API calls in plcore.c, plcont.c and plvpor.c. I have pretty
well tested plcont.c calls, but not plcore or plvpor calls.

You asked why every API call required a PLTRY block. You are correct
that actually they don't. Every API call that may result in a call to
plexit needs a PLTRY block. However, if we just have a policy that
every API call must have a PLTRY block then it makes it easy to ensure
nothing gets missed. But perhaps this is overkill. A number of
functions in plcore.c, particularly the "get" type ones are very
simple and would never call plexit. I'm happy to take advice on where
the balance should lie?

One other issue at the moment. I currently output a warning when
plfree is called on a pointer that is not being managed by the
PLStream. At present this will include all plfree calls made in
plend/plend1 and can include memory allocated by the contour routines.
The reason for this is due to the following chain of events
1) These bits of memory get allocated by plalloc in an API call
somewhere (e.g cmap0).
2) On exit from the API call plfreeall gets called
3) plfreeall spots the allocated memory that wasn't freed and checks
all the pointers that belong to the PLStream
4) It finds that the pointers belong to PLStream and need to not be
freed. It therefore stops tracking them.
5) In plend1 plfree is called on the pointer. plfree cannot see the
pointer in the list of tracked pointers so issues a warning.

There are a few options for dealing with this, but I haven't decided
which to use. For now my apologies for the list of warnings.

Thoughts welcome as ever.

Phil

On 17 July 2017 at 23:07, Alan W. Irwin <ir...@be...> wrote:
> On 2017-07-17 14:59-0700 Alan W. Irwin wrote:
>
>> With regard to testing the above memory_allocation2, I repeated the
>> same test as before (used the -DVALGRIND_ALL_TESTS=ON cmake option,
>> built the test_c_psc target, and checked for any imperfect valgrind
>> reports).  Examples 00 through 07 were fine, but example 08 segfaulted.
>> I have attached the associated valgrind report to help you to debug
>> this issue.
>
>
> Oops!  This time I really do attach that valgrind log.
>
>
> Alan
>
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

Re: [Plplot-devel] Install location logic substantially updated

[Ole S. <deb...@li...>]

Hi Alan,

On 22.07.2017 10:59, Alan W. Irwin wrote:
> So the user now has complete freedom to make a mess by attempting
> to specify inconsistent install location variables

I am very happy with that freedom :-) Thank you for these changes.

> @Orion:
> 
> In particular, does Fedora have policy about where *.tcl files should be
> installed that is consistent with Debian policy about that?
> 
> The reason I ask is if there is some consensus on that question I
> would likely change the default value of TCL_DIR from
> /usr/share/plplot5.12.0/tcl to that consensus.

One comment here: Debian has both the /usr/lib/tcltk and
/usr/share/tcltk paths usable; first for the architecture dependent
files (in an arch dependent subdir), second for architecture independent
files. This is, however, mostly to save space on multi-arch systems
(i686 and x64); as long as there are no tens of megabytes, I have no
objections to put everything together into /usr/lib/tcltk/<arch>/.
However, whatever goes into /usr/share/, must be truly arch independent.

> @Both:
> 
> Also note, that when our build system was first put together in the
> autotools era, Rafael insisted on using versioned plplot names
> throughout, (e.g, /usr/share/plplot5.12.0)  so I have stuck with that
> since.  But I
> am not as keen on that as he was so if your consensus (if it exists)
> for the TCL_DIR default is unversioned, e.g.,
> 
> /usr/share/tcltk/plplot
> 
> that would be fine with me.

Both is fine with me. We don't allow multi-version installations, so it
finally does not matter.

> @Ole:
> 
> You sure do ask interesting questions.  :-)
> 
> Seriously, though, keep them coming since such
> questions play an important role in improving
> PLplot as in these last few commits.
> 
> As far as I can remember, this current answer from me only leaves the
> spelling issues reported by lint (and you) that I need to answer, but
> can you recall any further topic between us that I still need to answer?

Not for the moment. The current Debian release (5.10.0) has some minor
multi-arch issues, but they seem to be a problem of the old packaging. I
think with the simplified packaging, this will go away.

It may be however be useful if you could create a "release candicate"
(or beta) tarball, and give some time to test it. I could create a
Debian package out of that and upload to our "experimental" branch.
While you do probably not want to wait until it builds on all platforms
(it has to go through review of our ftp-masters), I can do some tests
locally and report back (but please, give some time; it is holiday
season :-) )

Best regards

Ole

Re: [Plplot-devel] Install location logic substantially updated

[<p.d...@gm...>]

Just to check Alan, the mention of grep and all th linux centric paths listed made me wonder how (if at all) this affects Windows builds?

Phil

Sent from my Windows 10 phone

From: Alan W. Irwin
Sent: 22 July 2017 10:00
To: Ole Streicher; Orion Poplawski; PLplot development list
Subject: [Plplot-devel] Install location logic substantially updated

On 2017-07-13 09:12+0200 Ole Streicher wrote:

> Another point is that TCLDIR is currently set to
> /usr/share/plplot5.12.0/tcl, which is not in the package path for
> Tcl/Tk. Is there a way to change this to something like
> /usr/share/tcltk/plplot5.12.0 [1] with a cmake option so that I
> don't need a patch here?

To Ole and Orion:

@Both:

Ole's question (and I think Orion had similar concerns in the past)
inspired me to "open a can of worms" which was to substantially update
how we configured our install locations (see last 4 commits ending
with 4c90970).

Most of the previous relevant build system code was inspired by
autotools and has been untouched for a decade so I decided to rework a
lot of it after using a fairly elegant find .. grep stanza (see commit
message for commit 62e07d4 which is the most important commit
in this series) to determine a complete list of install
location variables (some mandatory, some optional) which are now
created and summarized consistently.

When I first implemented the original code I was quite concerned by
the user mess it would make if a user started changing install
locations inconsistently.  So the version of the code just prior
to this recent series of commits propagated that attitude and
therefore made it impossible to change many of the install-location
variables (e.g., TCL_DIR was unchangeable). But now I have decided to
allow users to change any of these variables any way they like.

So here is a recent summary (much longer than before because I found
many install location variables in our code that were not treated
consistently before) of our install location variables for the case where I
specified the -DTCL_DIR=/whatever1 cmake option.  This is taken
from the latest version of the cmake output:
_______________________________________________

Summary of CMake build system results for PLplot

Install location variables which can be set by the user.  N.B. These
variables are ordered by decreasing degree of generality, with the
_default_ values of the later ones in the list determined by the
values of variables earlier in the list.  So, for example, it is
usually sufficient in the vast majority of cases to just set
CMAKE_INSTALL_PREFIX, and the rest of these variables are adjusted
accordingly (at least for a fresh configuration), and it is rare
indeed that is is necessary for a user to set any variable here whose
name does not start with "CMAKE_INSTALL_".

CMAKE_INSTALL_PREFIX: /home/software/plplot/installcmake
CMAKE_INSTALL_EXEC_PREFIX: /home/software/plplot/installcmake
CMAKE_INSTALL_BINDIR: /home/software/plplot/installcmake/bin
CMAKE_INSTALL_DATADIR: /home/software/plplot/installcmake/share
CMAKE_INSTALL_LIBDIR: /home/software/plplot/installcmake/lib
CMAKE_INSTALL_INCLUDEDIR: /home/software/plplot/installcmake/include
CMAKE_INSTALL_INFODIR: /home/software/plplot/installcmake/share/info
CMAKE_INSTALL_MANDIR: /home/software/plplot/installcmake/share/man
CMAKE_INSTALL_PKG_CONFIG_DIR: /home/software/plplot/installcmake/lib/pkgconfig
DATA_DIR: /home/software/plplot/installcmake/share/plplot5.12.0
LIB_DIR: /home/software/plplot/installcmake/lib
INCLUDE_DIR: /home/software/plplot/installcmake/include/plplot
BIN_DIR: /home/software/plplot/installcmake/bin
DRV_DIR: /home/software/plplot/installcmake/lib/plplot5.12.0/drivers
DOC_DIR: /home/software/plplot/installcmake/share/doc/plplot
INFO_DIR: /home/software/plplot/installcmake/share/info
MAN_DIR: /home/software/plplot/installcmake/share/man
PKG_CONFIG_DIR: /home/software/plplot/installcmake/lib/pkgconfig
FORTRAN_MOD_DIR: /home/software/plplot/installcmake/lib/fortran/modules/plplot
JAR_DIR: /home/software/plplot/installcmake/share/java
JAVAWRAPPER_DIR: /home/software/plplot/installcmake/lib/jni
PYTHON_INSTDIR: /home/software/plplot/installcmake/lib/python3.4/site-packages
PLPLOT_OCTAVE_DIR: /home/software/plplot/installcmake/share/plplot_octave
OCTAVE_M_DIR: /home/software/plplot/installcmake/share/octave/site/m/PLplot
OCTAVE_OCT_DIR: /home/software/plplot/installcmake/lib/octave
TCL_DIR: /whatever1
CMake Warning at cmake/modules/summary.cmake:44 (message):
   TCL_DIR = /whatever1 has prefix inconsistency with CMAKE_INSTALL_PREFIX =
   /home/software/plplot/installcmake
Call Stack (most recent call first):
   CMakeLists.txt:277 (summary)

ADA_INCLUDE_DIR: /home/software/plplot/installcmake/share/ada/adainclude/plplotada
ADA_LIB_DIR: /home/software/plplot/installcmake/lib/ada/adalib/plplotada
LUA_DIR: /home/software/plplot/installcmake/lib/lua/5.2/plplot
OCAML_INSTALL_DIR: /home/software/plplot/installcmake/lib/ocaml

Other important CMake variables:
[....]
_______________________________________________

So the user now has complete freedom to make a mess by attempting
to specify inconsistent install location variables, but I am hoping
there won't be too many of such messes because they are warned above in the preamble of the
above section of cmake output how to avoid that situation, and they
are warned afterward (but not stopped if they are insistent on
shooting themselves in the foot) at least for the egregious case of
using inconsistent install prefixes for any of these variables as
in this example with TCL_DIR: /whatever1

The above commit was rather intrusive so I gave it a complete
noninteractive and interactive comprehensive test which passed with
only minor dashboard submission issues (see the commit message for
commit 62e07d4 for the details) which I have since fixed.

However, our comprehensive tests do not test the

make DESTDIR=<location of staging area> install

trick used by packagers.  So I would appreciate you doing that
and checking to see if this series of commits has introduced any
problems.

Also, please take a look at the documentation of each of the
cached variables above which appears in CMakeCache.txt in
the build tree, and also let me know if you would prefer any
of the above default values to change.

@Orion:

In particular, does Fedora have policy about where *.tcl files should be
installed that is consistent with Debian policy about that?

The reason I ask is if there is some consensus on that question I
would likely change the default value of TCL_DIR 
from /usr/share/plplot5.12.0/tcl to that consensus.

@Both:

Also note, that when our build system was first put together in the
autotools era, Rafael insisted on using versioned plplot names
throughout, (e.g, /usr/share/plplot5.12.0)  so I have stuck with that since.  But I
am not as keen on that as he was so if your consensus (if it exists)
for the TCL_DIR default is unversioned, e.g.,

/usr/share/tcltk/plplot

that would be fine with me.

@Ole:

You sure do ask interesting questions.  :-)

Seriously, though, keep them coming since such
questions play an important role in improving
PLplot as in these last few commits.

As far as I can remember, this current answer from me only leaves the
spelling issues reported by lint (and you) that I need to answer, but
can you recall any further topic between us that I still need to answer?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Plplot-devel mailing list
Plp...@li...
https://lists.sourceforge.net/lists/listinfo/plplot-devel

[Plplot-devel] Install location logic substantially updated

[Alan W. I. <ir...@be...>]

On 2017-07-13 09:12+0200 Ole Streicher wrote:

> Another point is that TCLDIR is currently set to
> /usr/share/plplot5.12.0/tcl, which is not in the package path for
> Tcl/Tk. Is there a way to change this to something like
> /usr/share/tcltk/plplot5.12.0 [1] with a cmake option so that I
> don't need a patch here?

To Ole and Orion:

@Both:

Ole's question (and I think Orion had similar concerns in the past)
inspired me to "open a can of worms" which was to substantially update
how we configured our install locations (see last 4 commits ending
with 4c90970).

Most of the previous relevant build system code was inspired by
autotools and has been untouched for a decade so I decided to rework a
lot of it after using a fairly elegant find .. grep stanza (see commit
message for commit 62e07d4 which is the most important commit
in this series) to determine a complete list of install
location variables (some mandatory, some optional) which are now
created and summarized consistently.

When I first implemented the original code I was quite concerned by
the user mess it would make if a user started changing install
locations inconsistently.  So the version of the code just prior
to this recent series of commits propagated that attitude and
therefore made it impossible to change many of the install-location
variables (e.g., TCL_DIR was unchangeable). But now I have decided to
allow users to change any of these variables any way they like.

So here is a recent summary (much longer than before because I found
many install location variables in our code that were not treated
consistently before) of our install location variables for the case where I
specified the -DTCL_DIR=/whatever1 cmake option.  This is taken
from the latest version of the cmake output:
_______________________________________________

Summary of CMake build system results for PLplot

Install location variables which can be set by the user.  N.B. These
variables are ordered by decreasing degree of generality, with the
_default_ values of the later ones in the list determined by the
values of variables earlier in the list.  So, for example, it is
usually sufficient in the vast majority of cases to just set
CMAKE_INSTALL_PREFIX, and the rest of these variables are adjusted
accordingly (at least for a fresh configuration), and it is rare
indeed that is is necessary for a user to set any variable here whose
name does not start with "CMAKE_INSTALL_".

CMAKE_INSTALL_PREFIX: /home/software/plplot/installcmake
CMAKE_INSTALL_EXEC_PREFIX: /home/software/plplot/installcmake
CMAKE_INSTALL_BINDIR: /home/software/plplot/installcmake/bin
CMAKE_INSTALL_DATADIR: /home/software/plplot/installcmake/share
CMAKE_INSTALL_LIBDIR: /home/software/plplot/installcmake/lib
CMAKE_INSTALL_INCLUDEDIR: /home/software/plplot/installcmake/include
CMAKE_INSTALL_INFODIR: /home/software/plplot/installcmake/share/info
CMAKE_INSTALL_MANDIR: /home/software/plplot/installcmake/share/man
CMAKE_INSTALL_PKG_CONFIG_DIR: /home/software/plplot/installcmake/lib/pkgconfig
DATA_DIR: /home/software/plplot/installcmake/share/plplot5.12.0
LIB_DIR: /home/software/plplot/installcmake/lib
INCLUDE_DIR: /home/software/plplot/installcmake/include/plplot
BIN_DIR: /home/software/plplot/installcmake/bin
DRV_DIR: /home/software/plplot/installcmake/lib/plplot5.12.0/drivers
DOC_DIR: /home/software/plplot/installcmake/share/doc/plplot
INFO_DIR: /home/software/plplot/installcmake/share/info
MAN_DIR: /home/software/plplot/installcmake/share/man
PKG_CONFIG_DIR: /home/software/plplot/installcmake/lib/pkgconfig
FORTRAN_MOD_DIR: /home/software/plplot/installcmake/lib/fortran/modules/plplot
JAR_DIR: /home/software/plplot/installcmake/share/java
JAVAWRAPPER_DIR: /home/software/plplot/installcmake/lib/jni
PYTHON_INSTDIR: /home/software/plplot/installcmake/lib/python3.4/site-packages
PLPLOT_OCTAVE_DIR: /home/software/plplot/installcmake/share/plplot_octave
OCTAVE_M_DIR: /home/software/plplot/installcmake/share/octave/site/m/PLplot
OCTAVE_OCT_DIR: /home/software/plplot/installcmake/lib/octave
TCL_DIR: /whatever1
CMake Warning at cmake/modules/summary.cmake:44 (message):
   TCL_DIR = /whatever1 has prefix inconsistency with CMAKE_INSTALL_PREFIX =
   /home/software/plplot/installcmake
Call Stack (most recent call first):
   CMakeLists.txt:277 (summary)

ADA_INCLUDE_DIR: /home/software/plplot/installcmake/share/ada/adainclude/plplotada
ADA_LIB_DIR: /home/software/plplot/installcmake/lib/ada/adalib/plplotada
LUA_DIR: /home/software/plplot/installcmake/lib/lua/5.2/plplot
OCAML_INSTALL_DIR: /home/software/plplot/installcmake/lib/ocaml

Other important CMake variables:
[....]
_______________________________________________

So the user now has complete freedom to make a mess by attempting
to specify inconsistent install location variables, but I am hoping
there won't be too many of such messes because they are warned above in the preamble of the
above section of cmake output how to avoid that situation, and they
are warned afterward (but not stopped if they are insistent on
shooting themselves in the foot) at least for the egregious case of
using inconsistent install prefixes for any of these variables as
in this example with TCL_DIR: /whatever1

The above commit was rather intrusive so I gave it a complete
noninteractive and interactive comprehensive test which passed with
only minor dashboard submission issues (see the commit message for
commit 62e07d4 for the details) which I have since fixed.

However, our comprehensive tests do not test the

make DESTDIR=<location of staging area> install

trick used by packagers.  So I would appreciate you doing that
and checking to see if this series of commits has introduced any
problems.

Also, please take a look at the documentation of each of the
cached variables above which appears in CMakeCache.txt in
the build tree, and also let me know if you would prefer any
of the above default values to change.

@Orion:

In particular, does Fedora have policy about where *.tcl files should be
installed that is consistent with Debian policy about that?

The reason I ask is if there is some consensus on that question I
would likely change the default value of TCL_DIR 
from /usr/share/plplot5.12.0/tcl to that consensus.

@Both:

Also note, that when our build system was first put together in the
autotools era, Rafael insisted on using versioned plplot names
throughout, (e.g, /usr/share/plplot5.12.0)  so I have stuck with that since.  But I
am not as keen on that as he was so if your consensus (if it exists)
for the TCL_DIR default is unversioned, e.g.,

/usr/share/tcltk/plplot

that would be fine with me.

@Ole:

You sure do ask interesting questions.  :-)

Seriously, though, keep them coming since such
questions play an important role in improving
PLplot as in these last few commits.

As far as I can remember, this current answer from me only leaves the
spelling issues reported by lint (and you) that I need to answer, but
can you recall any further topic between us that I still need to answer?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] PLplot Debian package (debian patch for toggle_plplot_use.m)

[Alan W. I. <ir...@be...>]

Hi Ole:

On 2017-07-13 09:12+0200 Ole Streicher wrote:

> Then, there is one remaining patch that may be interesting for you
> (attached). I have however no idea about it; it is a remnant from the
> past. Could you give me a hint whether it is needed for me or for you?

>From some digging with "git log" and "git blame" in the debian
subdirectory of the PLplot repository, it appears
debian/patches/proper-path-handling.diff has different metadata
and name _but the same contents_ as the patch you are concerned about.
And debian/changelog has the following comment about that
patch:

* Adjust the upstream code for working with Octave 3.6.1, by adding the
   following patches:

   + debian/patches/replace-deprecated-split.diff
   + debian/patches/proper-path-handling.diff
   + debian/patches/fix-boolean-operators.diff

Later on with the commit message "Remove obsolete debian patches." the
first and third of those three patches were removed, but the
proper-path-handling.diff patch remained as of commit
68e499b4ca^ (where commit 68e499b4ca was where I recently removed
the whole debian subdirectory).

The patch is for the bindings/octave/PLplot/toggle_plplot_use.m file
which we do not currently test in any automated way. That octave
function is apparently designed to toggle between native plot
functionality and the equivalent PLplot functionality.  Furthermore,
that function is currently designed to be used only for the installed
case where our various octave support files (*.m files) are installed in a
more rational set of directories than the build-tree case.

So I tested toggle_plplot_use.m as follows:

make -j4 install

# N.B. All tests below need to build nothing further and just refer to what
# is installed by the above make command.

# octavedir holds the appropriate octave installation directories for PLplot
# which will be used in a later octave run.  Note the installation
# prefix here, /home/software/plplot/installcmake/, will have to be
# changed if anyone else wants to follow this test procedure.

export \
octavedir="/home/software/plplot/installcmake/share/plplot_octave":"/home/software/plplot/installcmake/share/plplot_octave"/support:"/home/software/plplot/installcmake/share/octave/site/m"/PLplot:"/home/software/plplot/installcmake/lib/octave":/home/software/plplot/installcmake/share/plplot5.12.0/examples/octave/

# Test that non-PLplot (native) plotting works fine for octave.

irwin@raven> octave -f -q
octave:1> x=-pi:0.1:pi;
octave:2> plot(x,sin(x));
octave:3> exit

# Test that PLplot plotting works fine for octave (this is where we
use octavedir) with plplot_stub (which changes the plot environment to
the PLplot variant).

irwin@raven> octave -f -q
octave:1> warning("off","Octave:shadowed-function");
octave:2> addpath(getenv("octavedir"));
octave:3> plplot_stub;
octave:4> x=-pi:0.1:pi;
octave:5> plot(x,sin(x));
octave:6> exit

# Test of toggle_plplot_use to switch from PLplot to native plotting
irwin@raven> octave -f -q
octave:1> warning("off","Octave:shadowed-function");
octave:2> addpath(getenv("octavedir"));
octave:3> plplot_stub;
octave:4> toggle_plplot_use;
Use PLplot: on
octave:5> x=-pi:0.1:pi;
octave:6> plot(x,sin(x));
octave:7> toggle_plplot_use;
Use PLplot: off
octave:8> plot(x,sin(x));
octave:9> exit
error: closefig: A(I): index out of bounds; value 3 out of bound 1
error: called from:
error:   /home/software/plplot/installcmake/share/plplot_octave/closefig.m at line 40, column 5
error:   /home/software/plplot/installcmake/share/plplot_octave/closeallfig.m at line 33, column 7

The above worked (i.e., generated both a PLplot type of plot
originally but followed with a native type of plot at the end, but
generated those error messages on exit.  Furthermore,
other attempts to use toggle_plplot_use for switching back and
forth between the two plot enviroments either did not work or
ended up as segfaults.

I then modified that installed version of toggle_plplot_use.m
using the latest version (commit 68e499b4ca^) of debian/patches/proper-path-handling.diff
that exists in our repository.  That patch applied cleanly, i.e.,

software@raven> patch -p1 /home/software/plplot/installcmake/share/octave/site/m/PLplot/toggle_plplot_use.m /tmp/proper-path-handling.diff 
patching file /home/software/plplot/installcmake/share/octave/site/m/PLplot/toggle_plplot_use.m

but makes the corresponding octave toggle_plplot_use command unusable, i.e.,

octave:6> toggle_plplot_use;
error: sq_string cannot be indexed with {
error: evaluating argument list element number 1
error: called from:
error: /home/software/plplot/installcmake/share/octave/site/m/PLplot/toggle_plplot_use.m at line 42, column 3

So from these tests, unpatched toggle_plplot_use.m is problematic (it
sort of works with Octave-3.8.2, but there are issues including segfaults sometimes), but
patching it with this patch makes it not work at all!  So I suspect
when Rafael implemented this patch years ago, he never tested it, and
I guess the Debian users of this package never tried the
toggle_plplot_use command either (or gave up on it when it immediately
errored out as above).

Anyhow, Ole, if you can replicate the above tests (both without and
with the patch) using octave-4, then I suspect your decision will be
to drop this patch entirely.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] PLplot Debian package

[Alan W. I. <ir...@be...>]

Hi Ole:

On 2017-07-19 15:15+0200 Ole Streicher wrote:

> All is fine for me after disabling [1] the mentioned tests. Since UTF-8
> is one one case, I will keep the octave bindings in the 5.13.0 packaging.

I am glad that you were able to prove that non-ascii UTF-8 in the user
strings is the only Octave-4.2 issue left.

> One more small issue, which I reported some time ago to the Debian
> package [2]: The pkgconfig file specifies the need to link to libshp:
>
> $ pkg-config --libs plplotd
> -lplplotd -lltdl -lm -lshp -lcsirocsa -lcsironn -lqhull -lqsastime
>
> but in fact, -lshp is not needed. It would be nice if this linker flag
> could be removed here.
> (and, just to be sure: at that time, the library was called
> "libplotlibd", while now it is "libplotlib" without the "d", right?)

Yes on the change in library name without the suffix d.

However, the above pkg-config results are dated and come from an
era when we didn't properly distinguish the shared and static
cases.  But the above results which included -lshp were correct
for the static case although incomplete in other ways.

Here are the pkg-config results from plplot git master branch (for a
non-standard install location):

software@raven> pkg-config --libs plplot
-L/home/software/plplot/installcmake/lib -lplplot

software@raven> pkg-config --libs --static plplot
-L/home/software/plplot/installcmake/lib -L/usr/lib/x86_64-linux-gnu
-L/usr/lib/gcc/x86_64-linux-gnu/4.9 -lplplot -lltdl -lm -lshp
-lfreetype -lcsirocsa -lcsironn -lqhull -lqsastime -lstdc++ -lm -lc

So for the static case we do (correctly) add in -lshp (and the rest)
because the plplot core library depends on all those libraries (for
example -lshp allows PLplot to handle maps (e.g., for standard example 19)
using the shapelib capabilities) so they have to be specifically
mentioned when linking to the static version of libplplot.

So there should be no bug for you to deal with in this modern case
(and for 5.12.0, I believe).

The only case where -lshp should not be included in the static link
line is if the PLplot build system does not find libshp.so (or
libshp.a) in which case the PLplot core library map capability falls
back to a much less useful and therefore deprecated map format.
So that is a situation to be avoided by installing libshp whenever
possible.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] PLplot Debian package

[Ole S. <deb...@li...>]

Hi Alan,

On 18.07.2017 20:45, Alan W. Irwin wrote:
> To Ole and Orion:
> 
> Here is what I suggest you do to decide whether you want to package
> the Octave bindings of PLplot until this UTF-8 error is fixed.  The
> octave examples that are run in our tests are controlled by
> plplot_test/test_octave.sh.in. That file has the "p" list and
> standard list of examples.
> There is nothing but ascii strings in the "p" examples
> so leave that list as is.  Of the standard examples, 18, 24, 26,
> and 33 are the ones that use UTF-8 strings (I derived that list by running
> 
> grep utf-8 examples/python/x??.py
> 
> .)
> 
> So to create a pure ascii test of Octave you should exclude
> those examples from the standard examples list.
> 
> Then build the test_diff_psc target, and make your packaging decision
> based on the results of that pure-ascii test of octave.
> 
> By the way, please run that test verbosely, e.g.,
> 
> make VERBOSE=1 test_diff_psc >& test_diff_psc.out
> 
> and if that output file still shows some errors for the pure ascii
> Octave test case, please send it to me.

All is fine for me after disabling [1] the mentioned tests. Since UTF-8
is one one case, I will keep the octave bindings in the 5.13.0 packaging.

One more small issue, which I reported some time ago to the Debian
package [2]: The pkgconfig file specifies the need to link to libshp:

$ pkg-config --libs plplotd
-lplplotd -lltdl -lm -lshp -lcsirocsa -lcsironn -lqhull -lqsastime

but in fact, -lshp is not needed. It would be nice if this linker flag
could be removed here.
(and, just to be sure: at that time, the library was called
"libplotlibd", while now it is "libplotlib" without the "d", right?)

Best regards

Ole

[1]
https://anonscm.debian.org/cgit/debian-science/packages/plplot.git/tree/debian/patches/Disable-octave-UTF-8-tests.patch
[2] https://bugs.debian.org/787818

Re: [Plplot-devel] Planning for the release of 5.13.0

[Alan W. I. <ir...@be...>]

On 2017-07-18 12:36-0000 Arjen Markus wrote:

> I attach three tarballs, resulting from the comprehensive test
script on Cygwin and MinGW (interactive and non-interactive). Not all
is well with the non-interactive ones. On Cygwin it failed in the
install tree (no error message other than "there was an error"), on
MinGW in the non-dynamic case with an error message from gcc "Bad
address". Neither is something I can unravel at this moment (being
nearly off on a short holiday).

OK.  All you need to know for now was I received those tarballs and
analyzed them, and you should skip the rest, and go enjoy your
holiday.  :-)

But after your holiday you should look in detail at the results of
those analyses below.

I. The noninteractive comprehensive test for Cygwin.

* Find issues

My records indicate you last ran a comprehensive test for Cygwin on
2016-12-15.  The options for that test were the same as the present
ones, and that script was a complete success.  Those December results
for components that were present (as taken from 
shared/noninteractive/output_tree/cmake.out) were as follows:

December results:

DRIVERS_LIST:
cairo;qt;mem;ntk;null;ps;psttf;svg;tk;tkwin;wingcc;wxwidgets;xfig;xwin

Disabled components:

ENABLE_ada:             OFF
ENABLE_d:               OFF
ENABLE_java:            OFF
ENABLE_ocaml:           OFF
ENABLE_octave:          OFF
ENABLE_pdl:             OFF
ENABLE_pyqt5:           OFF

In comparison, the current results for the same quantities are as
follows:

DRIVERS_LIST:
cairo;qt;mem;ntk;null;ps;psttf;svg;tk;tkwin;wingcc;xfig;xwin

Disabled components:

ENABLE_ada:             OFF
ENABLE_d:               OFF
ENABLE_java:            OFF
ENABLE_ocaml:           OFF
ENABLE_octave:          OFF
ENABLE_pdl:             OFF
ENABLE_python:          OFF
ENABLE_pyqt4:           OFF
ENABLE_pyqt5:           OFF
ENABLE_itcl:            OFF
ENABLE_itk:             OFF
ENABLE_wxwidgets:       OFF

So the changes from December are the wxwidgets device driver is not present
and there is also a severe regression in enabled bindings.

Is this exactly the same Cygwin system and launch script that you used
before when testing just before the release of PLplot-5.12.0?  If so, you should be able to
replicate those good December results by switching to PLplot-5.12.0
(e.g., by running

git checkout plplot-5.12.0

) before running your launch script. If you can replicate those
good past find results, then obviously there are find issues that have
been introduced for the PLplot master branch version that
we have to fix for the Cygwin case.

* Build error in the CMake-based build tree for the installed examples

This did not occur for the December test which was a complete success so
it will be interesting to see if this build error was present for
the plplot-5.12.0 test I suggested above.

The following results:

irwin@raven> grep Built shared/noninteractive/output_tree/installed_make_noninteractive.out
[  0%] Built target ext-cairo-test
[  0%] Built target test_extcairo
[  1%] Built target x21c
[  1%] Built target x32c
[  1%] Built target x31c
[  2%] Built target x29c
[  2%] Built target x28c
[  4%] Built target x27c
[  4%] Built target x33c
[  4%] Built target x25c
[  5%] Built target x24c
[  7%] Built target x09c
[  8%] Built target x16c
[  8%] Built target x08c
[  9%] Built target x07c
[ 11%] Built target x15c
[ 11%] Built target x23c

show many of the C examples built without issues for this case before the build
error for x04c.c that was encountered.  And as you said, there is
a non-zero return code for that build command as indicated by

make[3]: *** [c/CMakeFiles/x04c.dir/build.make:104: c/x04c.exe] Error 1

but no accompanying diagnostic error message from gcc itself.

That lack of diagnostic error message concerns me because sometimes that
peculiar result can be a sign that there is an intermittent
hardware issue of some kind.

So to follow up, I suggest you cut and paste the exact /usr/bin/cc ... build command from that
file for example 4 to see whether you can consistently reproduce the issue
by hand and to see if any other error message is revealed that was
somehow not captured by the comprehensive test script.  And immediately
after running that /usr/bin/cc ... build command, you should test its
return code by

echo $?

If that result is non-zero, you know something was wrong with the
_immediately_ previous (in this case /usr/bin/cc ...) command even if there is no
accompanying error message.

Also note you have (by design) two different copies of the x04c.c code.
There is the original source-tree (the working directory
of your git repository) location and

irwin@raven> grep 'x04c.c$' comprehensive_test_listing.out
-rw-r--r--+ 1 markus Domain Users   4131 May 23  2016 ./shared/noninteractive/install_tree/share/plplot5.12.0/examples/c/x04c.c

You should double-check those two files are identical with each other
(in case the install location is afflicted with a disk error).

If you cannot replicate the non-zero return code by hand, then I
suggest you try running your launch script again to see if it again
consistently fails for example 4 for the CMake-based build system of
our installed examples. (And, of course, any lack of consistency from
one run to the next is likely due to a hardware issue unless your
launch script, Cygwin, or PLplot have changed between the two runs.)

II. The noninteractive comprehensive test for MinGW-w64/MSYS2

* Find issues:

DRIVERS_LIST: cairo;mem;ntk;null;pdf;ps;svg;wingcc;wxwidgets;xfig


ENABLE_ada:             OFF
ENABLE_d:               OFF
ENABLE_java:            OFF
ENABLE_ocaml:           OFF
ENABLE_octave:          OFF
ENABLE_pdl:             OFF
ENABLE_qt:              OFF
ENABLE_pyqt4:           OFF
ENABLE_pyqt5:           OFF
ENABLE_itcl:            OFF
ENABLE_itk:             OFF

The only good historical comparison we have to these results is Greg's (see
<https://sourceforge.net/p/plplot/wiki/Testing_PLplot/#Testing%20Reports>).

where ada, d, java, ocaml, octave, pyqt4, itcl, and itk were
not present in the tests for various reasons.  So the only real
regression above from those historical results was qt was disabled
(because it was not found).  So the above is a quite encouraging result
but you are not quite there yet.

To resolve that last included component regression, what Qt-related
packages (if any) have you installed?

If the answer is none, I would preserve your current MinGW-w64/MSYS2
snapshot (since a new snapshot might not work at all), modify your
automated script to install that platform from scratch to include Qt
(Qt4 is preferred but if packaging conflicts insist you need to use
Qt5, then use that instead) and run that script to install a new
snapshot with its own unique install prefix.

Then repeat this test on that new snapshot platform with
DCMAKE_CXX_FLAGS=-fabi-version=8 adjusted to a new number if that is
needed).

* ERROR: Traditional make test_noninteractive failed in the installed
examples tree (for the nondynamic case)

A google search for the term <"gcc.exe: Bad address"> didn't appear
to find anything relevant.  (There was one issue in Cygwin
for 2012 involving commands separated with a semicolon, but
we don't use a semicolon in the present case.)

As per usual when debugging such issues, you should cut and paste the
offending command from
nondynamic/noninteractive/output_tree/traditional_make_noninteractive.out
and make sure you can replicate the issue by hand.  Then follow up by
systematically debugging that hand command.  For example, it is
possible the command is too long so you might need to find another way
to, for example, read all those gcc options from a file.
Or one of those options might be in a form that is not acceptable
to the MinGW-w64/MSYS2 platform.

III. The interactive comprehensive test for MinGW-w64/MSYS2 that is
limited just to wxwidgets

All seems well here (indicating the uninitialized variable issue for
wxwidgets that is not yet fixed is not a major issue).  So after your
holiday please follow up by running the identical test again except
for updated PLplot (to get my fix for the uninitialized variable which
should be ready by then) and adding the
-DPAUSE_CERTAIN_INTERACTIVE_DEVICES=ON cmake option.  That option
means a lot more interactive work for you since you have to hit the
enter key to get through each page of each example that is displayed
and to exit each interactive example.  But it also allows you to look
at each page of the various wxwidgets results to check there are no
gross rendering issues.

Also, please extend the present test by dropping the following script
options that you are currently using:

--do_nondynamic no
--do_static no
--do_test_install_tree no
--do_test_traditional_install_tree no

When these options default to yes, instead of just testing the shared
library/build-tree case you will be testing the 9 combinations of the
three shared, non-dynamic, and static cases with the three build_tree,
install_tree, and traditional_install_tree cases.  That much wider
test coverage is quite worthwhile, but the amount of interactions you
will have to do to get through the test increases by a factor of 9 as
well.

Since I don't think rendering quality is going to be an issue for
these 9 different combinations, if the test of the above shared
library/build-tree case with -DPAUSE_CERTAIN_INTERACTIVE_DEVICES=ON
demonstrates good rendering quality, I suggest you also drop the
-DPAUSE_CERTAIN_INTERACTIVE_DEVICES=ON option (which would then
default to OFF) for this 9-times-longer test to save yourself from
excessive hitting of the enter key.  :-)

IV.  Then follow up with the noninteractive MSVC comprehensive test
and the equivalent of the 3 interactive tests above, i.e.,

* shared library/build tree -DPAUSE_CERTAIN_INTERACTIVE_DEVICES=OFF
   (by default) to demonstrate there are no major wxwidgets issues.

* shared library/build tree -DPAUSE_CERTAIN_INTERACTIVE_DEVICES=ON
   to check if there are any major rendering issues with wxwidgets.

* All 9 combinations with -DPAUSE_CERTAIN_INTERACTIVE_DEVICES=OFF
   (by default) to demonstrate no major wxwidgets issues for any
   of those combinations.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] PLplot Debian package

[Alan W. I. <ir...@be...>]

On 2017-07-18 15:32+0200 Ole Streicher wrote:

> On 14.07.2017 22:48, Orion Poplawski wrote:
>> Well, it compiles.  There are longstanding test failures though:
>>
>> 5: *** PLPLOT ERROR, ABORTING OPERATION ***
>> 5: UTF-8 string is malformed: ???M??????, aborting operation
>
> Just for the records, and to keep you informed about the Debian
> packaging process: I fixed swig in Debian (by backporting the same patch
> as Fedora), and could finally compile the octave bindings. Then I run
> into the same UTF-8 errors as Orion (with octave 4.2.1 and plplot 5.12.0).
>
> I am still unsure if this is minor (special case only), or if I should
> disable octave-plplot for 5.13 (but tend to keep it).

To Ole and Orion:

Here is what I suggest you do to decide whether you want to package
the Octave bindings of PLplot until this UTF-8 error is fixed.  The
octave examples that are run in our tests are controlled by
plplot_test/test_octave.sh.in. That file has the "p" list and
standard list of examples.
There is nothing but ascii strings in the "p" examples
so leave that list as is.  Of the standard examples, 18, 24, 26,
and 33 are the ones that use UTF-8 strings (I derived that list by running

grep utf-8 examples/python/x??.py

.)

So to create a pure ascii test of Octave you should exclude
those examples from the standard examples list.

Then build the test_diff_psc target, and make your packaging decision
based on the results of that pure-ascii test of octave.

By the way, please run that test verbosely, e.g.,

make VERBOSE=1 test_diff_psc >& test_diff_psc.out

and if that output file still shows some errors for the pure ascii
Octave test case, please send it to me.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] PLplot Debian package

[Ole S. <deb...@li...>]

On 14.07.2017 22:48, Orion Poplawski wrote:
> Well, it compiles.  There are longstanding test failures though:
> 
> 5: *** PLPLOT ERROR, ABORTING OPERATION ***
> 5: UTF-8 string is malformed: �M��, aborting operation

Just for the records, and to keep you informed about the Debian
packaging process: I fixed swig in Debian (by backporting the same patch
as Fedora), and could finally compile the octave bindings. Then I run
into the same UTF-8 errors as Orion (with octave 4.2.1 and plplot 5.12.0).

I am still unsure if this is minor (special case only), or if I should
disable octave-plplot for 5.13 (but tend to keep it).

Best regards

Ole

Re: [Plplot-devel] Planning for the release of 5.13.0

[Arjen M. <Arj...@de...>]

Hi Alan,



I attach three tarballs, resulting from the comprehensive test script on Cygwin and MinGW (interactive and non-interactive). Not all is well with the non-interactive ones. On Cygwin it failed in the install tree (no error message other than "there was an error"), on MinGW in the non-dynamic case with an error message from gcc "Bad address". Neither is something I can unravel at this moment (being nearly off on a short holiday).



Regards,



Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] First proof-of-concept versions of PLplot "memory allocation" and "exception handling" topics

[Alan W. I. <ir...@be...>]

On 2017-07-17 14:59-0700 Alan W. Irwin wrote:

> With regard to testing the above memory_allocation2, I repeated the
> same test as before (used the -DVALGRIND_ALL_TESTS=ON cmake option,
> built the test_c_psc target, and checked for any imperfect valgrind
> reports).  Examples 00 through 07 were fine, but example 08 segfaulted.
> I have attached the associated valgrind report to help you to debug
> this issue.

Oops!  This time I really do attach that valgrind log.

Alan

__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] First proof-of-concept versions of PLplot "memory allocation" and "exception handling" topics

[Alan W. I. <ir...@be...>]

On 2017-07-17 11:31+0100 Phil Rosenberg wrote:

> Hi Alan
> Please find attached a new patch series.
>
> I should have fixed the segfault issue. Note I have done some rebasing
> so you will probably need to create new branches. I think there is a
> memory leak somewhere. I will look into this.
>
> But you may have something you can play with to test a bit better now.
> I have added specific comments below.
>

Hi Phil:

Thanks for your reply with substantial updates to both branches.  To
make this process more convenient for me _next time_ would you be willing to
store your memory allocation patches in one tarball (e.g., your first
6 patches this time) and your exception handling patches (e.g., your
remaining 6 patches with the first 6 excluded) in another tarball?
This time I have already split the 12 patches into two separate
locations, see below, but that is a lot of error-prone cutting and
pasting of filenames so two tarballs next time would be much better.

[...]
> Yes, the memory management is independent of exception handling, but
> exception handling requires memory management to avoid memory leaks.

OK. See how I handled these patches below.

@Others:

This is how I applied Phil's patches.

git checkout master
# New branch
git checkout -b memory_allocation2
# Specific location where I stored the first 6 of Phil's series that had
# to do with memory allocation.  This cat trick is something I just
# discovered, and it makes it much easier to apply a series of patches
# like these.
software@raven> cat ~irwin/Phil.Rosenberg/20170717/memory_allocation/* |git am
Applying: Added functions that will record memory allocations in the PLStream
Applying: Part way through plmalloc implementation
Applying: Removed plinitializememorylist and plfreeall calls
Applying: Fix issue with growing memorylist
Applying: Added missing ;
Applying: Added Memory allocation function declarations to plplotP.h

git checkout master
# New branch
git checkout -b exception_handler2
# Apply patches in memory_allocation2 since this branch depends on
# that one being applied first.
git rebase memory_allocation2
# Specific location, etc.
software@raven> cat ~irwin/Phil.Rosenberg/20170717/exception_handler/* |git am
Applying: Added Exception handling macros/functions/variables
Applying: Add some test code for exception handling
Applying: Fixed PLENDTRY/PLTHROW bug
Applying: Remove plexit calls from advancejumpbuffer and plinitialisememorylist
Applying: Added some comments around PLTRY, PLCATCH and PLENDTRY
Applying: Free memory from jumpbuffer stack in plend1

@Phil:

Again all seems clean with applying your commits for both these topic branches.

> This [good valgrind results for memory_allocation branch] is not surprising. I Don't think the plmalloc or plrealloc
> functions are actually used anywhere yet :-D

I thought that might be the case, but these good valgrind results for
all C examples are a good benchmark, and, in fact, that good benchmark
has immediately paid off as a comparison for a bad result for thise
second version of the memory allocation topic, see below.

>
>>
>> Furthermore, I like what you have done with plmalloc, plrealloc, and
>> plfree where if inside those routines malloc, realloc, or free
>> respectively fail, plexit is called with appropriate error message.
>> Assuming, then you want to replace all our current direct calls to
>> malloc, realloc and free with these pl* variants, this solves a major
>> problem we have which was all the wide range of different treatments
>> of how malloc, realloc and free errors were checked, i.e., in the best
>> cases plexit is called, but more usually no error checking is done at
>> all!
>
> Not quite. Any pointers which we store in the plstream must use the
> original malloc, otherwise they will be freed as soon as the API call
> exits.
> There are two options here.
> 1) Rely on developers to choose malloc or plmalloc as needed. But when
> I looked through the code, it wasn't always obvious because there are
> some places where memory is allocated in one bit of code, then the
> assignment of the pointer in the plstream occurs somewhere else. I
> think this is quite messy.
> 2) Have plfreeall check that the memory allocations do not correspond
> to the pointers in plstream. This only requires that developers modify
> plfreeall if they add a new heap pointer to plstream. it may be that
> this is easy to miss as it would be done infrequently, but it would
> probably very quickly generate a segfault or valgrind would spot it
> easily.
>
> My preference is I think for 2, but I welcome comments.

I suggest instead that you create a list (which will, of course, need
to be maintained) of plstream heap pointers where a pointer to that
list is stored in plstream.  The advantage of this location for the
list is it can then be used not only in plfreeall, but also in
plmalloc, plcalloc, etc.  to do the right thing in those functions
whenever you are dealing with a plstream heap pointer.  Which, in
turn, would allow us to always use plmalloc, plcalloc, etc.,
everywhere in our C library.  Which is much less error prone (other
than the maintenance required for keeping that list updated, as you
point out above) since developers then know they must always use the
pl* versions of memory management calls while letting all those
different pl* memory allocation and freeing functions deal with
whether the heap pointers are in plstream or not.

With regard to testing the above memory_allocation2, I repeated the
same test as before (used the -DVALGRIND_ALL_TESTS=ON cmake option,
built the test_c_psc target, and checked for any imperfect valgrind
reports).  Examples 00 through 07 were fine, but example 08 segfaulted.
I have attached the associated valgrind report to help you to debug
this issue.

That finishes the memory allocation topic, and I now want to move
on to the exception handling topic.

Since this topic branch depends on the memory allocation branch, and
there is currently a showstopper segfault in that branch, I did
no tests of this exception handling topic branch other than
the one above that showed these commits applied cleanly.

However, I do have some further responses to your remarks about
this branch below.

[...]
> Hopefully some or all of this [segfault in the prior version of this branch] is fixed now. I would welcome another
> valgrind summary.

Current problem is in memory_allocation branch, see above.

>> * There is currently no ability for PLTHROW to propagate an error message.
>>   Is that a fundamental limitation of setjmp/longjmp based C exception
>>   handling or could that feature be easily implemented?  If the latter,
>>   then that gives us the option to use PLTHROW wherever errors occur
>>   rather than always calling plexit which then prints the error
>>   message before it uses PLTHROW.  So for example in plmalloc,
>>   plcalloc, plrealloc, and plfree you could use PLTHROW directly
>>   rather than calling plexit. However, if implementation of handling
>>   error messages for PLTHROW is difficult/impossible, then I would be
>>   content to call plexit everywhere so that there would only be one
>>   PLTHROW (within plexit after printing out the error message) in our
>>   code as now on this topic branch.
>
> We could throw an error code, but I don't think an error message.
> However, I think it would be better to put an error message/code in
> the plstream.

Agreed.  I personally don't like error codes that much.  It takes
me back to the 70's when I first had to deal with IBM "reason
codes"
<https://www.ibm.com/support/knowledgecenter/en/SS2L7A_5.1.0/doc/ccv-reason-codes.html>.
such as the dreaded OOC5 and OOC6.

> This would need to be on the heap rather than the stack
> otherwise it would be lost in the stack unwinding. I think we should
> maintain a function like plerror that would put the error code/message
> onto the stream, but maybe renamed to be called something like plthrow

Go for it.

[out of order]
>> * I suggest you protect your exception handling code (but not the
>>   separate topic branch code concerning pl* versions of malloc,
>>   calloc, realloc, and free) with an PL_IMPLEMENT_EXCEPTION_HANDLING
>>   macro that is controlled (using #cmakedefine in plplot_config.h.in)
>>   by a cmake option in cmake/modules/plplot.cmake of the same name
>>   which defaults to OFF and which is self-documented as highly
>>   experimental. Once you do this, then it makes it possible to merge
>>   this topic quite rapidly to master (to facilitate further
>>   collaborative development of our exception handling system) since
>>   the user would have to go out of their way to set
>>   -DPL_IMPLEMENT_EXCEPTION_HANDLING=ON once they found
>>   PL_IMPLEMENT_EXCEPTION_HANDLING in CMakeCache.txt. And that file
>>   includes the self-documentation of each option so it should be
>>   obvious to such users that turning that option ON is highly
>>   experimental.
>
> Would you be happy to add this variable to the build system for me?
> Your cmake knowledge is  much better than mine. Or do I just use that
> macro and adding of -DPL_IMPLEMENT_EXCEPTION_HANDLING=ON to the
> command line automagically generate the equivalent #define.

This is the way that we implement most of our optional C
macros so you should be able to deal with this yourself
either following the above description or following the pattern of any
of the other macros set in plplot_config.h.in.  But if you
prefer not to do that, than you could leave the CMake part of
this to me, and as a temporary measure place

#define PL_IMPLEMENT_EXCEPTION_HANDLING

in plplot.h and then do the more painstaking part of this task
which is to sprinkle

#ifdef PL_IMPLEMENT_EXCEPTION_HANDLING

and appropriate #else and #endif lines throughout wherever your
changes have added to the existing code.  The goal here, of course, is
the net result of your changes is exactly equivalent to the memory
allocation branch if PL_IMPLEMENT_EXCEPTION_HANDLING is not #defined,
and I will follow up by creating a CMake option to insure this macro
is not #defined by default unless you decide you want to do that
yourself following the above instructions.

>
>>
>> * Can you explain why you have implemented all the PLTRY blocks
>>   in src/plvpor.c (with presumably the idea in mind to use PLTRY
>>   in a lot more places?)  Wouldn't it be better to use just one
>>   PLTRY block?  For example, could you get away with just that one PLTRY
>>   block in the library initialization code? If so, the corresponding
>>   PLCATCH block there could write out a message saying you reached
>>   that catch block (to confirm that the whole exception handling system is
>>   fundamentally working within our C library), and then exit (for now)
>>   in just that one place rather than doing nothing about whatever
>>   PLplot library error was detected.
>
> I didn't check the API, but I thought that the naming convention was
> that every function beginning c_ is part of the public API. Is this
> not correct? EVERY public API function would need a PLTRY block. I
> only chose this file because it contains plenv, which was one of the
> first API functions called by x00c

You are correct, everything in c_* is in the public API.  However, I still
don't see the need for PLTRY and PLCATCH blocks for each of our public
API's.

I assume this gets into the question of how you are going to propagate
errors outside the library.  I cannot remember the details of your
previous discussion of that topic, but I was left with the impression
you were spoiled for choice.  Furthermore, if you were planning to
propagate the error outside the PLplot library (to, e.g., x00c.c) for
each of the currently empty PLCATCH blocks in src/plvpor.c, I claim
could instead be done at one central location in the plplot library
(i.e., in a PLCATCH block in the library initialization code that also
shuts down most/all of the library from that one central location).  I
think it is a no-brainer to handle library errors centrally like that
if possible since it is less intrusive and therefore much less editing
work and much less error prone, but if I am missing something (e.g.,
that is not possible with any of the propagation methods you had in
mind), then the ideal way to demonstrate that is to include an example
in your proof-of-concept of propagating a PLplot library error to
x00c.c.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________