plplot-devel — Technical discussion about PLplot development

Re: [Plplot-devel] patch 31 which contains wincairo fixes

[Hazen B. <hba...@ma...>]

On 07/06/2015 01:26 PM, Greg Jung wrote:
> I have been using the calls
> -    SetWindowLong( aStream->hwnd, GWL_USERDATA, (long) pls );
> +    SetWindowLongPtr( aStream->hwnd, GWLP_USERDATA, (LONG_PTR) pls );
>
> in both 32- and 64-bit compiled wincairo application for months, as well as
> in wingcc.  They work.  AFAICT there's
> no reason to maintain #ifdef _WIN64 conditionals as long as you go with
> these calls.
>
> I hadn't noticed the omission of <cairo-win32.h> except it now removes a
> worrisome warning I got
> in 64-bit compilation (the 32-bit comnpilation didn't need this, evidently).

Ok, I just pushed this so let me know if it causes any problems.

-Hazen

Re: [Plplot-devel] [Swig-user] swig 3.0.6 breaks plplot builds

[Alan W. I. <ir...@be...>]

On 2015-07-07 22:32-0600 Orion Poplawski wrote:

> On 07/07/2015 04:15 PM, Orion Poplawski wrote:
>> Updating from 3.0.5 to 3.0.6 in Fedora Rawhide has broken plplot builds. 
>> New
>> error:
>> 
>> cd /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python && 
>> /usr/bin/swig
>> -python -DPL_DOUBLE_INTERFACE -DSWIG_PYTHON -DPYTHON_HAVE_PYBUFFER -outdir
>> /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python
>> -I/builddir/build/BUILD/plplot-5.11.0/include
>> -I/builddir/build/BUILD/plplot-5.11.0/lib/qsastime
>> -I/builddir/build/BUILD/plplot-5.11.0/bindings/tcl
>> -I/builddir/build/BUILD/plplot-5.11.0/bindings/tk
>> -I/builddir/build/BUILD/plplot-5.11.0/fedora
>> -I/builddir/build/BUILD/plplot-5.11.0/fedora/include
>> -I/builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python -I/usr/include
>> -I/usr/include -I/usr/include -I/usr/include/python2.7
>> -I/usr/lib/python2.7/site-packages/numpy/core/include/numpy
>> -I/builddir/build/BUILD/plplot-5.11.0/bindings/swig-support -o
>> /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python/plplotcmodulePYTHON_wrap.c
>> /builddir/build/BUILD/plplot-5.11.0/bindings/python/plplotcmodule.i
>> /builddir/build/BUILD/plplot-5.11.0/bindings/swig-support/plplotcapi.i:367:
>> Error: Line indented less than expected (line 2 of pythoncode)
>> 
>> Line 367 is a function def:
>> 
>> 367> plgcolbg( PLINT *OUTPUT, PLINT *OUTPUT, PLINT *OUTPUT );
>
> turns out that this is fixed with the attached patch.  The error message was 
> very cryptic, and I'm not sure why the docstring is being checked for 
> indentation, but I have a fix.

As a PLplot developer, I do not know why swig-3.0.6 is suddenly
sensitive to the case where the documentation string for a given
function starts with a blank (which is what Orion's patch for PLplot
fixed).  That sounds like a swig regression to me.  Nevertheless, to
work around that issue (or to provide a fix if that was an intended
swig change) I have now updated the script that generates that file
for PLplot master branch tip (currently PLplot commit id = 273ec0d) to
produce the equivalent of Orion's patch.  (This way of fixing the
issue not only provides a workaround now, but also in the
future just in case an extra blank is inserted by accident when
editing the PLplot doc/docbook/src/api.xml file which is the source of
the specially-generated file that Orion patched.)

The two remaining issues discussed by Orion below appear to be swig
issues alone since the PLplot octave binding relies strictly on swig to
include the appropriate Octave headers into
plplot_octaveOCTAVE_wrap.cxx.

>
> On to the next error though:
>
> /usr/lib64/ccache/c++   -DPLPLOT_HAVE_CONFIG_H -Dplplot_octave_EXPORTS -O2 -g 
> -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions 
> -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches 
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic  -fPIC 
> -I/builddir/build/BUILD/plplot-5.11.0/include 
> -I/builddir/build/BUILD/plplot-5.11.0/lib/qsastime 
> -I/builddir/build/BUILD/plplot-5.11.0/fedora 
> -I/builddir/build/BUILD/plplot-5.11.0/fedora/include 
> -I/builddir/build/BUILD/plplot-5.11.0/fedora/bindings/octave 
> -I/usr/include/octave-4.0.0 -I/usr/include/octave-4.0.0/octave 
> -I/builddir/build/BUILD/plplot-5.11.0/bindings/swig-support    -o 
> CMakeFiles/plplot_octave.dir/plplot_octaveOCTAVE_wrap.cxx.o -c 
> /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/octave/plplot_octaveOCTAVE_wrap.cxx
> /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/octave/plplot_octaveOCTAVE_wrap.cxx:1638:18: 
> error: 'hid_t' has not been declared
>       save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats) {
>                  ^
>
> #if defined (HAVE_HDF5)
>    virtual bool
>      save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats) {
>      return true;
>    }
>
> This now appears to be caused by the update to octave 4.0.0.  With 3.8.2:
>
> /usr/include/octave-3.8.2/octave/ov.h:#include "oct-hdf5.h"
> /usr/include/octave-3.8.2/octave/oct-hdf5.h:#include <hdf5.h>
>
> so ov.h included <hdf5.h>.  But now:
>
> /usr/include/octave-4.0.0/octave/oct-hdf5.h:#include <hdf5.h>
> /usr/include/octave-4.0.0/octave/ls-hdf5.h:#include "oct-hdf5.h"
>
> only ls-hdf5.h or oct-hdf5.h will bring in <hdf5.h>.
>
> Also, octave's definition of save_hdf5 has changed from:
>
> bool save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats)
>
> to
>
> bool save_hdf5 (octave_hdf5_id loc_id, const char *name, bool 
> save_as_floats);
>
> So it looks like Lib/octave/octrun.swg needs to get updated to handle that?
>
> -- 
> Orion Poplawski
> Technical Manager                     303-415-9701 x222
> NWRA/CoRA Division                    FAX: 303-415-9702
> 3380 Mitchell Lane                  or...@co...
> Boulder, CO 80301              http://www.cora.nwra.com
>

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] [Swig-user] swig 3.0.6 breaks plplot builds

[Orion P. <or...@co...>]

On 07/07/2015 04:15 PM, Orion Poplawski wrote:
> Updating from 3.0.5 to 3.0.6 in Fedora Rawhide has broken plplot builds.  New
> error:
>
> cd /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python && /usr/bin/swig
> -python -DPL_DOUBLE_INTERFACE -DSWIG_PYTHON -DPYTHON_HAVE_PYBUFFER -outdir
> /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python
> -I/builddir/build/BUILD/plplot-5.11.0/include
> -I/builddir/build/BUILD/plplot-5.11.0/lib/qsastime
> -I/builddir/build/BUILD/plplot-5.11.0/bindings/tcl
> -I/builddir/build/BUILD/plplot-5.11.0/bindings/tk
> -I/builddir/build/BUILD/plplot-5.11.0/fedora
> -I/builddir/build/BUILD/plplot-5.11.0/fedora/include
> -I/builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python -I/usr/include
> -I/usr/include -I/usr/include -I/usr/include/python2.7
> -I/usr/lib/python2.7/site-packages/numpy/core/include/numpy
> -I/builddir/build/BUILD/plplot-5.11.0/bindings/swig-support -o
> /builddir/build/BUILD/plplot-5.11.0/fedora/bindings/python/plplotcmodulePYTHON_wrap.c
> /builddir/build/BUILD/plplot-5.11.0/bindings/python/plplotcmodule.i
> /builddir/build/BUILD/plplot-5.11.0/bindings/swig-support/plplotcapi.i:367:
> Error: Line indented less than expected (line 2 of pythoncode)
>
> Line 367 is a function def:
>
> 367> plgcolbg( PLINT *OUTPUT, PLINT *OUTPUT, PLINT *OUTPUT );

turns out that this is fixed with the attached patch.  The error message 
was very cryptic, and I'm not sure why the docstring is being checked 
for indentation, but I have a fix.

On to the next error though:

/usr/lib64/ccache/c++   -DPLPLOT_HAVE_CONFIG_H -Dplplot_octave_EXPORTS 
-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 
-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 
-grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 
-m64 -mtune=generic  -fPIC -I/builddir/build/BUILD/plplot-5.11.0/include 
-I/builddir/build/BUILD/plplot-5.11.0/lib/qsastime 
-I/builddir/build/BUILD/plplot-5.11.0/fedora 
-I/builddir/build/BUILD/plplot-5.11.0/fedora/include 
-I/builddir/build/BUILD/plplot-5.11.0/fedora/bindings/octave 
-I/usr/include/octave-4.0.0 -I/usr/include/octave-4.0.0/octave 
-I/builddir/build/BUILD/plplot-5.11.0/bindings/swig-support    -o 
CMakeFiles/plplot_octave.dir/plplot_octaveOCTAVE_wrap.cxx.o -c 
/builddir/build/BUILD/plplot-5.11.0/fedora/bindings/octave/plplot_octaveOCTAVE_wrap.cxx
/builddir/build/BUILD/plplot-5.11.0/fedora/bindings/octave/plplot_octaveOCTAVE_wrap.cxx:1638:18: 
error: 'hid_t' has not been declared
        save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats) {
                   ^

#if defined (HAVE_HDF5)
     virtual bool
       save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats) {
       return true;
     }

This now appears to be caused by the update to octave 4.0.0.  With 3.8.2:

/usr/include/octave-3.8.2/octave/ov.h:#include "oct-hdf5.h"
/usr/include/octave-3.8.2/octave/oct-hdf5.h:#include <hdf5.h>

so ov.h included <hdf5.h>.  But now:

/usr/include/octave-4.0.0/octave/oct-hdf5.h:#include <hdf5.h>
/usr/include/octave-4.0.0/octave/ls-hdf5.h:#include "oct-hdf5.h"

only ls-hdf5.h or oct-hdf5.h will bring in <hdf5.h>.

Also, octave's definition of save_hdf5 has changed from:

bool save_hdf5 (hid_t loc_id, const char *name, bool save_as_floats)

to

bool save_hdf5 (octave_hdf5_id loc_id, const char *name, bool 
save_as_floats);

So it looks like Lib/octave/octrun.swg needs to get updated to handle that?

-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  or...@co...
Boulder, CO 80301              http://www.cora.nwra.com

[Plplot-devel] Comprehensive testing of wxwidgets has been reinstated

[Alan W. I. <ir...@be...>]

See the commit message for e7b8f63 for my reasons for this change.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Alan W. I. <ir...@be...>]

On 2015-07-07 05:53-0000 Arjen Markus wrote:

> Well, that was spot on :). See the attached tarball. The test script completed without any complaints and the comparisons are clean.

Hi Arjen:

I agree with your conclusions about the the clean nature of these
results for the largest number of PLplot components tested on Cygwin
so far.  My thanks for your essential help in achieving this high
point for Cygwin!

That said, your report does indicate there is still one noninteractive issue
that we should deal with for this release cycle, and my apologies for
missing it until now. Your latest result uses

cmake_added_options=-DENABLE_octave=OFF -DENABLE_ada=OFF

to workaround some build system issues on the Cygwin platform.
-DENABLE_octave=OFF is something you adopted on June 3rd after installing the
final Octave-related Cygwin package that was required.  Your comment
back then was

-        Allowing Octave gave Cmake errors about two versus four
arguments. I turned that off.

As far as I know we never followed up further on this build-system
issue for configuring the Octave binding of PLplot, but I would like
to do that now.

So when you get a change could you run the script again
without -DENABLE_octave=OFF so I can diagnose (and hopefully fix) this
issue?

I believe this issue will be the last noninteractive one I will
attempt to deal with for the Cygwin platform in this release cycle.
Of course, you also use -DENABLE_ada=OFF to workaround the well-known
Ada language support issue on Cygwin, but I expect you will be
continuing with that workaround indefinitely because it is likely I
will only be able to deal with this issue some time after the planned
5.11.1 release.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Arjen M. <Arj...@de...>]

Hi Alan,



Well, that was spot on :). See the attached tarball. The test script completed without any complaints and the comparisons are clean.



Regards,



Arjen



> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
>
> The net result of this is I am positive that for best visibility results on all platforms
>
> extern PLDLLIMPORT char * plplotLibDir;
>
> should be replaced by
>
> extern PLDLLIMPEXP_TCLTK_DATA( char * ) plplotLibDir;
>
> I have done that change (as of commit ae0e9da) and comprehensively tested it on
> Linux with no visibility issues showing up for those tests.
>
> So please try running scripts/comprehensive_test.sh again to see if ae0e9da solves
> this final noninteractive issue completely for Cygwin (which would be a new high in
> Cygwin comprehensive testing).
>
> Alan


DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Alan W. I. <ir...@be...>]

On 2015-07-06 10:43-0700 Alan W. Irwin wrote:

> On 2015-07-06 16:11-0000 Arjen Markus wrote:
>
>> Alas, the error I reported persists, even now that /usr/local plays
> no role anymore in the build. See the attached tarball.
>
> Hi Arjen:
>
> I was happy to see that the Cygwin itk fix I implemented a while back
> actually works for you and that all the /usr/local stuff is no longer
> contaminating the result in your report.  Your elimination of the
> possibility of such contamination will be a big help in finding the
> source of the remaining issue you are reporting.
>
> With regard to that issue, I still think it is some visibility problem
> since the code that defines the missing symbol should be part of
> the PLplot library for the nondynamic case. So could you check both
> the definition and visibility of that missing symbol in the PLplot
> library for the nondynamic case using the nm method I suggested
> before?  i.e.,
>
> # change directory to the nondynamic build tree
> cd nondynamic/build_tree/
>
> # Check for symbol definition in PLplot library
> nm --defined-only <plplot library location> |grep plplotLibDir
>
> # Check for symbol visibility in PLplot library
> nm --defined-only --extern-only <plplot library location> |grep plplotLibDir
>
> Thanks in advance.

Hi Arjen:

I followed up by doing my own comprehensive test on Linux, and I
cannot replicate what I anticipate will be your report of visibility
issues from the experiments above.  For example, for the nondynamic
case I get

software@raven> nm --defined-only --extern-only src/libplplot.so |grep plplotLibDir
00000000002cdd10 B plplotLibDir

i.e., plplotLibDir is both defined and visible in the PLplot library
built on the Linux platform for the nondynamic case.

However, I then looked further at where plplotLibDir was defined, and
I found the following line that does that in

bindings/tcl/tclAPI.c

extern PLDLLIMPORT char * plplotLibDir;


But that is a non-standard use of PLDLLIMPORT.  In fact, the only other
place I can find it in our code base (other than in include/pldll.h.in) is in

software@raven> find . -type f |grep -v .git |xargs grep PLDLLIMPORT |grep -v pldll.h
./bindings/tcl/tclAPI.c:extern PLDLLIMPORT char * plplotLibDir;
./include/plplotP.h:// extern PLStream PLDLLIMPORT *plsc;

and in that latter case that commented out code was replaced by

extern PLDLLIMPEXP_DATA( PLStream * ) plsc;

The net result of this is I am positive that for best visibility
results on all platforms

extern PLDLLIMPORT char * plplotLibDir;

should be replaced by

extern PLDLLIMPEXP_TCLTK_DATA( char * ) plplotLibDir;

I have done that change (as of commit ae0e9da) and comprehensively tested it
on Linux with no visibility issues showing up for those
tests.

So please try running scripts/comprehensive_test.sh again to see if ae0e9da solves
this final noninteractive issue completely for Cygwin (which would be a
new high in Cygwin comprehensive testing).

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Alan W. I. <ir...@be...>]

On 2015-07-06 16:11-0000 Arjen Markus wrote:

> Alas, the error I reported persists, even now that /usr/local plays
no role anymore in the build. See the attached tarball.

Hi Arjen:

I was happy to see that the Cygwin itk fix I implemented a while back
actually works for you and that all the /usr/local stuff is no longer
contaminating the result in your report.  Your elimination of the
possibility of such contamination will be a big help in finding the
source of the remaining issue you are reporting.

With regard to that issue, I still think it is some visibility problem
since the code that defines the missing symbol should be part of
the PLplot library for the nondynamic case. So could you check both
the definition and visibility of that missing symbol in the PLplot
library for the nondynamic case using the nm method I suggested
before?  i.e.,

# change directory to the nondynamic build tree
cd nondynamic/build_tree/

# Check for symbol definition in PLplot library
nm --defined-only <plplot library location> |grep plplotLibDir

# Check for symbol visibility in PLplot library
nm --defined-only --extern-only <plplot library location> |grep plplotLibDir

Thanks in advance.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] patch 31 which contains wincairo fixes

[Greg J. <gv...@gm...>]

I have been using the calls
-    SetWindowLong( aStream->hwnd, GWL_USERDATA, (long) pls );
+    SetWindowLongPtr( aStream->hwnd, GWLP_USERDATA, (LONG_PTR) pls );

in both 32- and 64-bit compiled wincairo application for months, as well as
in wingcc.  They work.  AFAICT there's
no reason to maintain #ifdef _WIN64 conditionals as long as you go with
these calls.

I hadn't noticed the omission of <cairo-win32.h> except it now removes a
worrisome warning I got
in 64-bit compilation (the 32-bit comnpilation didn't need this, evidently).


On Sun, Jul 5, 2015 at 3:46 AM, Hazen Babcock <hba...@ma...> wrote:

> On 07/04/2015 12:15 PM, Alan W. Irwin wrote:
> > Hi Hazen:
> >
> > A PLplot user has created a patch containing wincairo fixes (see
> > <http://sourceforge.net/p/plplot/patches/31/>).  As the originator of
> > the wincairo device could you please evaluate this 3-line change to
> > see whether we should apply it now to master so it will be included in
> > the planned 5.11.1 bugfix release?
>
> Hi Alan,
>
> I think these changes are the right thing to do based on reading up on
> Microsoft's documentation and I would just go ahead and apply it.
> However since I no longer have a Windows development box I can't test
> it. The best test would be to make sure that it doesn't break 32 bit
> Windows.
>
> -Hazen
>
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Plplot-devel mailing list
> Plp...@li...
> https://lists.sourceforge.net/lists/listinfo/plplot-devel
>

Re: [Plplot-devel] Status report on remaining issues to be addressed for the forthcoming 5.11.1 release (wxwidgets issues)

[Alan W. I. <ir...@be...>]

On 2015-07-06 09:52+0100 Phil Rosenberg wrote:

> Hi Alan - this is now slightly out of context as I found it in my
> drafts folder unsent.
> I can confirm similar odd timing results on my Centos machine. I did
> have a couple of exit() calls in wxPLViewer, which to be honest was
> just lazy of me. On was used to close the app when -np was used. I
> have now replaced these both with clean exit methods. this ensures all
> destructors are called and I can now say that at least as far as I can
> tell on Windows the shared memory is _always_ released. It is
> certainly released when the window is closed using -np. This is on the
> viewer side. I presume the use of -np still gives a clean exit on the
> console side and doesn't invoke any exit calls anywhere?
>
> Unfortunately the odd timings still persist. Note that the application
> only asks for 1 MB of shared memory so even if not freed I wouldn't
> have thought it would slow anything down really after just one run.
>
> I haven't got time right now to set up a decent debugging environment
> on my Centos machine, but I will try to double check the shared memory
> is released on that system as soon as I can.

Hi Phil:

Thanks for confirming those odd time results for second and subsequent
runs of the wxwidgets examples on CentOS, and I hope that you and
Andrew also try the same experiment on more modern Linux platforms as
well just to check whether the issue is due to an issue with mmapped
shared memory for older Linux kernels (such as those for CentOS and
Debian Wheezy) that has been fixed for modern Linux kernels (say those
used for Debian testing and unstable and for the latest Ubuntu).

Another question to consider is does this odd timing issue occur
immediately for a fresh reboot or does the platform have to accumulate
a relatively long uptime with lots of wxPLViewer use before the odd
timing results appear? In the former case, the issue is unlikely to be
due to some costly emergency response to some mmap resource exhaustion
and will therefore be a lot easier to track down (assuming modern
Linux kernels show the issue).

For the case where freshly booted modern Linux kernels show the
problem and normal debug methods cannot find the source of it pretty
quickly, then I think it is time to implement a really simple mmapped
shared memory example following your present IPC method.  The client
should loop through nmax different requests for double precision
values from a server and sum those values.  The server should respond
to those requests by delivering 0., 1., 2., etc. The client can
compare the computed sum of those numbers with the known
nmax*(nmax-1)/2 sum to double-check the server is delivering the
values properly.

With such a simple mmapped shared memory example implemented (with
nmax set at a large enough value so the example takes a second or so)
then you can test for odd timing issues like above.  If the simple
example demonstrates that issue for a freshly booted modern Linux
kernel, then the simple example will be a big help when preparing a
bug report for the Linux kernel development team. (Note, I should
probably be the one to present that bug report since I have fairly
good contact with a RedHat employee that works every day on the Linux
kernel, and from experience several years ago with a Linux kernel
issue I was having back then he is very accommodating about running
simple tests for the very latest Linux kernel.)

Also, implementation of such a simple example of mmapped shared memory
IPC should allow you to debug any excessive wait times in your IPC
method that should never occur for such a flood of requests for double
precision values from the client.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Arjen M. <Arj...@de...>]

Hi Alan,



Alas, the error I reported persists, even now that /usr/local plays no role anymore in the build. See the attached tarball.

(To get Itk to work I had to install Iwidgets, but once that was done, it was accepted by CMake.)


> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Sunday, July 05, 2015 10:48 PM
> To: Arjen Markus
> Cc: plp...@li...
> Subject: RE: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk
>
> Hi Arjen:
>
> Thanks for that report.
>
> You said:
>
>
> Please run another noninteractive test with the above issues completely addressed
> as soon as is convenient.  You may find the undefined-symbol issue you discovered
> for this test completely disappears for that further test because there is no longer any
> chance for inconsistency between pure Cygwin packages and /usr/local results you
> have built yourself.
>
> Alan


DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Arjen M. <Arj...@de...>]

Hi Alan,



> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Sunday, July 05, 2015 10:48 PM
> To: Arjen Markus
> Cc: plp...@li...
> Subject: RE: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk
>
> Hi Arjen:
>
> Thanks for that report.
>
> You said:
>
> > I have attached the result of the comprehensive_test.sh script on
> Cygwin. This time I had an X server running, so that Tk is properly recognised as a
> valid option.
>
> Good.  I hope this part of the X setup is put into your script that sets up and runs
> scripts/comprehensive_test.sh so this issue will not come back to haunt us again.
>
> > The "shared" build succeeds, but the
> "nondynamic" build fails:
>
> There are some lingering but important Cygwin setup issues that you should address
> just as soon as that is convenient for you.
>
> I. As I mentioned before you are running scripts/comprehensive_test.sh with
>
> --cmake_added_options "-DENABLE_octave=OFF -DENABLE_ada=OFF -
> DENABLE_itk=OFF"
>
> Please drop that last option, i.e., use
>
> --cmake_added_options "-DENABLE_octave=OFF -DENABLE_ada=OFF"
>
> instead to test my fix for the (default) -DENABLE_itk=ON case that I implemented
> some time ago.
>
> II. As I mentioned before these tests should be based strictly on Cygwin packages
> rather than something you built and installed in /usr/local at some unknown time in
> the past.  And given a choice, CMake will always prefer to use the /usr/local version.
> So I strongly suggest you completely move /usr/local to somewhere else for the time
> being to make sure you have a pure Cygwin result.
>
Oh dear, I had hidden the Tcl libraries, but not the header files. Will redo that test now that these are also hidden.

> Here are environment variables that are set that involve usr/local.
>
> irwin@raven> grep usr/local comprehensive_test_env.out PATH=/lib/python2.7/site-
> packages:/usr/local/bin:/usr/bin:/cygdrive/c/ProgramData/Oracle/Java/javapath:/cygd
> rive/d/CAF/bin:/cygdrive/c/GTK/bin:/cygdrive/c/tcl/bin:/cygdrive/c/windows/system32:
> /usr/lib/lapack
> INFOPATH=/usr/local/info:/usr/share/info:/usr/info
>
> These environmental variable results (which I assume are setup by default on
> Cygwin) show that /usr/local/bin is on the PATH and /usr/local/info is on the
> INFOPATH. That will be fine once you do the above suggested move of /usr/local to
> somewhere else, but with your present setup with /usr/local containing old results
> you have compiled yourself, these settings for PATH and INFOPATH do
> contaminate your results as seen below.
>
> Here are the CMake results concerning usr/local.
>
> irwin@raven> grep usr/local shared/output_tree/cmake.out
> -- TCL_INCLUDE_PATH = /usr/local/include
> -- FindShapelib: Found shapelib header directory, /usr/local/include, and library,
> /usr/local/lib/libshp.a.
> -- tk_COMPILE_FLAGS = -I"/usr/local/include" -I"/usr/include" -I"/usr/include" -
> I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tcl -I"/cygdrive/d/plplot-
> svn/comprehensive_test_disposeable/shared/build_tree"/bindings/tcl -
> I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tk
> -- ntk_COMPILE_FLAGS = -I"/usr/local/include" -I"/usr/include" -I"/usr/include"
> -- tkwin_COMPILE_FLAGS = -I"/usr/local/include" -I"/usr/include" -I"/usr/include" -
> I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tcl -I"/cygdrive/d/plplot-
> svn/comprehensive_test_disposeable/shared/build_tree"/bindings/tcl -
> I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tk-x-plat -I"/cygdrive/d/plplot-svn/plplot-
> git"/bindings/tk
> libplplot_LINK_FLAGS =
> /usr/lib/libltdl.dll.a;/usr/lib/libdl.a;/usr/local/lib/libshp.a;/usr/lib/libfreetype.dll.a;-
> lcsirocsa;-lcsironn;-lqhull;-lqsastime
>
> I assume all those /usr/local results being found by CMake will be replaced by the
> correct pure Cygwin results once you do the suggested renaming of /usr/local.  But,
> of course, you should check shared/output_tree/cmake.out (generated very early in
> the test) to make sure there are no warnings about missing components of PLplot
> that correspond to a Cygwin package (e.g., the development package for
> shapelib) that is not installed yet.
>
> Please run another noninteractive test with the above issues completely addressed
> as soon as is convenient.  You may find the undefined-symbol issue you discovered
> for this test completely disappears for that further test because there is no longer any
> chance for inconsistency between pure Cygwin packages and /usr/local results you
> have built yourself.
>
Hm, it seems that the shapelib library was installed under /usr/local, as well as the LibLASi static library. I did not compile them myself.

I have renamed the original /usr/local and created an empty one now, let us see what this brings.

Regards,

Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Status report on remaining issues to be addressed for the forthcoming 5.11.1 release (wxwidgets issues)

[Phil R. <p.d...@gm...>]

Hi Alan - this is now slightly out of context as I found it in my
drafts folder unsent.
I can confirm similar odd timing results on my Centos machine. I did
have a couple of exit() calls in wxPLViewer, which to be honest was
just lazy of me. On was used to close the app when -np was used. I
have now replaced these both with clean exit methods. this ensures all
destructors are called and I can now say that at least as far as I can
tell on Windows the shared memory is _always_ released. It is
certainly released when the window is closed using -np. This is on the
viewer side. I presume the use of -np still gives a clean exit on the
console side and doesn't invoke any exit calls anywhere?

Unfortunately the odd timings still persist. Note that the application
only asks for 1 MB of shared memory so even if not freed I wouldn't
have thought it would slow anything down really after just one run.

I haven't got time right now to set up a decent debugging environment
on my Centos machine, but I will try to double check the shared memory
is released on that system as soon as I can.

Phil

On 3 July 2015 at 20:32, Alan W. Irwin <ir...@be...> wrote:
> To Phil and Andrew:
>
> @Andrew: I am addressing you directly (in addition to Phil) because I
> have questions for you below concerning whether you can replicate my
> weird timing results for wxwidgets and also concerning the issue of
> shared memory leaks on Linux.
>
>
> On 2015-07-03 13:53+0100 Phil Rosenberg wrote:
>
>> Hi Alan
>> So I think you should now find the wxPLViewer speed has significantly
>> improved. The item I have dealt with now was that there is a timer at
>> the OS level (initiated by wxWidgets) which sends a message to the
>> viewer which initiates checking for new commands from the console app.
>> At this point I used to check for a single message and return after
>> dealing with it. However it seems that for whatever reason (different
>> OS, different wxWidgets version?) on some systems the minimum time
>> between the timer calls was quite long. This meant that when we were
>> dealing with text which required a lot of small transmissions to the
>> viewer there was a huge overhead. I have now modified things so that
>> on every timer event I check for new transmissions at least 100 times
>> with a 1 millisecond sleep between each. This means that for many
>> small transmissions in a short time we remove most of the overhead.
>>
>> On my CentOS machine the execution time reported by time x08c -dev
>> wxWidgets -np has dropped from 40s to about 2 s. Note that this only
>> times the console part, not the viewer, but still we have clearly
>> moved to something I would call acceptable.
>>
>> Note that there will always be a significant overhead for the
>> transmission so this will never compete with Cairo, but this is
>> unavoidable because wxWidgets cannot be run in a library without
>> running it as a separate thread and PlPlot isn't thread safe. Note
>> also that when using the wxWidgets driver via the wxWidgets binding
>> (I.e. from within a wxWidgets app) this all becomes irrelevant as the
>> viewer is not used.
>>
>> Anyway I hope this fix closes the issue and you are happy Alan.
>
>
> @Phil:
>
> The timing results for current master tip (commit id 5e74b6a6,
> "Modification to previous wxPLViewer optimisation") are much improved
> on first execution of examples.  So big congratulations on that result!
>
> However, an important issue still remains as shown by the following
> repeated timing results for both xcairo (as a typical benchmark) and
> wxwidgets:
>
> software@raven> time examples/c/x00c -dev xcairo -np
>
> real    0m0.168s
> user    0m0.008s
> sys     0m0.016s
> software@raven> time examples/c/x00c -dev xcairo -np
>
> real    0m0.129s
> user    0m0.020s
> sys     0m0.004s
> software@raven> time examples/c/x00c -dev xcairo -np
>
> real    0m0.114s
> user    0m0.020s
> sys     0m0.008s
>
> software@raven> time examples/c/x00c -dev wxwidgets -np
>
> real    0m0.365s
> user    0m0.008s
> sys     0m0.012s
> software@raven> time examples/c/x00c -dev wxwidgets -np
>
> real    0m1.231s
> user    0m0.008s
> sys     0m0.012s
> software@raven> time examples/c/x00c -dev wxwidgets -np
>
> real    0m5.587s
> user    0m0.008s
> sys     0m0.012s
>
> So the xcairo case has subsequent time results that are up to 1.5
> times faster (in real time which is the most important component) then
> the first execution of the example while the wxwidgets case has
> subsequent time results that are up to 15 (!) times slower.  The
> initial fast wxwidgers result seems guaranteed if you wait a minute or
> so since your last attempt to use wxwidgets.  I mostly tested the
> above wxwidgets time results with the -np option (for convenience),
> but I also tried the test a few times without -np and got the fast,
> slow, slow,... pattern for that case as well.  Also, I usually get a
> consistent fast, slow, slow,.... pattern for wxwidgets, but just now I
> tried it again, and I got the fast result for something like 10 times
> in a row, then the slow result. So the wxwidgets inconsistency in time
> results is sometimes inconsistent. :-)
>
> The above results for the xcairo case are quite typical of all
> non-wxwidgets cases I have ever investigated before, where real time
> results are significantly shorter on second and subsequent
> execution because of the well-known effect on Linux of the system
> caching all reasonably small files in memory to improve small file
> access times for subsequent use.
>
> The very unusual much longer times that often but not always
> immediately appear on second and subsequent executions that occur for
> the wxwidgets case above strongly suggest to me a system resource is
> not being properly released by the first execution of the example so
> getting that resource back again often takes a lot of extra time on
> the second and subsequent runs.
>
> I brought up this issue with you off list much earlier this week, but
> you did not respond at that time, but I hope you do that now.  The
> first step in your response should be to try the above bash commands
> for at least the wxwidgets case on your Windows and Linux platforms.
> to (a) see if this issue also occurs on Windows, and (b) see if this
> issue occurs on all Linux systems accessible to you.
>
> @Andrew: could you please try the above time tests as well
> on your Linux platforms?
>
> @Both:
>
> If the issue is a Linux resource which I am chronically short of
> because of my prior wxwidgets testing and a shared memory leak (see
> discussion below) in wxwidgets, then you might not be able to
> replicate the issue on Linux until you do sufficient wxwidgets
> testing.  My Linux up time is currently 42 days, and if I reboot
> (which I won't do lightly because there are two users on the system
> and it is a bit of a pain for us to reset all desktops the way we like
> them), I might find the issue goes away for a while.
>
> @Phil:
>
> If the issue is a Linux-only one, then an obvious candidate for a
> system resource grabber is the shm_open calls in
> drivers/wxwidgets_comms.cpp that helps to allow shared memory access
> between the wxwidgets device driver and wxPLViewer.  I have just now
> read the shm_overview and shm_open/shm_unlink man pages, and it
> appears that shm_unlink must be called by _both_ the wxwidgets device
> driver and wxPLViewer to properly release the shared memory.  So a
> code review for those two cases to make sure that _always_ happens
> (i.e., there is no shared memory leak) is indicated.
>
> @Andrew: will you comment further here please about whether we should
> be concerned about possible shared memory leaks?  I now realize, for
> example, that both Phil's and my calling the IPC method that is used
> "shared memory" is likely not specific enough because it does not
> distinguish the old-fashioned shmget method (which we do not use) with
> the modern mmap-based method that we do use.  See
> <http://stackoverflow.com/questions/5656530/how-to-use-shared-memory-with-linux-in-c>
> for comments on this important "shared-memory" distinction. Furthermore,
> from the discussion at
> <http://stackoverflow.com/questions/22691621/how-to-avoid-shared-memory-leaks>
> it appears the mmap-based method cannot actually leak memory.  Is that
> your interpretation of those remarks as well?
>
> @Phil:
>
> One thing I noticed from reading those man pages is it is suggested
> the name used for shm_open should always start with a "/" for the most
> portable results.  It appears our current wxwidgets code does not
> follow that suggestion so this may affect Unix platforms that are not
> Linux.  So I suggest you will want to address that issue (even though
> it has nothing to do with timing.
>
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

Re: [Plplot-devel] Status report on remaining issues to be addressed for the forthcoming 5.11.1 release (wxwidgets issues)

[Phil R. <p.d...@gm...>]

Hi Alan - this is now slightly out of context as I found it in my
drafts folder unsent.
I can confirm similar odd timing results on my Centos machine. I did
have a couple of exit() calls in wxPLViewer, which to be honest was
just lazy of me. On was used to close the app when -np was used. I
have now replaced these both with clean exit methods. this ensures all
destructors are called and I can now say that at least as far as I can
tell on Windows the shared memory is _always_ released. It is
certainly released when the window is closed using -np. This is on the
viewer side. I presume the use of -np still gives a clean exit on the
console side and doesn't invoke any exit calls anywhere?

Unfortunately the odd timings still persist. Note that the application
only asks for 1 MB of shared memory so even if not freed I wouldn't
have thought it would slow anything down really after just one run.

I haven't got time right now to set up a decent debugging environment
on my Centos machine, but I will try to double check the shared memory
is released on that system as soon as I can.

Phil

On 3 July 2015 at 20:32, Alan W. Irwin <ir...@be...> wrote:
> To Phil and Andrew:
>
> @Andrew: I am addressing you directly (in addition to Phil) because I
> have questions for you below concerning whether you can replicate my
> weird timing results for wxwidgets and also concerning the issue of
> shared memory leaks on Linux.
>
>
> On 2015-07-03 13:53+0100 Phil Rosenberg wrote:
>
>> Hi Alan
>> So I think you should now find the wxPLViewer speed has significantly
>> improved. The item I have dealt with now was that there is a timer at
>> the OS level (initiated by wxWidgets) which sends a message to the
>> viewer which initiates checking for new commands from the console app.
>> At this point I used to check for a single message and return after
>> dealing with it. However it seems that for whatever reason (different
>> OS, different wxWidgets version?) on some systems the minimum time
>> between the timer calls was quite long. This meant that when we were
>> dealing with text which required a lot of small transmissions to the
>> viewer there was a huge overhead. I have now modified things so that
>> on every timer event I check for new transmissions at least 100 times
>> with a 1 millisecond sleep between each. This means that for many
>> small transmissions in a short time we remove most of the overhead.
>>
>> On my CentOS machine the execution time reported by time x08c -dev
>> wxWidgets -np has dropped from 40s to about 2 s. Note that this only
>> times the console part, not the viewer, but still we have clearly
>> moved to something I would call acceptable.
>>
>> Note that there will always be a significant overhead for the
>> transmission so this will never compete with Cairo, but this is
>> unavoidable because wxWidgets cannot be run in a library without
>> running it as a separate thread and PlPlot isn't thread safe. Note
>> also that when using the wxWidgets driver via the wxWidgets binding
>> (I.e. from within a wxWidgets app) this all becomes irrelevant as the
>> viewer is not used.
>>
>> Anyway I hope this fix closes the issue and you are happy Alan.
>
>
> @Phil:
>
> The timing results for current master tip (commit id 5e74b6a6,
> "Modification to previous wxPLViewer optimisation") are much improved
> on first execution of examples.  So big congratulations on that result!
>
> However, an important issue still remains as shown by the following
> repeated timing results for both xcairo (as a typical benchmark) and
> wxwidgets:
>
> software@raven> time examples/c/x00c -dev xcairo -np
>
> real    0m0.168s
> user    0m0.008s
> sys     0m0.016s
> software@raven> time examples/c/x00c -dev xcairo -np
>
> real    0m0.129s
> user    0m0.020s
> sys     0m0.004s
> software@raven> time examples/c/x00c -dev xcairo -np
>
> real    0m0.114s
> user    0m0.020s
> sys     0m0.008s
>
> software@raven> time examples/c/x00c -dev wxwidgets -np
>
> real    0m0.365s
> user    0m0.008s
> sys     0m0.012s
> software@raven> time examples/c/x00c -dev wxwidgets -np
>
> real    0m1.231s
> user    0m0.008s
> sys     0m0.012s
> software@raven> time examples/c/x00c -dev wxwidgets -np
>
> real    0m5.587s
> user    0m0.008s
> sys     0m0.012s
>
> So the xcairo case has subsequent time results that are up to 1.5
> times faster (in real time which is the most important component) then
> the first execution of the example while the wxwidgets case has
> subsequent time results that are up to 15 (!) times slower.  The
> initial fast wxwidgers result seems guaranteed if you wait a minute or
> so since your last attempt to use wxwidgets.  I mostly tested the
> above wxwidgets time results with the -np option (for convenience),
> but I also tried the test a few times without -np and got the fast,
> slow, slow,... pattern for that case as well.  Also, I usually get a
> consistent fast, slow, slow,.... pattern for wxwidgets, but just now I
> tried it again, and I got the fast result for something like 10 times
> in a row, then the slow result. So the wxwidgets inconsistency in time
> results is sometimes inconsistent. :-)
>
> The above results for the xcairo case are quite typical of all
> non-wxwidgets cases I have ever investigated before, where real time
> results are significantly shorter on second and subsequent
> execution because of the well-known effect on Linux of the system
> caching all reasonably small files in memory to improve small file
> access times for subsequent use.
>
> The very unusual much longer times that often but not always
> immediately appear on second and subsequent executions that occur for
> the wxwidgets case above strongly suggest to me a system resource is
> not being properly released by the first execution of the example so
> getting that resource back again often takes a lot of extra time on
> the second and subsequent runs.
>
> I brought up this issue with you off list much earlier this week, but
> you did not respond at that time, but I hope you do that now.  The
> first step in your response should be to try the above bash commands
> for at least the wxwidgets case on your Windows and Linux platforms.
> to (a) see if this issue also occurs on Windows, and (b) see if this
> issue occurs on all Linux systems accessible to you.
>
> @Andrew: could you please try the above time tests as well
> on your Linux platforms?
>
> @Both:
>
> If the issue is a Linux resource which I am chronically short of
> because of my prior wxwidgets testing and a shared memory leak (see
> discussion below) in wxwidgets, then you might not be able to
> replicate the issue on Linux until you do sufficient wxwidgets
> testing.  My Linux up time is currently 42 days, and if I reboot
> (which I won't do lightly because there are two users on the system
> and it is a bit of a pain for us to reset all desktops the way we like
> them), I might find the issue goes away for a while.
>
> @Phil:
>
> If the issue is a Linux-only one, then an obvious candidate for a
> system resource grabber is the shm_open calls in
> drivers/wxwidgets_comms.cpp that helps to allow shared memory access
> between the wxwidgets device driver and wxPLViewer.  I have just now
> read the shm_overview and shm_open/shm_unlink man pages, and it
> appears that shm_unlink must be called by _both_ the wxwidgets device
> driver and wxPLViewer to properly release the shared memory.  So a
> code review for those two cases to make sure that _always_ happens
> (i.e., there is no shared memory leak) is indicated.
>
> @Andrew: will you comment further here please about whether we should
> be concerned about possible shared memory leaks?  I now realize, for
> example, that both Phil's and my calling the IPC method that is used
> "shared memory" is likely not specific enough because it does not
> distinguish the old-fashioned shmget method (which we do not use) with
> the modern mmap-based method that we do use.  See
> <http://stackoverflow.com/questions/5656530/how-to-use-shared-memory-with-linux-in-c>
> for comments on this important "shared-memory" distinction. Furthermore,
> from the discussion at
> <http://stackoverflow.com/questions/22691621/how-to-avoid-shared-memory-leaks>
> it appears the mmap-based method cannot actually leak memory.  Is that
> your interpretation of those remarks as well?
>
> @Phil:
>
> One thing I noticed from reading those man pages is it is suggested
> the name used for shm_open should always start with a "/" for the most
> portable results.  It appears our current wxwidgets code does not
> follow that suggestion so this may affect Unix platforms that are not
> Linux.  So I suggest you will want to address that issue (even though
> it has nothing to do with timing.
>
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

Re: [Plplot-devel] Status report on remaining issues to be addressed fortheforthcoming 5.11.1 release (wxwidgets issues)

[Phil R. <p.d...@gm...>]

Hi Alan
Regarding the rendering, this is buffered. Hence the bit that actually displays to screen is a blt, which is close to instant - it should take significantly less than one screen refresh. Well that would be the case if using an accelerated method. With gdi, gdi+ or gtk which wxWidgets use it may take a bit longer.

So in that sense the viewer is doing what it should I think. I will look at leaving the previous plot visible, but I think it will be low on my list of priorities at the moment

Phil

-----Original Message-----
From: "Alan W. Irwin" <ir...@be...>
Sent: ‎05/‎07/‎2015 19:48
To: "Phil Rosenberg" <p.d...@gm...>
Cc: "PLplot development list" <Plp...@li...>
Subject: RE: Status report on remaining issues to be addressed fortheforthcoming 5.11.1 release (wxwidgets issues)

On 2015-07-05 09:25+0100 Phil Rosenberg wrote:

> Hi Alan

> Could describe exactly what the -np option should do and also add
that to the documentation. At the moment for wxWidgets as soon as a
page is rendered we move to the next page, which as you state causes
at most a brief flicker of a plot. But this is exactly what the
documentation states it should do.

Hi Phil:

The -np option simply means "no pause" for human interaction
(such as waiting for a human to hit the "enter" key to move from one page to the next or
to terminate the plot if the last page is being displayed).

Let me ask you a question in return.  What general method is used by
wxPLViewer to actually render a page to the screen?  (Note the
question concerns the rendering part alone and not the steps leading
up to the start of that rendering.) Is that general rendering process
expected to be extremely short?

For example, I just checked

examples/c/x08c -dev wxwidgets -bg 0000FF

(note without the -np option), and for each page the associated
wxPLViewer gives you a black screen for a while, and then an
"instantaneous" render of the entire page including the blue PLplot
background specified by the -bg 0000FF option above (as opposed to the
default black PLplot background to keep the PLplot background
distinguisable from the black background of the wxPLViewer GUI). Such
instantaneous rendering results are fine, but I am curious about the
general method used to do that instanteous rendering.

I have also done some experiments with

examples/c/x08c -dev xwin -bg 0000FF

The initial render of a page is pretty fast but still slow enough so
you can see it happen. But on resizes (which I believe uses the same
general method as wxPLViewer) it appears to be instantanous despite
the complexities of that plot (which is fine).

If you confirm the rendering part should be essentially instantaneous
for wxPLViewer, then it appears the wxwidgets device is handling the
-np option correctly.

However, I strongly suggest a change so the -np option produces more
meaningful looking results on multipage examples which is to allow the
previous page to be displayed on the screen until the present page is
ready to be rendered.  So with this model, wxPLViewer would initialize
the GUI (which presumably would create the black background), process
a page to collect all the data needed for rendering, render the page,
terminate the page, then continue with processing the next page
without reinitializing the GUI so you would never see the black "GUI"
background after the first page was rendered.

This change although extremely useful for multipage examples would still
leave single-page examples rendered as a brief flicker, but I don't
think that can be helped unless we were willing to put pauses in when
PLplot exited which I would think would be a bad idea.

> Regarding the warnings, I am not sure what to do. How long do you
think is sensible to wait for a response from the viewer? This is a
balance, because if someone kills the viewer or it crashes then the
console will be waiting for a response that never comes. I will look
at tuning this, but suggestions welcome.

Instead of treating symptoms here I would try and cure the disease
which is the rather long times that the combination of wxPLViewer and
-dev wxwidgets are currently waiting for each other.  For example, if
you cure why the second and subsubsequent runs of an example can take
up to 16 times longer than the first run of an example, that cure
might have implications for runs of single examples, i.e., their wait
times might be substantially reduced so the above warning issue might
just go away.

> I will look into x17 and x26.

Thanks, and I look forward to your conclusions concerning those
examples.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Alan W. I. <ir...@be...>]

Hi Arjen:

Thanks for that report.

You said:

> I have attached the result of the comprehensive_test.sh script on
Cygwin. This time I had an X server running, so that Tk is properly
recognised as a valid option.

Good.  I hope this part of the X setup is put into your script that
sets up and runs scripts/comprehensive_test.sh so this issue will not
come back to haunt us again.

> The "shared" build succeeds, but the
"nondynamic" build fails:

There are some lingering but important Cygwin setup issues that you
should address just as soon as that is convenient for you.

I. As I mentioned before you are running scripts/comprehensive_test.sh
with

--cmake_added_options "-DENABLE_octave=OFF -DENABLE_ada=OFF -DENABLE_itk=OFF"

Please drop that last option, i.e., use

--cmake_added_options "-DENABLE_octave=OFF -DENABLE_ada=OFF"

instead to test my fix for the (default) -DENABLE_itk=ON case that I implemented
some time ago.

II. As I mentioned before these tests should be based strictly on
Cygwin packages rather than something you built and installed in
/usr/local at some unknown time in the past.  And given a choice,
CMake will always prefer to use the /usr/local version.  So I strongly
suggest you completely move /usr/local to somewhere else for the time
being to make sure you have a pure Cygwin result.

Here are environment variables that are set that involve usr/local.

irwin@raven> grep usr/local comprehensive_test_env.out 
PATH=/lib/python2.7/site-packages:/usr/local/bin:/usr/bin:/cygdrive/c/ProgramData/Oracle/Java/javapath:/cygdrive/d/CAF/bin:/cygdrive/c/GTK/bin:/cygdrive/c/tcl/bin:/cygdrive/c/windows/system32:/usr/lib/lapack
INFOPATH=/usr/local/info:/usr/share/info:/usr/info

These environmental variable results (which I assume are setup by
default on Cygwin) show that /usr/local/bin is on the PATH and
/usr/local/info is on the INFOPATH. That will be fine once you do the
above suggested move of /usr/local to somewhere else, but with your
present setup with /usr/local containing old results you have compiled
yourself, these settings for PATH and INFOPATH do contaminate your
results as seen below.

Here are the CMake results concerning usr/local.

irwin@raven> grep usr/local shared/output_tree/cmake.out
-- TCL_INCLUDE_PATH = /usr/local/include
-- FindShapelib: Found shapelib header directory, /usr/local/include, and library, /usr/local/lib/libshp.a.
-- tk_COMPILE_FLAGS = -I"/usr/local/include" -I"/usr/include" -I"/usr/include" -I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tcl -I"/cygdrive/d/plplot-svn/comprehensive_test_disposeable/shared/build_tree"/bindings/tcl -I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tk
-- ntk_COMPILE_FLAGS = -I"/usr/local/include" -I"/usr/include" -I"/usr/include"
-- tkwin_COMPILE_FLAGS = -I"/usr/local/include" -I"/usr/include" -I"/usr/include" -I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tcl -I"/cygdrive/d/plplot-svn/comprehensive_test_disposeable/shared/build_tree"/bindings/tcl -I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tk-x-plat -I"/cygdrive/d/plplot-svn/plplot-git"/bindings/tk
libplplot_LINK_FLAGS = /usr/lib/libltdl.dll.a;/usr/lib/libdl.a;/usr/local/lib/libshp.a;/usr/lib/libfreetype.dll.a;-lcsirocsa;-lcsironn;-lqhull;-lqsastime

I assume all those /usr/local results being found by CMake will be
replaced by the correct pure Cygwin results once you do the suggested
renaming of /usr/local.  But, of course, you should check
shared/output_tree/cmake.out (generated very early in the test) to
make sure there are no warnings about missing components of PLplot
that correspond to a Cygwin package (e.g., the development package for
shapelib) that is not installed yet.

Please run another noninteractive test with the above issues
completely addressed as soon as is convenient.  You may find the
undefined-symbol issue you discovered for this test completely
disappears for that further test because there is no longer any chance
for inconsistency between pure Cygwin packages and /usr/local results
you have built yourself.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Status report on remaining issues to be addressed for theforthcoming 5.11.1 release (wxwidgets issues)

[Alan W. I. <ir...@be...>]

On 2015-07-05 09:25+0100 Phil Rosenberg wrote:

> Hi Alan

> Could describe exactly what the -np option should do and also add
that to the documentation. At the moment for wxWidgets as soon as a
page is rendered we move to the next page, which as you state causes
at most a brief flicker of a plot. But this is exactly what the
documentation states it should do.

Hi Phil:

The -np option simply means "no pause" for human interaction
(such as waiting for a human to hit the "enter" key to move from one page to the next or
to terminate the plot if the last page is being displayed).

Let me ask you a question in return.  What general method is used by
wxPLViewer to actually render a page to the screen?  (Note the
question concerns the rendering part alone and not the steps leading
up to the start of that rendering.) Is that general rendering process
expected to be extremely short?

For example, I just checked

examples/c/x08c -dev wxwidgets -bg 0000FF

(note without the -np option), and for each page the associated
wxPLViewer gives you a black screen for a while, and then an
"instantaneous" render of the entire page including the blue PLplot
background specified by the -bg 0000FF option above (as opposed to the
default black PLplot background to keep the PLplot background
distinguisable from the black background of the wxPLViewer GUI). Such
instantaneous rendering results are fine, but I am curious about the
general method used to do that instanteous rendering.

I have also done some experiments with

examples/c/x08c -dev xwin -bg 0000FF

The initial render of a page is pretty fast but still slow enough so
you can see it happen. But on resizes (which I believe uses the same
general method as wxPLViewer) it appears to be instantanous despite
the complexities of that plot (which is fine).

If you confirm the rendering part should be essentially instantaneous
for wxPLViewer, then it appears the wxwidgets device is handling the
-np option correctly.

However, I strongly suggest a change so the -np option produces more
meaningful looking results on multipage examples which is to allow the
previous page to be displayed on the screen until the present page is
ready to be rendered.  So with this model, wxPLViewer would initialize
the GUI (which presumably would create the black background), process
a page to collect all the data needed for rendering, render the page,
terminate the page, then continue with processing the next page
without reinitializing the GUI so you would never see the black "GUI"
background after the first page was rendered.

This change although extremely useful for multipage examples would still
leave single-page examples rendered as a brief flicker, but I don't
think that can be helped unless we were willing to put pauses in when
PLplot exited which I would think would be a bad idea.

> Regarding the warnings, I am not sure what to do. How long do you
think is sensible to wait for a response from the viewer? This is a
balance, because if someone kills the viewer or it crashes then the
console will be waiting for a response that never comes. I will look
at tuning this, but suggestions welcome.

Instead of treating symptoms here I would try and cure the disease
which is the rather long times that the combination of wxPLViewer and
-dev wxwidgets are currently waiting for each other.  For example, if
you cure why the second and subsubsequent runs of an example can take
up to 16 times longer than the first run of an example, that cure
might have implications for runs of single examples, i.e., their wait
times might be substantially reduced so the above warning issue might
just go away.

> I will look into x17 and x26.

Thanks, and I look forward to your conclusions concerning those
examples.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Arjen M. <Arj...@de...>]

Hi Alan,



I had a quick look at the source code: only the Tcl bindings use this global variable plplotLibDir to communicate the location of the library. If we use a small "setter" function instead, then the complications of exporting/importing a variable from the PLplot library is not needed anymore. Something like:



void pl_set_library_dir( char *dir ) {

    plplotLibDir = plstrdup(dir);

}

Regards,

Arjen

From: Arjen Markus [mailto:Arj...@de...]
Sent: Sunday, July 05, 2015 3:57 PM
To: Alan W. Irwin
Cc: plp...@li...
Subject: Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk


Hi Alan,



I have attached the result of the comprehensive_test.sh script on Cygwin. This time I had an X server running, so that Tk is properly recognised as a valid option. The "shared" build succeeds, but the "nondynamic" build fails:



CMakeFiles/plplot.dir/__/bindings/tcl/tclAPI.c.o:tclAPI.c:(.text+0xacc): undefined reference to `__imp_plplotLibDir'

CMakeFiles/plplot.dir/__/bindings/tcl/tclAPI.c.o:tclAPI.c:(.text+0xacc): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp_plplotLibDir'

collect2: error: ld returned 1 exit status



I guess this is due to confusion between static and dynamic linking - plplotLibDir has the attribute PLDLLIMPEXP. I have not tried to analyse the precise cause.



Regards,



Arjen


Arjen Markus
Sr. Adviseur/Onderzoeker
T

+31(0)88335 8559

E

arj...@de...




[Logo]<http://www.deltares.com/>

 www.deltares.com<http://www.deltares.com/>

Postbus 177
2600 MH Delft

[Deltares Twitter]
<http://www.twitter.com/deltares>

[Deltares LinkedIn]<http://www.linkedin.com/company/217430>

[Deltares Facebook]<https://www.facebook.com/pages/Deltares/154189334634001>



[Think before printing]Please consider the environment before printing this email


> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Wednesday, June 17, 2015 9:09 PM
> To: Arjen Markus
> Cc: plp...@li...
> Subject: RE: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk and
> Ada
>
> On 2015-06-17 10:39-0000 Arjen Markus wrote:
>
> > Hi Alan,
> >
> >
> >
> > Hm, something is failing in the non_dynamic tests:
> >
> > CMakeFiles/pltcl.dir/pltcl.c.o:pltcl.c:(.text+0x20e): undefined reference to
> `__imp_Pltcl_Init'
> >
> > CMakeFiles/pltcl.dir/pltcl.c.o:pltcl.c:(.text+0x20e): relocation truncated to fit:
> R_X86_64_PC32 against undefined symbol `__imp_Pltcl_Init'
> >
> > So your latest changes regarding the reduction of the llibraries for the non-dynamic
> drivers seem to have introduced a bug. I have attached the complete tarball.
>
> Hi Arjen:
>
> Thanks for this test.
>
> I. Comments on your setup of this test.
>
> For some reason there was a regression in your X setup so that you are now
> encountering that DISPLAY issue again.  The message (in
> cmake.out) was
>
> Application initialization failed: no display name and no $DISPLAY environment
> variable
>
> I encourage you to automate your X setup to avoid this regression in behaviour in
> future.
>
> Also please drop the -DENABLE_itk=OFF workaround which should not be
> necessary if you have all Tcl/Tk/Itcl/Itk/Iwidgets packages properly installed on
> Cygwin.  And if those are not properly installed, please fix that.
>
> Also, in retrospect it was a good idea for you to be cautious and test first with --
> do_test_interactive no.  But once the current issue is fixed, I will encourage you to
> drop that restriction so that all the Tcl, etc., interactive results can also be run-time
> tested.
>
> II. Comments on the bug you found.
>
> The link command that is failing above is (after line wrapping at each white space to
> help clarify the command and redacting the many irrelevant
> parts)
>
> /usr/bin/cc
> -Wl,--enable-auto-import
> CMakeFiles/pltcl.dir/pltcl.c.o
> -o
> pltcl.exe
> -Wl,--out-implib,libpltcl.dll.a
> -Wl,--major-image-version,0,--minor-image-version,0
> ../../dll/libplplottcltk_Main.dll.a
> /usr/lib/itcl3.4/libitcl3.4.dll.a
> -ltcl
> ../../dll/libplplot.dll.a
> [...]
> /usr/local/lib/libLASi.dll.a
> [...]
> /usr/local/lib/libshp.a
> [...]
>
> II A.  The /usr/local issue.
>
> Sorry I did not notice this issue before.  My assumption is that anything in /usr/local
> cannot be due to an official Cygwin install so should be avoided (at least for now).
> So to follow up on this I looked for "local" in cmake.out and found the following:
>
> -- TCL_INCLUDE_PATH = /usr/local/include
>
> -- FindShapelib: Found shapelib header directory, /usr/local/include, and library,
> /usr/local/lib/libshp.a.
>
> libplplot_LINK_FLAGS =
> /usr/lib/libltdl.dll.a;/usr/lib/libdl.a;/usr/local/lib/libshp.a;/usr/lib/libfreetype.dll.a;-
> lcsirocsa;-lcsironn;-lqhull;-lqsastime
>
> So please do the appropriate /usr/local renaming (perhaps of all of
> /usr/local) to force CMake to find the official Cygwin versions of Tcl, shapelib, and
> libLASi which I presume you have long-since already installed.  The result should be
> no mention of /usr/local in cmake.out the CMake cache files or any of your build
> commands.
>
> I suspect all those /usr/local results are leftovers from an epa_build attempt you did
> long ago.  At some point you might want to try epa_build again (but with an install
> prefix other that /usr/local so the results don't necessarily take CMake precedence
> over all others), but only after you make sure all the official Cygwin packages work.
>
> II B. The linking issue with pltcl that you discovered above.
>
> The above link command for pltcl shows that libplplot is properly linked.  So one
> possibility to explain this is either Pltcl_Init is not defined in the plplot library or it is
> defined in that library but not visible.  To check that, please use the nm command
> (which likely is already installed, but if not you should be able to find it in
> binutils-2.25-2) as follows:
>
> # Change to _nondynamic_ build directory
> cd ../comprehensive_test_disposeable/nondynamic/build_tree/
> # Build plplot library (which was cleaned by the script) in the # nondynamic case
> make plplot # Test for definition of Pltcl_Init in that library nm --defined-only
> <library_name> |grep Pltcl_Init
>
> where <library_name> is the appropriate name of the plplot library you just built.
> Here that is src/libplplot.so, but I am aware there are two locations (in src and in dll)
> of the plplot library for Cygwin, and I don't know which one you should be using with
> the nm command so you will have to experiment to find which to use with nm.
>
> The result of the above command here is
>
> 000000000008f120 T Pltcl_Init
>
> which indicates that symbol is defined in the plplot library and you should be getting
> something similar there.
>
> Once you have proved that symbol is defined in the plplot library the next issue is
> whether that symbol is externally visible.  "nm" tests symbol visibility with the --
> extern-only option, e.g.,
>
> nm --extern-only --defined-only <library_name> |grep Pltcl_Init
>
> and if Pltcl_Init is externally visible you should get the same result whether --extern-
> only is used or not.  And that proved to be the case here when I compiled with
>
> CXXFLAGS=-O3 -fvisibility=hidden -Wuninitialized
> CFLAGS=-O3 -fvisibility=hidden -Wuninitialized
>
> which should make gcc act very similarly on Linux to the way it handles visibility
> issues for C or C++ library on Windows.
>
> So from my good visibility results here, I am pretty sure visibility should not be an
> issue for you there, but please get back to me with the Pltcl_Init definition and
> visibility results via the "nm" command for the plplot library for the nondynamic case
> to confirm that (or not), and we can take it from there.
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state implementation for
> stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net);
> PLplot scientific plotting software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure
> Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________
>
DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.
DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk

[Arjen M. <Arj...@de...>]

Hi Alan,



I have attached the result of the comprehensive_test.sh script on Cygwin. This time I had an X server running, so that Tk is properly recognised as a valid option. The "shared" build succeeds, but the "nondynamic" build fails:



CMakeFiles/plplot.dir/__/bindings/tcl/tclAPI.c.o:tclAPI.c:(.text+0xacc): undefined reference to `__imp_plplotLibDir'

CMakeFiles/plplot.dir/__/bindings/tcl/tclAPI.c.o:tclAPI.c:(.text+0xacc): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp_plplotLibDir'

collect2: error: ld returned 1 exit status



I guess this is due to confusion between static and dynamic linking - plplotLibDir has the attribute PLDLLIMPEXP. I have not tried to analyse the precise cause.



Regards,



Arjen


Arjen Markus
Sr. Adviseur/Onderzoeker
T

+31(0)88335 8559

E

arj...@de...




[Logo]<http://www.deltares.com/>

 www.deltares.com<http://www.deltares.com/>

Postbus 177
2600 MH Delft

[Deltares Twitter]
<http://www.twitter.com/deltares>

[Deltares LinkedIn]<http://www.linkedin.com/company/217430>

[Deltares Facebook]<https://www.facebook.com/pages/Deltares/154189334634001>



[Think before printing]Please consider the environment before printing this email


> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Wednesday, June 17, 2015 9:09 PM
> To: Arjen Markus
> Cc: plp...@li...
> Subject: RE: [Plplot-devel] Comprehensive testing on Cygwin with Tcl/Tk/Itcl/Itk and
> Ada
>
> On 2015-06-17 10:39-0000 Arjen Markus wrote:
>
> > Hi Alan,
> >
> >
> >
> > Hm, something is failing in the non_dynamic tests:
> >
> > CMakeFiles/pltcl.dir/pltcl.c.o:pltcl.c:(.text+0x20e): undefined reference to
> `__imp_Pltcl_Init'
> >
> > CMakeFiles/pltcl.dir/pltcl.c.o:pltcl.c:(.text+0x20e): relocation truncated to fit:
> R_X86_64_PC32 against undefined symbol `__imp_Pltcl_Init'
> >
> > So your latest changes regarding the reduction of the llibraries for the non-dynamic
> drivers seem to have introduced a bug. I have attached the complete tarball.
>
> Hi Arjen:
>
> Thanks for this test.
>
> I. Comments on your setup of this test.
>
> For some reason there was a regression in your X setup so that you are now
> encountering that DISPLAY issue again.  The message (in
> cmake.out) was
>
> Application initialization failed: no display name and no $DISPLAY environment
> variable
>
> I encourage you to automate your X setup to avoid this regression in behaviour in
> future.
>
> Also please drop the -DENABLE_itk=OFF workaround which should not be
> necessary if you have all Tcl/Tk/Itcl/Itk/Iwidgets packages properly installed on
> Cygwin.  And if those are not properly installed, please fix that.
>
> Also, in retrospect it was a good idea for you to be cautious and test first with --
> do_test_interactive no.  But once the current issue is fixed, I will encourage you to
> drop that restriction so that all the Tcl, etc., interactive results can also be run-time
> tested.
>
> II. Comments on the bug you found.
>
> The link command that is failing above is (after line wrapping at each white space to
> help clarify the command and redacting the many irrelevant
> parts)
>
> /usr/bin/cc
> -Wl,--enable-auto-import
> CMakeFiles/pltcl.dir/pltcl.c.o
> -o
> pltcl.exe
> -Wl,--out-implib,libpltcl.dll.a
> -Wl,--major-image-version,0,--minor-image-version,0
> ../../dll/libplplottcltk_Main.dll.a
> /usr/lib/itcl3.4/libitcl3.4.dll.a
> -ltcl
> ../../dll/libplplot.dll.a
> [...]
> /usr/local/lib/libLASi.dll.a
> [...]
> /usr/local/lib/libshp.a
> [...]
>
> II A.  The /usr/local issue.
>
> Sorry I did not notice this issue before.  My assumption is that anything in /usr/local
> cannot be due to an official Cygwin install so should be avoided (at least for now).
> So to follow up on this I looked for "local" in cmake.out and found the following:
>
> -- TCL_INCLUDE_PATH = /usr/local/include
>
> -- FindShapelib: Found shapelib header directory, /usr/local/include, and library,
> /usr/local/lib/libshp.a.
>
> libplplot_LINK_FLAGS =
> /usr/lib/libltdl.dll.a;/usr/lib/libdl.a;/usr/local/lib/libshp.a;/usr/lib/libfreetype.dll.a;-
> lcsirocsa;-lcsironn;-lqhull;-lqsastime
>
> So please do the appropriate /usr/local renaming (perhaps of all of
> /usr/local) to force CMake to find the official Cygwin versions of Tcl, shapelib, and
> libLASi which I presume you have long-since already installed.  The result should be
> no mention of /usr/local in cmake.out the CMake cache files or any of your build
> commands.
>
> I suspect all those /usr/local results are leftovers from an epa_build attempt you did
> long ago.  At some point you might want to try epa_build again (but with an install
> prefix other that /usr/local so the results don't necessarily take CMake precedence
> over all others), but only after you make sure all the official Cygwin packages work.
>
> II B. The linking issue with pltcl that you discovered above.
>
> The above link command for pltcl shows that libplplot is properly linked.  So one
> possibility to explain this is either Pltcl_Init is not defined in the plplot library or it is
> defined in that library but not visible.  To check that, please use the nm command
> (which likely is already installed, but if not you should be able to find it in
> binutils-2.25-2) as follows:
>
> # Change to _nondynamic_ build directory
> cd ../comprehensive_test_disposeable/nondynamic/build_tree/
> # Build plplot library (which was cleaned by the script) in the # nondynamic case
> make plplot # Test for definition of Pltcl_Init in that library nm --defined-only
> <library_name> |grep Pltcl_Init
>
> where <library_name> is the appropriate name of the plplot library you just built.
> Here that is src/libplplot.so, but I am aware there are two locations (in src and in dll)
> of the plplot library for Cygwin, and I don't know which one you should be using with
> the nm command so you will have to experiment to find which to use with nm.
>
> The result of the above command here is
>
> 000000000008f120 T Pltcl_Init
>
> which indicates that symbol is defined in the plplot library and you should be getting
> something similar there.
>
> Once you have proved that symbol is defined in the plplot library the next issue is
> whether that symbol is externally visible.  "nm" tests symbol visibility with the --
> extern-only option, e.g.,
>
> nm --extern-only --defined-only <library_name> |grep Pltcl_Init
>
> and if Pltcl_Init is externally visible you should get the same result whether --extern-
> only is used or not.  And that proved to be the case here when I compiled with
>
> CXXFLAGS=-O3 -fvisibility=hidden -Wuninitialized
> CFLAGS=-O3 -fvisibility=hidden -Wuninitialized
>
> which should make gcc act very similarly on Linux to the way it handles visibility
> issues for C or C++ library on Windows.
>
> So from my good visibility results here, I am pretty sure visibility should not be an
> issue for you there, but please get back to me with the Pltcl_Init definition and
> visibility results via the "nm" command for the plplot library for the nondynamic case
> to confirm that (or not), and we can take it from there.
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state implementation for
> stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net);
> PLplot scientific plotting software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure
> Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________
>

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] make test fails on cygwin

[Arjen M. <Arj...@de...>]

Hi Chris,



Under "bare" Windows you need to expand the PATH variable in the same way. It has to do with the way Windows works. If you run the comprehensive tests, then the PATH variable is manipulated to make it point to the right build (several build configurations are tested by the comprehensive_test.sh script).



Regards,



Arjen



> -----Original Message-----
> From: Chris Marshall [mailto:dev...@gm...]
> Sent: Sunday, July 05, 2015 3:25 PM
> To: plp...@li...
> Subject: [Plplot-devel] make test fails on cygwin
>
> Running:
>
>    cmake -DBUILD_TEST=ON ../
>    make
>    make test
>
> all tests fail.  When I add the dll/ directory to the path by hand then everything
> appears to be running ok.  I'll let you know how it proceeds.
>
> Is there a reason that 'make test' on cygwin requires a hand fix?
> As far as I know this is an issue for windows and cygwin dlls.
> How is the problem handled with windows?
>
> Cheers,
> Chris
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that you need to
> offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Plplot-devel mailing list
> Plp...@li...
> https://lists.sourceforge.net/lists/listinfo/plplot-devel

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] make test fails on cygwin

[Chris M. <dev...@gm...>]

I'm happy to report:

  make test
Running tests...
Test project /home/chm/pdl/git/plplot/cygbuild
       Start  1: examples_c
  1/20 Test  #1: examples_c ....................... Passed   26.68 sec
       Start  2: examples_cxx
  2/20 Test  #2: examples_cxx ..................... Passed   27.62 sec
       Start  3: examples_f95
  3/20 Test  #3: examples_f95 ..................... Passed   17.01 sec
       Start  4: examples_python
  4/20 Test  #4: examples_python .................. Passed   39.27 sec
       Start  5: examples_tcl
  5/20 Test  #5: examples_tcl ..................... Passed   32.31 sec
       Start  6: examples_lua
  6/20 Test  #6: examples_lua ..................... Passed   19.29 sec
       Start  7: examples_psttfc
  7/20 Test  #7: examples_psttfc .................. Passed   33.39 sec
       Start  8: examples_svg
  8/20 Test  #8: examples_svg ..................... Passed   91.28 sec
       Start  9: examples_pscairo
  9/20 Test  #9: examples_pscairo ................. Passed   57.67 sec
       Start 10: examples_pngcairo
10/20 Test #10: examples_pngcairo ................ Passed   24.25 sec
       Start 11: examples_xfig
11/20 Test #11: examples_xfig .................... Passed   17.21 sec
       Start 12: examples_bmpqt
12/20 Test #12: examples_bmpqt ................... Passed   46.06 sec
       Start 13: examples_jpgqt
13/20 Test #13: examples_jpgqt ................... Passed   45.02 sec
       Start 14: examples_pngqt
14/20 Test #14: examples_pngqt ................... Passed   46.44 sec
       Start 15: examples_ppmqt
15/20 Test #15: examples_ppmqt ................... Passed   45.22 sec
       Start 16: examples_tiffqt
16/20 Test #16: examples_tiffqt .................. Passed   47.26 sec
       Start 17: examples_svgqt
17/20 Test #17: examples_svgqt ................... Passed   81.65 sec
       Start 18: examples_epsqt
18/20 Test #18: examples_epsqt ................... Passed   73.34 sec
       Start 19: examples_pdfqt
19/20 Test #19: examples_pdfqt ................... Passed   52.69 sec
       Start 20: examples_compare
20/20 Test #20: examples_compare ................. Passed   10.87 sec

100% tests passed, 0 tests failed out of 20

Total Test time (real) = 834.62 sec


On 7/5/2015 09:24, Chris Marshall wrote:
> Running:
>
>   cmake -DBUILD_TEST=ON ../
>   make
>   make test
>
> all tests fail.  When I add the dll/ directory to the path by hand
> then everything appears to be running ok.  I'll let you know how
> it proceeds.
>
> Is there a reason that 'make test' on cygwin requires a hand fix?
> As far as I know this is an issue for windows and cygwin dlls.
> How is the problem handled with windows?
>
> Cheers,
> Chris

[Plplot-devel] make test fails on cygwin

[Chris M. <dev...@gm...>]

Running:

   cmake -DBUILD_TEST=ON ../
   make
   make test

all tests fail.  When I add the dll/ directory to the path by hand
then everything appears to be running ok.  I'll let you know how
it proceeds.

Is there a reason that 'make test' on cygwin requires a hand fix?
As far as I know this is an issue for windows and cygwin dlls.
How is the problem handled with windows?

Cheers,
Chris

Re: [Plplot-devel] patch 31 which contains wincairo fixes

[Hazen B. <hba...@ma...>]

On 07/04/2015 12:15 PM, Alan W. Irwin wrote:
> Hi Hazen:
>
> A PLplot user has created a patch containing wincairo fixes (see
> <http://sourceforge.net/p/plplot/patches/31/>).  As the originator of
> the wincairo device could you please evaluate this 3-line change to
> see whether we should apply it now to master so it will be included in
> the planned 5.11.1 bugfix release?

Hi Alan,

I think these changes are the right thing to do based on reading up on 
Microsoft's documentation and I would just go ahead and apply it. 
However since I no longer have a Windows development box I can't test 
it. The best test would be to make sure that it doesn't break 32 bit 
Windows.

-Hazen

Re: [Plplot-devel] Status report on remaining issues to be addressedfor the forthcoming 5.11.1 release (wxwidgets issues)

[Phil R. <p.d...@gm...>]

Hi Jim
I fought with x17 too. Philosophically the only correct behaviour for a driver to do on resize is to replot the whole buffer. For x17, this is clearly much more plotting than necessary. However, the only alternative is for each driver to have a buffer parser which checks which part of the buffer to plot. This should not be included in the drivers, but could be a future optimisation in Plplot core code. Already I think the division of labour between drivers and core is more blurred than it should be and including parsing in the drivers would worsen, not improve this.

Phil

-----Original Message-----
From: "Jim Dishaw" <ji...@di...>
Sent: ‎04/‎07/‎2015 19:58
To: "Alan W. Irwin" <ir...@be...>
Cc: "Phil Rosenberg" <p.d...@gm...>; "PLplot development list" <Plp...@li...>
Subject: Re: [Plplot-devel] Status report on remaining issues to be addressedfor the forthcoming 5.11.1 release (wxwidgets issues)

The behavior with the -np flag is something I have grappled with while implementing the new windows driver. Example 17 is particularly challenging because of the way the stripchart function is implemented.

Splitting out the wait for user input from the EOP handler is part of the fix. If that function is not defined, that might be part of the problem. 

As for example 17, what should the behavior be on a resize event?  Right now I play the back the entire buffer, which shows all the rescalings. I think the correct behavior on a resize would be to render the most recent frame. 



> On Jul 4, 2015, at 1:51 PM, Alan W. Irwin <ir...@be...> wrote:
> 
> Hi Phil:
> 
> Your recent efficiency improvements means it is now a lot more
> convenient to test the wxwidgets device.  So I have done that and
> noticed the following issues that appear to be caused by your
> efficiency improvements.
> 
> * The -np option no longer works properly.  For example, before these
> efficency improvements
> 
> examples/c/x08c -dev wxwidgets -np
> 
> would (extremely slowly) render each page of that example and then exit.
> 
> Now, the rendering for each page simply presents a a black screen
> before the exit occurs.  Sometimes, just as the example exits you will
> see a quick flash of the last page, but usually not.
> 
> In addition, the example now produces the following WARNING message
> for 5 of the pages
> 
> *** PLPLOT WARNING ***
> Failed to get text size from wxPLViewer - timeout
> 
> Without the -np option, none of these problems occur, and the
> time to render all the example 8 pages has a noticeable efficiency
> improvement consistent with the length of time the black screen
> is rendered when the -np option is used.
> 
> * examples/c/x17c -dev wxwidgets shows huge efficiency improvements,
> but it has changed from an interactive plot showing all the
> intermediate results to a black screen until the very end which then
> shows the final results. In other words, the interactive nature of
> this example has been lost.
> 
> * examples/c/x26c -dev wxwidgets shows something is wrong with the
> delivery of string-length calculations between -dev wxwidgets and
> wxPLViewer.  (For this case I am not sure whether this issue was
> introduced right when you implemented the new string-length
> calculation for wxwidgets or is the result of your recent efficiency
> changes.) The delivery of string-length information should be done
> independently for each page of this example (since the Russian text of
> the second page is longer than the English text of the first page).
> 
> What goes on here is that _for each page of the example_ the pllegend
> call internally calls plstrl which then requests the wxwidgets device
> to deliver plsc->string_length (used by pllegend to adjust legend-box
> size) before wxwidgets actually renders the string.  Of course, the
> complication is that -dev wxwidgets measures string lengths and
> renders those strings indirectly via wxPLViewer.  So obviously there
> has to be communications between -dev wxwidgets and wxPLViewer for
> every different page of example 26 to get that done properly.
> 
> Instead what happens now is that examples/c/x26c -dev wxwidgets 
> incorrectly finishes (i.e., you get a command-line prompt and/or time
> results if requested) as soon as the first page of that example has 
> been rendered by wxPLViewer. And when the second page is viewed by
> hitting the enter key for the wxPLViewer GUI, the legend box (which
> should be controlled by the string-length calculation for the Russian
> text of that page) appears to be the same size as the first page so
> that the longer Russian text for the second page overflows the legend
> box for that page.
> 
> Alan
> __________________________
> Alan W. Irwin
> 
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
> 
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
> 
> Linux-powered Science
> __________________________
> 
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Plplot-devel mailing list
> Plp...@li...
> https://lists.sourceforge.net/lists/listinfo/plplot-devel

Re: [Plplot-devel] Status report on remaining issues to be addressed for theforthcoming 5.11.1 release (wxwidgets issues)

[Phil R. <p.d...@gm...>]

Hi Alan
Could describe exactly what the -np option should do and also add that to the documentation. At the moment for wxWidgets as soon as a page is rendered we move to the next page, which as you state causes at most a brief flicker of a plot. But this is exactly what the documentation states it should do.

Regarding the warnings, I am not sure what to do. How long do you think is sensible to wait for a response from the viewer? This is a balance, because if someone kills the viewer or it crashes then the console will be waiting for a response that never comes. I will look at tuning this, but suggestions welcome.

I will look into x17 and x26. I think I can guess the issue for 17, but unless 26 is doing something obscure I think it should have just worked.

Phil

-----Original Message-----
From: "Alan W. Irwin" <ir...@be...>
Sent: ‎04/‎07/‎2015 18:51
To: "Phil Rosenberg" <p.d...@gm...>
Cc: "PLplot development list" <Plp...@li...>
Subject: Re: Status report on remaining issues to be addressed for theforthcoming 5.11.1 release (wxwidgets issues)

Hi Phil:

Your recent efficiency improvements means it is now a lot more
convenient to test the wxwidgets device.  So I have done that and
noticed the following issues that appear to be caused by your
efficiency improvements.

* The -np option no longer works properly.  For example, before these
efficency improvements

examples/c/x08c -dev wxwidgets -np

would (extremely slowly) render each page of that example and then exit.

Now, the rendering for each page simply presents a a black screen
before the exit occurs.  Sometimes, just as the example exits you will
see a quick flash of the last page, but usually not.

In addition, the example now produces the following WARNING message
for 5 of the pages

*** PLPLOT WARNING ***
Failed to get text size from wxPLViewer - timeout

Without the -np option, none of these problems occur, and the
time to render all the example 8 pages has a noticeable efficiency
improvement consistent with the length of time the black screen
is rendered when the -np option is used.

* examples/c/x17c -dev wxwidgets shows huge efficiency improvements,
but it has changed from an interactive plot showing all the
intermediate results to a black screen until the very end which then
shows the final results. In other words, the interactive nature of
this example has been lost.

* examples/c/x26c -dev wxwidgets shows something is wrong with the
delivery of string-length calculations between -dev wxwidgets and
wxPLViewer.  (For this case I am not sure whether this issue was
introduced right when you implemented the new string-length
calculation for wxwidgets or is the result of your recent efficiency
changes.) The delivery of string-length information should be done
independently for each page of this example (since the Russian text of
the second page is longer than the English text of the first page).

What goes on here is that _for each page of the example_ the pllegend
call internally calls plstrl which then requests the wxwidgets device
to deliver plsc->string_length (used by pllegend to adjust legend-box
size) before wxwidgets actually renders the string.  Of course, the
complication is that -dev wxwidgets measures string lengths and
renders those strings indirectly via wxPLViewer.  So obviously there
has to be communications between -dev wxwidgets and wxPLViewer for
every different page of example 26 to get that done properly.

Instead what happens now is that examples/c/x26c -dev wxwidgets 
incorrectly finishes (i.e., you get a command-line prompt and/or time
results if requested) as soon as the first page of that example has 
been rendered by wxPLViewer. And when the second page is viewed by
hitting the enter key for the wxPLViewer GUI, the legend box (which
should be controlled by the string-length calculation for the Russian
text of that page) appears to be the same size as the first page so
that the longer Russian text for the second page overflows the legend
box for that page.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________