plplot-devel — Technical discussion about PLplot development

Re: [Plplot-devel] The status of the wxwidgets IPC development

[Alan W. I. <ir...@be...>]

Hi Phil:

At long last I have some preliminary code (commit 635df69) for you to
look at that produces (for -DPL_WXWIDGETS_IPC2=ON) good results for
one-page plots such as x00c, x01c, and x24c, but which currently has a
bug for multipage plots which I am trying to understand.

Sorry this took so long, but I implemented many iterations of this
code as I refined my ideas.  And my debugging efforts were
considerably slowed down by a major g++/gdb issue where the "next"
command does not reliably indicate exactly what command it is going to
be executing, i.e., it can be a couple of lines off (even for the
completely unoptimized case).  So for a while there I was trying to
figure out why the code was jumping outside loops according to "next"
until I finally realized what was going on.

The new approach separates the IPC from the assembly of information on
the -dev wxwidgets side and the rendering of that data on the
wxPLViewer side.  It also clearly separates plbuf data from header
data and never uses the shared memory space for semi-permanent storage
of header information.  (Instead, header information is transmitted
from dev -wxwidgets to wxPLViewer or vice versa via a specific header
location in shared memory and stored in local private variables.) The
result is each call to transmitBuffer by -dev wxwidgets sends separate
header information and (if needed) plbuf information, while a call to
ReadTransmission by wxPLViewer reads all headers and accumulates and
plots plbuf information sent by those transmitBuffer calls.

In my opinion the result of the above separations is much improved
code clarity, and I hope you agree with that assessment.  Also, the
new code appears to be fast; all the plbuf data was assembled and
transferred to wxPLViewer in less than a second for the 10 pages of
example 8.

I am currently trying to track down a bug (uncaught exception on the
wxPLViewer side) that shows up whenever there is an attempt to plot
multiple pages.  I tried to copy the code stanzas from the old
wxplframe.cpp code to the #ifdef PL_WXWIDGETS_IPC2 case for each
transmission type, but I may have missed something that is critical to
the multipage case.  Also for #ifdef PL_WXWIDGETS_IPC2, one call to
ReadTransmission reads all pages of plbuf and should plot those pages.
That's a big chance in design from ReadTransmission which for the
#ifndef PL_WXWIDGETS_IPC2 case reads and plots at most one page.

Is there something more I have to do to accomodate that design change
so that the multipage case is displayed properly by wxPLViewer
when #ifdef PL_WXWIDGETS_IPC?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Best git version for Windows?

[Phil R. <p.d...@gm...>]

Hi Jim
Good luck with the restoration. If this helps for the future then as a
windows user you can get 1 TB of OneDrive space as part of an Office
365 subscription. I basically symlinked my Documents folder into my
OneDrive folder so that everything I do gets continually synced with
the cloud.

Anyway, as far as git is concerned I use Git within Cygwin even for my
Windows development. I always feel the Linux environment that goes
along with this gives the best development environment. You could
probably get an identical setup by using the new Bash on Ubuntu on
Windows feature if you are using Windows 10. Windows now ships with
the ability to install a native but stripped down (command line only)
version of Ubuntu within Windows. This runs native Linux executables
and you have access to apt-get so it's pretty handy!

Phil

On 21 February 2017 at 21:45, Alan W. Irwin <ir...@be...> wrote:
> To Jim, Phil, and Arjen:
>
> On 2017-02-20 00:38-0500 Jim Dishaw wrote:
>
>> I need your help to sort out git.  Long story short, I had a disk
>
> corruption on the VM that I was doing my development work and had to
> recover my work.  I manage to get my wingdi driver recovered from the
> smoking heap.
>
> @Jim:
>
> My sympathies concerning your hardware troubles.  I am especially
> sensitive to that issue because I just went through a hardware scare
> myself (spent a lot of yesterday running hardware tests when I ran
> into the *.pyc corruption issue).  But amazingly this 9-year-old
> hardware (with an ASUS motherboard which might be the reason for this
> longevity) still passes all hardware tests, and I have concluded (with
> a fair amount of confidence) that the *.pyc corruption issue must be
> due to some Python bug.  So I plan to keep using this hardware for
> a while longer.
>
>> Unfortunately, I appear to be having problems with the git
>
> repository on SourceForge and I am not sure of the cause—I cannot even
> clone from SourceForge.
>
> @Jim, Phil, and Arjen:
>
> I used the git SF server just this morning with no issues.  Also, for
> the reasons discussed in README.developers you should avoid all gui
> versions or "enhanced" versions of git (i.e., try to stick as much as
> possible to the real thing).  Bearing those constraints in mind, that
> file recommends <https://github.com/msysgit/msysgit> for Windows
> users, but I just discovered from looking at that site that it has
> been obsoleted and msysgit developers now recommend using the "Git for
> Windows" <https://git-for-windows.github.io/> version of git instead.
> (I confirmed from that website it considers itself light-weight
> [check!] and it does have a command-line version [check!]).  So please
> give the command-line version of that project a try, and let us know
> whether it works well for you (which would allow us to recommend that
> Windows version of git in our README.developers file).
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

Re: [Plplot-devel] Best git version for Windows?

[Arjen M. <Arj...@de...>]

Hi Alan,

Never tried it before - I have sofar relied on Cygwin to do the job. I made a note to look into this after my holiday next week.

Regards,

Arjen

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

Re: [Plplot-devel] A mysterious pyc file corruption issue

[Alan W. I. <ir...@be...>]

On 2017-02-21 11:01-0800 Alan W. Irwin wrote:

> Yesterday for the second time in two months an interactive
> comprehensive test failed with a "ValueError: bad marshal data
> (unknown type code)" error for bindings/python/Plframe.py.
[.... These errors were so common]
> that the python developers list in 2013 became
> concerned that python would be subject to race conditions when
> generating these files and thus was the author of at least some of
> these corruptions (see discussion thread at
> <https://mail.python.org/pipermail/python-dev/2013-May/126241.html>
> with the subject line "[Python-Dev] Mysterious Python pyc file
> corruption problems".

I have now been in contact with the OP, Barry Warsaw of python.org, of
that thread who was quite helpful.  For example, Barry told me that
Python is designed so it is frankly impossible for

import Plframe
from Plframe import *

to race (i.e., the first import completely finishes before the second
one starts).  And I cannot find any other cases where Plframe is
imported.  So I think the best bet for explaining this *.pyc
Python-generated file corruption is some unknown Python 2 bug that
does not have anything to do with races.  I got the sense from
what Barry said that he feels Python 3 is now much more reliable than
Python 2.  So this may be another instance of that general idea.

Anyhow, I think the next step is to test whether this corruption
occurs for Python 3.  (And if it does I get the sense that Barry would
be anxious to figure out what that Python 3 bug was.)

@Hazen:

This issue lends lots of additional motivation for making PLplot work
correctly with Python 3.  So please go ahead and push your Python 3
topic as soon as it is in reasonable state, and we can mature it
further (if necessary) from there.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] The new wingdi device has been pushed

[Alan W. I. <ir...@be...>]

On 2017-02-20 00:38-0500 Jim Dishaw wrote:

> I need your help to sort out git.
[....]

> This a pure GDI implementation and it works fairly well.  The font
sizes are a bit off, printing causes a core dump, and the busy cursor
does not go away after a resize. I think a may have lost a commit when
I lost the VM.  This driver outputs a lot of debugging messages if
debugging is turned on.  The minimum version of Windows is XP—it will
not work on anything less.  I am not sure if I will be able to
maintain Windows XP compatibility as this driver evolves.  I have not
implemented unicode support.

The unicode support issue is an all-important one in my opinion, but
this does sound like a good start.

> [Because of my current git troubles] I have attached the patches
[for my wingdi development] to this email,
would it be possible for you to apply them?

Yes, pushing your commits was no problem at all (see the recent flurry
of SF commits leading up to and including e639c33).  That last commit
is mine and resulted from running

scripts/remove_trailing_whitespace.sh

and

scripts/style_source.sh

I do not mind doing such cleanups myself, but if you like you
should be able to run those scripts yourself if you have access to a
platform with Unix command-line capabilities (such as Mac OS X,
Cygwin, or MinGW-w64).

Note, I also heavily tested commit e639c33 to assure none of your
changes to common driver-related files interfered with the Linux
device drivers.

So it appears all was well with your series of commits, and I
encourage you to follow up on this good start by continuing with the
plans you discussed to implement access to unicode-aware Windows
system fonts using Uniscribe for the wingdi device and implementing an
additional device that uses Direct2D/DirectWrite to handle text for
Windows platforms that support those more advanced text capabilities.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] Best git version for Windows?

[Alan W. I. <ir...@be...>]

To Jim, Phil, and Arjen:

On 2017-02-20 00:38-0500 Jim Dishaw wrote:

> I need your help to sort out git.  Long story short, I had a disk
corruption on the VM that I was doing my development work and had to
recover my work.  I manage to get my wingdi driver recovered from the
smoking heap.

@Jim:

My sympathies concerning your hardware troubles.  I am especially
sensitive to that issue because I just went through a hardware scare
myself (spent a lot of yesterday running hardware tests when I ran
into the *.pyc corruption issue).  But amazingly this 9-year-old
hardware (with an ASUS motherboard which might be the reason for this
longevity) still passes all hardware tests, and I have concluded (with
a fair amount of confidence) that the *.pyc corruption issue must be
due to some Python bug.  So I plan to keep using this hardware for
a while longer.

> Unfortunately, I appear to be having problems with the git
repository on SourceForge and I am not sure of the cause—I cannot even
clone from SourceForge.

@Jim, Phil, and Arjen:

I used the git SF server just this morning with no issues.  Also, for
the reasons discussed in README.developers you should avoid all gui
versions or "enhanced" versions of git (i.e., try to stick as much as
possible to the real thing).  Bearing those constraints in mind, that
file recommends <https://github.com/msysgit/msysgit> for Windows
users, but I just discovered from looking at that site that it has
been obsoleted and msysgit developers now recommend using the "Git for
Windows" <https://git-for-windows.github.io/> version of git instead.
(I confirmed from that website it considers itself light-weight
[check!] and it does have a command-line version [check!]).  So please
give the command-line version of that project a try, and let us know
whether it works well for you (which would allow us to recommend that
Windows version of git in our README.developers file).

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] We appear to be ready for CMake-3.7.2

[Alan W. I. <ir...@be...>]

I just built CMake-3.7.2 with the bootstrap method and used it for a
comprehensive test of PLplot with no constraints on the tests. The
first time I ran the test I ran into the corrupted Plframe.pyc issue I
recently discussed here.  But once I moved that offending corrupted
Plframe.pyc file out of the way the comprehensive test passed with
flying colours.  So it appears our current master branch code is in
good shape (unless there is something we can do concerning that
intermittent Plframe.py corruption issue) and ready for CMake-3.7.2.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] A mysterious pyc file corruption issue

[Alan W. I. <ir...@be...>]

Yesterday for the second time in two months an interactive
comprehensive test failed with a "ValueError: bad marshal data
(unknown type code)" error for bindings/python/Plframe.py.  This is a
rather common error with python and typically means the associated
*.pyc that is generated by python has been corrupted.  I moved that
corresponding *.pyc file out of the way, and the comprehensive test
(with *.pyc regenerated by python) sailed through afterward without
issues.  For the record, this issue occurred on my Debian Jessie
platform with python version string of

irwin@raven> python --version
Python 2.7.9

There are lots of potential reasons for such *.pyc corruption issues
such as a change in python version and hardware issues, but these
errors are so common that the python developers list in 2013 became
concerned that python would be subject to race conditions when
generating these files and thus was the author of at least some of
these corruptions (see discussion thread at
<https://mail.python.org/pipermail/python-dev/2013-May/126241.html>
with the subject line "[Python-Dev] Mysterious Python pyc file
corruption problems".

I did an octal dump of the corrupted file versus the uncorrupted
regenerated one, and as far as I can tell the only difference is
a missing byte in the corrupted file.  (If anyone is interested
I can send those files to you for inspection.)

Yesterday I did do some obvious tests (with memtest, fsck, and git
fsck) of my PC hardware (which is 9 [!] years old, but still going
strong), and all was well.  Furthermore, the above octal dumps showed
no i/o issue with the corrupted file, and the problem always occurs
(so far) with just this particular file.  And these rare errors only
started when I started enabling testing of examples/python/pytkdemo
(our only file that imports PLframe which would generate the *.pyc as
a byproduct of that import) with the test_pytkdemo target. So I am
pretty sure this evidence largely rules out any hardware issue.  And I
have not been fiddling with my python versions, and in any case those
changes should just change a version stamp (at least two bytes) in the
file and not simply remove one byte.

So by a process of elimination, I think this is likely one more
candidate for the mysterious python pyc corruption issue.  However, if
the source of this corruption is a race condition in the python
generation of these files, I believe that would only be an issue if
there are simultaneous attempts to generate this file.  The tests I
run do use parallel builds but the test_pytkdemo target is implemented
with a CMake custom target where there should be no build race
conditions (attempting to build that target twice) unless there is a
bug in either CMake or make.  But if that were the case, we would be
seeing similar errors for our other python test targets, and we don't.
However, if you look at examples/python/pytkdemo, it is interesting
that it imports PLframe in two ways, i.e.

import Plframe
from Plframe import *

This is a fairly common (but sloppy) python idiom for importing both a
namespaced and unnamespaced version of PLframe (because some of our
code uses the namespaced version and some of our code does not).
However, the only way you get a race out of that is if python looks
ahead and starts doing the second import (which would attempt to also
generate a PLframe.pyc file) before the first import was finished, and
I have no idea whether that is a possibility or not.

Anyhow, in the near future I plan to track down all our references to
the version of PLframe that is not namespaced and convert it to the
namespaced version so that second import can be eliminated.  And it
will be interesting to see if that makes this corruption issue
disappear.

Meanwhile, if anyone else can replicate this issue that would even be
stronger evidence it is not due to my hardware.  So if you want to
help out with that, you should run the test_pytkdemo target but only
after touching examples/python/Plframe.py (which would force python to
regenerate the *.pyc file when the test_pytkdemo target is run).  And
you should do this test from time to time under a variety of load
conditions so generating the above error even once may be difficult to
accomplish.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] The status of the wxwidgets IPC development

[Alan W. I. <ir...@be...>]

Hi Phil:

I am still progressing at a steady pace with this wxwidgets IPC
development effort. There are no additional commits to share with you
at this time (but coming soon!), and I appreciate that you haven't
caused me any conflict resolution issues as I work on what previously
had been largely your wxwidgets code.

I have wxPLDevice::TransmitBuffer completely implemented for
for -DPL_WXWIDGETS_IPC2=ON.  There I transmit the bytes in two
batches.  One batch for everything but the PLplot buffer data,
and one batch (if needed) exclusively for that buffer data.  The
result builds without issues, but I haven't committed it yet
because I am not (quite) in a position to test it yet.

I have started the corresponding -DPL_WXWIDGETS_IPC2=ON implementation
of wxPlFrame::ReadTransmission, and when that is done I should be able
to test my work and commit it.  That commit will ignore the issue of
transmitting interactive keystrokes back from wxPLViewer to -dev
wxwidgets, but I have made a lot of general progress in that area as
well, see below.

I have figured out what I think is a fool-proof way for wxPLViewer to
use the read semaphore to coordinate when -dev wxwidgets needs to read
what wxPLViewer has written to the PLMemoryMap.  I implemented this
idea as two additional methods of the PLMemoryMap class.  I tested
that new code by using it to communicate the wxPLViewer update of

     header->viewerOpenFlag = 1;

back to -dev wxwidgets (to signal back all is well with the
PLMemoryMap instance on the wxPLViewer), and I got a good result with
no requirement of having to go into a wxMilliSleep loop on the -dev
wxwidgets side to wait for wxPLViewer to make contact.  So I think
using this general approach is also going to work well for
transmitting back interactive keystroke information from wxPLViewer to
-dev wxwidgets.

Note, these two new methods could also be used to send back text size
information (if, for example, we wanted to do something other than a
simple scaling of text size when resizing the graph).  However, I just
checked with the second page of the second example (where lots of
numbers are inscribed almost exactly in boxes) and both -dev xwin and
-dev qtwidget clearly do nothing but simple scaling of the text size
when resizing. Thus, for the -dev wxwidgets case there is no need to
send back text size with each resize event, and indeed today I
discovered (as expected now I have seen what is done for other
interactive devices) all the text size measurement was done completely
on the -dev wxwidgets side with no feedback at all from the wxPLViewer
side.  But if you wanted to consolidate that measurement of text size
with the actual rendering of that same text (all on the wxPLViewer
side), it could be done with this method.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] The status of the wxwidgets IPC development

[Alan W. I. <ir...@be...>]

Hi Phil:

I have found a bug in the "catch(char *)" commands used in
drivers/wxwidgets.cpp.  I have discovered this bug from my attempts to
use throw("some message") style exceptions in my (currently unpushed)
-DPL_WXWIDGETS_IPC2=ON version of wxPLDevice::TransmitBuffer.  Those
catches did not work (i.e., the messages were being ignored) until I
changed them to "catch(const char *)" (because const char * and NOT
char * is the type of any fixed string such as "some message"). Note
this fix is recommended by
<http://stackoverflow.com/questions/33410536/how-to-catch-char-exceptions-in-c/33410593>.

Now that I am finally getting specific exception messages for my
-DPL_WXWIDGETS_IPC2=ON code, I am debugging a specific exception that
has been revealed by this change.  When that effort is done (some time
tomorrow after I get some sleep), I plan to change "catch(char *)" to
"catch(const char *)" everywhere in drivers/wxwidgets.cpp to follow
what I have done for the -DPL_WXWIDGETS_IPC2=ON version of
wxPLDevice::TransmitBuffer. This fix should make the messages for all
exceptions caught by that drivers/wxwidgets.cpp code much more
specific.

I think you can likely tell from this particular episode and other
evidence that I am getting much more comfortable with C++ now.  So
that, the above bug fix, and some of your reorganization ideas
inspired by my transmitBytes and receiveBytes methods of the
PLMemoryMap class are already good results from this effort.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] The status of the wxwidgets IPC development

[Alan W. I. <ir...@be...>]

On 2017-02-13 20:24-0000 p.d...@gm... wrote:

> I have absolutely no efficiency concerns. The cpu cycles will be
totally negligible compared to the time to copy the memory. To be
honest I rarely stress about efficiency. Premature optimisation is
usually a bad thing – don't forget that the compiler has the ability
to totally optimise away your classes anyway if it sees fit.

Hi Phil:

I completely agree with you on the perils of premature optimization,
and it sounds like for both of us, if it is a choice between code
clarity and some minor efficiency issue, code clarity wins every time.

> My concern is more style and maintenance related. Placing all the
communication code in one class or a family of parent and child
classes makes sense logically which makes maintenance easier and
allows different communication methods to be swapped in should we need
in the future – for example it would mean that the change from the
current ipc method to a new one would be a single line of code, once
that new method is written.

Good point which I finally understand now.  And it sounds from what
you have said that my mental model of what needs to be done to convert
my present local instance method to the private object method is
correct so I will just go ahead and make that implementation change
without bothering with testing the efficiency of the local instance
versus private object methods.

Thanks very much for the above instructive feedback!

Now that this issue has been cleared up, I hope to have some useful
results to report soon.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] The status of the wxwidgets IPC development

[<p.d...@gm...>]

I have absolutely no efficiency concerns. The cpu cycles will be totally negligible compared to the time to copy the memory. To be honest I rarely stress about efficiency. Premature optimisation is usually a bad thing – don't forget that the compiler has the ability to totally optimise away your classes anyway if it sees fit.

My concern is more style and maintenance related. Placing all the communication code in one class or a family of parent and child classes makes sense logically which makes maintenance easier and allows different communication methods to be swapped in should we need in the future – for example it would mean that the change from the current ipc method to a new one would be a single line of code, once that new method is written.

If the idea of creating a “null” object then calling a create method to fully initialise it does not sit right with you then you could instead set up the private member as a pointer which is initialised to null and use new to create the object when needed. But this doesn't really have any advantages. The class could have a method called something like isInitialised which allows it to be checked just like checking a pointer is null. But with a pointer there could be a path where its deletion is missed and the pointer goes out of scope and resources leak.

The other option is to use a smart pointer. This is a class that acts like a pointer but will automatically delete the object when it is no longer needed – the exact timing depends on the type of smart pointer, which may be shared or unique.

This is the joy of C++ 😊 We can automate a lot of things that need to be dealt with manually in C and if you make use of them it can avoid bugs ever occurring. So it makes sense to use them to their max. Read the book I recommended 😉 it actually has a whole section on writing a smart pointer class – although it is a bit out of date, as since C++ 11 there are built in smart pointers called std::unique_ptr and std::shared_ptr, but reading about how they work from first principles is a really good intro to good C++ style.

Phil
 
Sent from my Windows 10 phone

From: Alan W. Irwin
Sent: 13 February 2017 19:05
To: Phil Rosenberg
Cc: PLplot development list
Subject: Re: [Plplot-devel] The status of the wxwidgets IPC development

On 2017-02-12 22:51-0800 Alan W. Irwin wrote:

> I plan to stick with my local instance approach for now (automatically
> constructing a PLTwoSemaphore instance and destroying it again for
> each call to my transmitBytes and receiveBytes methods), but move to
> the private object approach for efficiency later on once I understand
> (hopefully with some further help from you) exactly what it does at
> run time compared with the local instance approach.

Hi Phil:

I have thought a lot more about this efficiency question using my
current mental model of the private object approach (if we applied it
for the case of PLMemoryMap accessing PLTwoSemapores). Please correct
that mental model if I have made a mistake with it, but if that model
is correct, then I predict (see below) there will be negligible
efficiency differences between the present local instance approach and
the private object approach.

I assume to make PLTwoSemapores accessible as a private object from
PLMemoryMap, I would move the current code in the PLTwoSemapores
constructor to a create method, replace the current PLTwoSemapores
constructor with one without arguments that basically forces the two
semaphore addresses to be invalid (NULL) values, and move the current
destructor code to a delete method so the destructor becomes a no-op.
Then whenever PLMemoryMap was instantiated, the private PLTwoSemapores
object that is mentioned in its private: section is also automatically
instantiated (with the invalid addresses).  Then within transmitBytes
and receiveBytes methods, the create method for the private
PLTwoSemapores object is called first, and the destroy method for that
same object is called last.  And ultimately when the PLMemoryMap
object is destroyed, the no-op destructor for its private
PLTwoSemapores object is also automatically called.

If that is the correct model, then the code in create and destroy is
the same as in the current PLTwoSemapores instantiation and
destruction that occurs in the current transmitBytes and receiveBytes
methods.  So is your efficiency concern not that create and destroy
code, but other code that must be executed if you do a full-blown
instantiation and destruction of the PLTwoSemapores object?

Or are you also concerned with the cpu cycles required to call the
create and destroy methods for PLTwoSemapores every time the
transmitBytes and receiveBytes methods were called?  For example, you
_could_ call the appropriate create and destroy methods outside
transmitBytes and receiveBytes (and likely less often if you used a
lot of care), but I would prefer not to do that because that
non-locality makes the code much harder to understand (say when we
look at it again 5 years from now) and I believe the number of cpu
cycles saved should be negligible compared with the number of cpu
cycles actually required to execute transmitBytes and receiveBytes.

Come to think of it, the number of cpu cycles needed for a full-blown
instantiation and destruction of a PLTwoSemapores object within
transmitBytes and receiveBytes should also be negligible compared to
even just the cpu costs of the memcpy calls that do the necessary
chunking up of the array of bytes to be sent via shared memory with
transitBytes and the similar memcpy costs on the receiving end to
reassemble the array of bytes, and the argument becomes stronger if
you add in the times for sem_wait to finish on the transmission
and receive ends.

So unless my mental model above is incorrect, I predict we will find
no discernible difference in efficiency when comparing between the
current local instance approach and the private object approach at
least for the use of PLTwoSemapores by the transmitBytes and
receiveBytes methods of PLMemoryMap.  But I am willing to implement
the private object approach variant (controlled by a preprocessing
macro) and measure its efficiency compared with the current local
instance approach to confirm (or disprove) that prediction, and once
we have those test results, we can decide which approach to take.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] The status of the wxwidgets IPC development

[Alan W. I. <ir...@be...>]

On 2017-02-12 22:51-0800 Alan W. Irwin wrote:

> I plan to stick with my local instance approach for now (automatically
> constructing a PLTwoSemaphore instance and destroying it again for
> each call to my transmitBytes and receiveBytes methods), but move to
> the private object approach for efficiency later on once I understand
> (hopefully with some further help from you) exactly what it does at
> run time compared with the local instance approach.

Hi Phil:

I have thought a lot more about this efficiency question using my
current mental model of the private object approach (if we applied it
for the case of PLMemoryMap accessing PLTwoSemapores). Please correct
that mental model if I have made a mistake with it, but if that model
is correct, then I predict (see below) there will be negligible
efficiency differences between the present local instance approach and
the private object approach.

I assume to make PLTwoSemapores accessible as a private object from
PLMemoryMap, I would move the current code in the PLTwoSemapores
constructor to a create method, replace the current PLTwoSemapores
constructor with one without arguments that basically forces the two
semaphore addresses to be invalid (NULL) values, and move the current
destructor code to a delete method so the destructor becomes a no-op.
Then whenever PLMemoryMap was instantiated, the private PLTwoSemapores
object that is mentioned in its private: section is also automatically
instantiated (with the invalid addresses).  Then within transmitBytes
and receiveBytes methods, the create method for the private
PLTwoSemapores object is called first, and the destroy method for that
same object is called last.  And ultimately when the PLMemoryMap
object is destroyed, the no-op destructor for its private
PLTwoSemapores object is also automatically called.

If that is the correct model, then the code in create and destroy is
the same as in the current PLTwoSemapores instantiation and
destruction that occurs in the current transmitBytes and receiveBytes
methods.  So is your efficiency concern not that create and destroy
code, but other code that must be executed if you do a full-blown
instantiation and destruction of the PLTwoSemapores object?

Or are you also concerned with the cpu cycles required to call the
create and destroy methods for PLTwoSemapores every time the
transmitBytes and receiveBytes methods were called?  For example, you
_could_ call the appropriate create and destroy methods outside
transmitBytes and receiveBytes (and likely less often if you used a
lot of care), but I would prefer not to do that because that
non-locality makes the code much harder to understand (say when we
look at it again 5 years from now) and I believe the number of cpu
cycles saved should be negligible compared with the number of cpu
cycles actually required to execute transmitBytes and receiveBytes.

Come to think of it, the number of cpu cycles needed for a full-blown
instantiation and destruction of a PLTwoSemapores object within
transmitBytes and receiveBytes should also be negligible compared to
even just the cpu costs of the memcpy calls that do the necessary
chunking up of the array of bytes to be sent via shared memory with
transitBytes and the similar memcpy costs on the receiving end to
reassemble the array of bytes, and the argument becomes stronger if
you add in the times for sem_wait to finish on the transmission
and receive ends.

So unless my mental model above is incorrect, I predict we will find
no discernible difference in efficiency when comparing between the
current local instance approach and the private object approach at
least for the use of PLTwoSemapores by the transmitBytes and
receiveBytes methods of PLMemoryMap.  But I am willing to implement
the private object approach variant (controlled by a preprocessing
macro) and measure its efficiency compared with the current local
instance approach to confirm (or disprove) that prediction, and once
we have those test results, we can decide which approach to take.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] The status of the wxwidgets IPC development

[Alan W. I. <ir...@be...>]

On 2017-02-12 23:08-0000 Phil Rosenberg wrote:

> Hi Alan
>
> The short answer is yes use a transmit and receive function to send
> the data, but no don't create a new object every time you want to
> send something. The current private object is definitely the way to
> go.

OK.

I plan to stick with my local instance approach for now (automatically
constructing a PLTwoSemaphore instance and destroying it again for
each call to my transmitBytes and receiveBytes methods), but move to
the private object approach for efficiency later on once I understand
(hopefully with some further help from you) exactly what it does at
run time compared with the local instance approach.

> I should have used a transmit and receive function in the current
> code. If I wasn't going to trample your uncommitted changes I would
> actually do that right now. The reason is partitioning of
> functionality. The job of the Memory map, as it stands is to provide a
> space in memory where the wxDevice can write data. You are absolutely
> correct that it would be much better to have a send and receive method

Yes, that was my exact motivation.

[...]
> However, if you pull out the semaphores and have wxPLDevice look after
> those then we have messed up our nice separation of functionality
> again. Our wxPLDevice now knows that we are communicating with
> semaphores and memory maps.

> So what I should have done (and this probably came from the way things
> evolved and got more complicated than I expected) is created a
> PLCommunicator class. Then I could have inherited from this to make a
> PLMemoryMapCommunicator and all the code from wxDevice::Transmit,
> should go in there. In fact if you can give me some time to do that
> without stepping on your uncommitted edits then I will as I think it
> will make your life easier.

Actually, I am right in the middle of modifying
wxPLDevice::TransmitBuffer and wxPlFrame::ReadTransmission to use the
transmitBytes and receiveBytes methods of PLMemoryMap (which are now
completely implemented and they build, but I haven't pushed that
commit because I have no way of testing those methods until the
modifications to wxPLDevice::TransmitBuffer and
wxPlFrame::ReadTransmission are done).  So I have invested
considerable time in understanding those routines as they are now
organized.  Also, my ongoing changes for the -DPL_WXWIDGETS_IPC2=ON
case clearly separate assembling the data in
wxPLDevice::TransmitBuffer and acting on it in
wxPlFrame::ReadTransmission from the task of transmitting the
assembled array of bytes from wxPLDevice::TransmitBuffer and receiving
that same array of bytes in wxPlFrame::ReadTransmission.  So my
separation work (already mostly completed now) should give you a good
start on that desired reorganization.  So please hold off until later
to reorganize the code.

To summarize, here are the next steps I plan to do.

(1) Get the two-semaphores approach fundamentally working one
way from -dev wxwidgets to wxPLViewer by ignoring interactive examples
and temporarily dropping text size feedback from wxPLViewer back to
-dev wxwidgets and ultimately the PLplot core.

(2) Refine that approach to make it two-way so that the
-DPL_WXWIDGETS_IPC2=ON result passes all tests (including text size
feedback as well as including example 1 with the -locate option) that
are currently passed by the -DPL_WXWIDGETS_IPC2=OFF case.

(3) Measure the efficiency of that local instance method and then try
the private object method (once I understand exactly what it does at
run-time) to see how much that efficiency improves.

(4) Decide (with you) based on the simplicity and speed of the
two-semaphores approach compared with the one-semaphore approach which
approach we will take after that.

(5) If the decision is for two semaphores, then the further steps I
have in mind would be to (i) (Alan) implement a _named_ two-semaphores
variation on the unnamed two-semaphores approach, (ii) (Phil)
implement a named two-semaphores approach for the Windows case, and
(iii) (Alan or Phil) completely remove all one-semaphore code.  But if
our decision goes against the two-semaphores approach, then (i') (Alan
or Phil) strip all the two-semaphores code out instead but (ii')
(Phil) keep the idea of separating the transmitBytes and receiveBytes
methods (which have no logic other than to transmit and receive arrays
of bytes) from everything else.

It is not at all clear how long these steps are going to take, but I am
hoping for a time scale of something like one day each for (1) through
(3).  And I hope you don't mind holding off any of the changes you are
thinking about (especially the reorganizing ones) since such changes
would slow me down, and we, in any case, really have no idea where we
are going with the wxwidgets IPC until (4) is done.

More later as I continue to make progress on (1) through (3).

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] C++ class constructor question

[Phil R. <p.d...@gm...>]

Hi Alan

The short answer is yes use a transmit and receive function to send
the data, but no don't create a new object every time you want to
send something. The current private object is definitely the way to
go.

I should have used a transmit and receive function in the current
code. If I wasn't going to trample your uncommitted changes I would
actually do that right now. The reason is partitioning of
functionality. The job of the Memory map, as it stands is to provide a
space in memory where the wxDevice can write data. You are absolutely
correct that it would be much better to have a send and receive method
- in reality the wxDevice does not care how the data gets sent, either
via shared memory, over a network port, by writing to a file on the
disk drive or even by messenger pigeon - it just wants to send it and
sometimes get some back. By adding those functions we totally separate
out the transmission from the other stuff that wxPLDevice does. This
would mean actually we could create different classes that do the
transmission in different ways and we could use different objects as
slot in replacements if we liked.

However, if you pull out the semaphores and have wxPLDevice look after
those then we have messed up our nice separation of functionality
again. Our wxPLDevice now knows that we are communicating with
semaphores and memory maps.

So what I should have done (and this probably came from the way things
evolved and got more complicated than I expected) is created a
PLCommunicator class. Then I could have inherited from this to make a
PLMemoryMapCommunicator and all the code from wxDevice::Transmit,
should go in there. In fact if you can give me some time to do that
without stepping on your uncommitted edits then I will as I think it
will make your life easier.

Now in terms of having a default constructor that initialises
everything invalid, well there is nothing wrong with that. I don't see
how constructing a new object for every use is simpler, in fact the
opposite. I'm also not sure where exactly you will be creating your
shared memory. In PLMemoryMap? If you keep creating and destroying
your PLMemoryMap objects then you will keep scrapping and recreating
your shared memory won't you? It also seems that although you would
remove a small amount of code from PlMemory map, you instead make the
class more difficult to use, so it doesn't seem like the right choice
for me in many ways.

I hope that makes sense.

Phil


On 11 February 2017 at 21:46, Alan W. Irwin <ir...@be...> wrote:
> Hi Phil:
>
> I have noticed the style of communications you are using between various
> classes typically involves private variables such as
>
> PLMemoryMap  m_outputMemoryMap;
>
> for wxPLDevice, a default constructors for
> PLMemoryMap that essentially sets everything relevant to invalid
> values, and no explicit use of the constructor that does valid
> initialization for PLMemoryMap from within wxPLDevice.  (Instead an
> explicit create method is called to do the required valid
> initialization.)
>
> Would there be anything wrong with dropping that private variable,
> dropping the default invalid constructor, dropping the create method,
> creating an instance of PLMemoryMap (with correct explicit valid
> constructor with a mustExist argument to decide if you are going to
> reuse external memory locations from a previously initialized
> instance), and using the methods of that instance when you need them?
>
> That latter simplified approach is the one I have tentatively adopted
> for the new PLTwoSemaphores class I have just implemented (see commit
> cd13d65).  The (only) constructor checks that the argument values
> corresponding to the addresses of the two semaphores are non-NULL, and
> then the internal private variables of the instance that correspond to
> those two addresses are updated.  If mustExist is true, nothing more
> is done, but if mustExist is false, then sem_init is called for each
> of those pointers. The (only) destructor for the PLTwoSemaphores class
> calls sem_destroy for each of the (internal) pointers and sets those
> internal pointers to NULL.  And all other methods of PLTwoSemaphores
> that use those internal pointers check that they are non-NULL.
>
> So far the two-semaphores implementation is incomplete so I have only
> tested the new PLTwoSemaphores class with a build (with the g++
> compiler finding no issue with it).  Do you anticipate I will be
> running into trouble later on with the above simplified constructor
> approach once I start to use PLTwoSemaphores class instances (e.g., in
> the arguments for the currently empty PLMemoryMap::sendBytes and
> PLMemoryMap::receiveBytes methods?
>
> If it turns out you do believe the current simplified PLTwoSemaphores
> constructor approach will work OK, should we also switch to that
> simplified approach everywhere else (e.g., to handle wxPLDevice's use
> of PLMemoryMap)?
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

[Plplot-devel] C++ class constructor question

[Alan W. I. <ir...@be...>]

Hi Phil:

I have noticed the style of communications you are using between various
classes typically involves private variables such as

PLMemoryMap  m_outputMemoryMap;

for wxPLDevice, a default constructors for
PLMemoryMap that essentially sets everything relevant to invalid
values, and no explicit use of the constructor that does valid
initialization for PLMemoryMap from within wxPLDevice.  (Instead an
explicit create method is called to do the required valid
initialization.)

Would there be anything wrong with dropping that private variable,
dropping the default invalid constructor, dropping the create method,
creating an instance of PLMemoryMap (with correct explicit valid
constructor with a mustExist argument to decide if you are going to
reuse external memory locations from a previously initialized
instance), and using the methods of that instance when you need them?

That latter simplified approach is the one I have tentatively adopted
for the new PLTwoSemaphores class I have just implemented (see commit
cd13d65).  The (only) constructor checks that the argument values
corresponding to the addresses of the two semaphores are non-NULL, and
then the internal private variables of the instance that correspond to
those two addresses are updated.  If mustExist is true, nothing more
is done, but if mustExist is false, then sem_init is called for each
of those pointers. The (only) destructor for the PLTwoSemaphores class
calls sem_destroy for each of the (internal) pointers and sets those
internal pointers to NULL.  And all other methods of PLTwoSemaphores
that use those internal pointers check that they are non-NULL.

So far the two-semaphores implementation is incomplete so I have only
tested the new PLTwoSemaphores class with a build (with the g++
compiler finding no issue with it).  Do you anticipate I will be
running into trouble later on with the above simplified constructor
approach once I start to use PLTwoSemaphores class instances (e.g., in
the arguments for the currently empty PLMemoryMap::sendBytes and
PLMemoryMap::receiveBytes methods?

If it turns out you do believe the current simplified PLTwoSemaphores
constructor approach will work OK, should we also switch to that
simplified approach everywhere else (e.g., to handle wxPLDevice's use
of PLMemoryMap)?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Plplot-devel] C++ OOP reference

[Alan W. I. <ir...@be...>]

Hi Phil:

Thanks to a google search I think I have found a pretty good C++ OOP
tutorial reference which is
<https://www3.ntu.edu.sg/home/ehchua/programming/cpp/cp3_OOP.html>.

Up to now I had only understood bits and pieces of C++ OOP detail via
reading through and sometimes modifying PLplot code (by rote without
much understanding) or asking you pretty specific questions.  So I
felt my knowledge of the C++ and OOP overview was way too limited.
But this reference appears to pull together much of what I need in an
organized way, and after one read-through I feel fairly close to
understanding the C++ OOP overview (and many details) at this level.

As a result I hope my C++ questions to you will be (a) fewer and (b) a
little less elementary.  :-)

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Recent progress with wxwidgets IPC

[Alan W. I. <ir...@be...>]

On 2017-02-07 22:03-0000 p.d...@gm... wrote:

> Hi Alan

> It'll definitely be interesting to see the differences. I wasn't
trying to put you off, just hoping you can get a boost from the things
that i learnt as I went along. That was the first time I'd tried ipc
locally, although I did some one way network stuff years ago. It did
turn out to be more difficult than I expected, especially once I
realised there needed to be some 2 way comms.

> The only thing about sending amount, yes you are correct about the
amountToCopy variable, but that is for exactly one transfer. However,
i don’t think there is a reliable end of page command so you can't
store up commands and send them all in one go when the plot is
complete. Also the viewer never knows if the page is complete so must
always keep rechecking for new data.

Thanks, Phil.  I will keep everything you have said in mind, and I
will likely have some specific questions later for you as I gain a
deeper understanding of all the constraints on the wxwidgets IPC.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Recent progress with wxwidgets IPC

[<p.d...@gm...>]

Hi Alan
It'll definitely be interesting to see the differences. I wasn't trying to put you off, just hoping you can get a boost from the things that i learnt as I went along. That was the first time I'd tried ipc locally, although I did some one way network stuff years ago. It did turn out to be more difficult than I expected, especially once I realised there needed to be some 2 way comms.

The only thing about sending amount, yes you are correct about the amountToCopy variable, but that is for exactly one transfer. However, i don’t think there is a reliable end of page command so you can't store up commands and send them all in one go when the plot is complete. Also the viewer never knows if the page is complete so must always keep rechecking for new data.

Phil

Sent from my Windows 10 phone

From: Alan W. Irwin
Sent: 07 February 2017 20:51
To: Phil Rosenberg
Cc: PLplot development list
Subject: Re: [Plplot-devel] Recent progress with wxwidgets IPC

Hi Phil:

You make a lot of points about some uncertainties in what I propose to
do.  And I do agree there are such uncertainties.  So this is
definitely a "show them the code" moment.  At worst, I will strip it
all out again because it will turn out to be complex and slow.  But it
could be significantly less complex (no polling!) and just as
efficient or better.  So we will see.

Further discussion below about that "no polling" point.

On 2017-02-07 13:51-0000 Phil Rosenberg wrote:

[...]
> In a simple
> restart-from-the-beginning buffer like you seem to be proposing the
> sender must wait until the reader has read all the data from the
> buffer before it can send more data.

True, but neverthless the mini-project demonstrated this was an
efficient method of moving many MBytes at one go since effectively the
only costs (assuming overheads are small) are a memcpy of those bytes
on the sending end and a memcpy of those bytes on the receiving end,
and (this is the important point) without any polling at all!  Note,
in principle for GigaHertz machines a 1 GByte memcpy should only
require 1 second or so.  So we are discussing really small
inefficiencies here so long as the size of the buffer is large enough
to make the overheads of the method (the overhead of setting up the
transfer of control to the memcpy routine for each chunk, and the
overhead of checking semaphores as each chunk is passed).  So I would
expect the method would be inefficient for really small buffer sizes
(such as 100 bytes or so).  But, for example, I was already getting
quite efficient results with a buffer size of just 1K (!) so I don't
think the overheads of this two-semaphore method are that big a deal.

Of course, if you add polling to the mix, then that would introduce a
lot of efficiency concern. But there is no polling needed or used in
this mini-project so that is why it is efficient, and I believe I can
avoid polling as well for the wxwidgets IPC method, see below.

[...]
> Don't forget also that while waiting for new data
> the semaphore cannot block indefinitely. To do so would hang
> wxPLViewer or the sender software. I think also there is no way to
> tell if a page is finished or whether there is more data to come.
> Therefore you must use non- blocking semaphores and poll them on
> regular intervals.

My assumption is the sending side knows exactly how many bytes it
needs to send.  And my preliminary analysis is that is exactly what
the present code calculates with the amountToCopy variable
in wxPLDevice::TransmitBuffer.  So with that assumption (and as
demonstrated by the mini-project) there is no need for non-blocking
semaphores or polling if you use the two-semaphore method.

Of course, this line of reasoning completely falls apart if
amountToCopy does not do what its name implies so please let
me know if there is some case where that calculation is unreliable.

My efficiency test results for the case where the -np option is not
used show the inefficiences of the present wxwidgets IPC are
negligible compared to wxPLViewer taking a long time (30 seconds for
example 8) to render the plot while other interactive devices take the
order of a second for this same task.  Most of this large time
interval occurs after -dev wxwidgets is completely finished so IPC
ineffiency cannot be the source of this wxPLViewer inefficiency for
cases like example 8 where large numbers of graphical elements are
being plotted.  Therefore, from this evidence you do have the polling
interval used in the present one-semaphore method tuned up fairly well
(at least for typical PC hardware). So my fundamental goal here is
making our wxwidgets IPC a lot simpler for POSIX systems by
eliminating the polling and the rest of the circular buffer logic for
that case.  I am hoping for some noticeable improvement in efficiency
due to this, but I am not counting on anything showing at all in that
regard until at least the non-IPC inefficiency of wxPLViewer is
addressed.

In sum, it is "show the code" time.  That is, it is pretty clear what
I have said above has speculative elements and similarly for any
further replies you make (unless you know of some cases where
amountToCopy is definitely unreliable). So my focus from now on will
be to continue my project of implementing the two-semaphore method for
wxwidgets IPC. Once I have completed that implementation, we should
evaluate that code and obviously if it is simpler and there is at
least no drop in efficiency we should adopt it but otherwise not.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Recent progress with wxwidgets IPC

[Alan W. I. <ir...@be...>]

Hi Phil:

You make a lot of points about some uncertainties in what I propose to
do.  And I do agree there are such uncertainties.  So this is
definitely a "show them the code" moment.  At worst, I will strip it
all out again because it will turn out to be complex and slow.  But it
could be significantly less complex (no polling!) and just as
efficient or better.  So we will see.

Further discussion below about that "no polling" point.

On 2017-02-07 13:51-0000 Phil Rosenberg wrote:

[...]
> In a simple
> restart-from-the-beginning buffer like you seem to be proposing the
> sender must wait until the reader has read all the data from the
> buffer before it can send more data.

True, but neverthless the mini-project demonstrated this was an
efficient method of moving many MBytes at one go since effectively the
only costs (assuming overheads are small) are a memcpy of those bytes
on the sending end and a memcpy of those bytes on the receiving end,
and (this is the important point) without any polling at all!  Note,
in principle for GigaHertz machines a 1 GByte memcpy should only
require 1 second or so.  So we are discussing really small
inefficiencies here so long as the size of the buffer is large enough
to make the overheads of the method (the overhead of setting up the
transfer of control to the memcpy routine for each chunk, and the
overhead of checking semaphores as each chunk is passed).  So I would
expect the method would be inefficient for really small buffer sizes
(such as 100 bytes or so).  But, for example, I was already getting
quite efficient results with a buffer size of just 1K (!) so I don't
think the overheads of this two-semaphore method are that big a deal.

Of course, if you add polling to the mix, then that would introduce a
lot of efficiency concern. But there is no polling needed or used in
this mini-project so that is why it is efficient, and I believe I can
avoid polling as well for the wxwidgets IPC method, see below.

[...]
> Don't forget also that while waiting for new data
> the semaphore cannot block indefinitely. To do so would hang
> wxPLViewer or the sender software. I think also there is no way to
> tell if a page is finished or whether there is more data to come.
> Therefore you must use non- blocking semaphores and poll them on
> regular intervals.

My assumption is the sending side knows exactly how many bytes it
needs to send.  And my preliminary analysis is that is exactly what
the present code calculates with the amountToCopy variable
in wxPLDevice::TransmitBuffer.  So with that assumption (and as
demonstrated by the mini-project) there is no need for non-blocking
semaphores or polling if you use the two-semaphore method.

Of course, this line of reasoning completely falls apart if
amountToCopy does not do what its name implies so please let
me know if there is some case where that calculation is unreliable.

My efficiency test results for the case where the -np option is not
used show the inefficiences of the present wxwidgets IPC are
negligible compared to wxPLViewer taking a long time (30 seconds for
example 8) to render the plot while other interactive devices take the
order of a second for this same task.  Most of this large time
interval occurs after -dev wxwidgets is completely finished so IPC
ineffiency cannot be the source of this wxPLViewer inefficiency for
cases like example 8 where large numbers of graphical elements are
being plotted.  Therefore, from this evidence you do have the polling
interval used in the present one-semaphore method tuned up fairly well
(at least for typical PC hardware). So my fundamental goal here is
making our wxwidgets IPC a lot simpler for POSIX systems by
eliminating the polling and the rest of the circular buffer logic for
that case.  I am hoping for some noticeable improvement in efficiency
due to this, but I am not counting on anything showing at all in that
regard until at least the non-IPC inefficiency of wxPLViewer is
addressed.

In sum, it is "show the code" time.  That is, it is pretty clear what
I have said above has speculative elements and similarly for any
further replies you make (unless you know of some cases where
amountToCopy is definitely unreliable). So my focus from now on will
be to continue my project of implementing the two-semaphore method for
wxwidgets IPC. Once I have completed that implementation, we should
evaluate that code and obviously if it is simpler and there is at
least no drop in efficiency we should adopt it but otherwise not.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Recent progress with wxwidgets IPC

[Phil R. <p.d...@gm...>]

Hi Alan
That seems reasonable. I'm not sure what the benefits are though.
Should the new way be quicker?

In terms of the overall complexity here are some of the things I found
while setting up the current system that each made the system more
complex than I initially envisaged. I imagine you will come across
many of these too.

The reason that I went for a circular buffer is that data can
continuously be being sent even while the reader is reading. That is
until the writer catches up with the reader. In a simple
restart-from-the-beginning buffer like you seem to be proposing the
sender must wait until the reader has read all the data from the
buffer before it can send more data.

In fact I have a feeling that the system as I have it now makes little
or no use of the actual semaphores and just uses flags in specified
locations in the shared memory. I would have to look through the code
properly to check. Don't forget also that while waiting for new data
the semaphore cannot block indefinitely. To do so would hang
wxPLViewer or the sender software. I think also there is no way to
tell if a page is finished or whether there is more data to come.
Therefore you must use non- blocking semaphores and poll them on
regular intervals.


If you want to do two way communications you will need two areas of
shared memory, one for each direction, otherwise you will likely have
a race condition for which process will write next. As it happens,
like you said the transfer is almost all one way, with just a small
amount being returned. So I just allocated a small portion of the
shared memory to represent a struct to hold some specific returned
information. I'm doing this from memory, but I think it would work
like this for the case of getting text size.

Sender zeros the flag which indicates a text size is available.
Sender sends a message to the Viewer saying it wants a text size and
with the string it wants sizing
Sender starts checking the flag to say that the text size is available.
Viewer receives the message requesting text size
Viewer determines the text size and writes it to the location in
shared memory reserved for text size
Viewer sets the flag which indicates text size is available to 1
Sender sees the text size flag it one
Sender reads the text size from the location in shared memory that is
reserved for text size.

The same is basically true for getting position information for
pllocate calls. But there is a pause on the Viewer side while it waits
for user input.

Of course the alternate is to set up a totally generic symmetric
system, I'm not sure if one is easier or faster.

I guess all these things contribute to the complexity of the code that
is there, but I'm not sure that it is more complex than necessary.
I'll be interested to see how your setup differs :-)

Phil



On 7 February 2017 at 02:01, Alan W. Irwin <ir...@be...> wrote:
> On 2017-02-06 23:52-0000 p.d...@gm... wrote:
>
>> Hi Alan
>
>
>> Not exactly sure what you mean by complex? It is not always possible
>
> to send all data, as the shared memory is finite size and therefore
> the data to be sent may be bigger than the shared memory.
>
> Hi Phil:
>
> To get a preview of what I mean by implementing an approach that is
> simpler than the current one, I suggest you take a look at the code in
> cmake/test_linux_ipc.  There, the shared memory buffer size is
> relatively small, that buffer is _not_ a circular buffer, and
> typically the amount of data to be transmitted >> shared memory buffer
> size.  The data are split up into chunks that fit into the buffer on
> the sending side and those chunks are assembled on the receiving side
> under the control of two semaphores.  The result is an efficient
> transfer of the entirety of what can be an extremely large amount of
> data (25MB in one test) between sender and receiver with relatively
> simple code and relatively small shared memory buffer.
>
> My current plan is to have a generic function "send" for sending a
> generic array of bytes and a generic function "receive" for receiving
> that array where those functions contain all the details of the
> two-semaphore method for transmitting and receiving a generic data
> array.  Then higher level code would create an array to be sent or
> received and then they would use this send/receive API to transfer
> those arrays.  Of course, the usual case is that -dev wxwidgets
> normally would call the "send" API and wxPlViewer normally would
> normally call the "receive" API, but when those roles are reversed
> (e.g., when transmitting back the physical size of displayed text
> strings), then wxPlViewer will be calling the "send" API and -dev
> wxwidgets will be using the "receive" API.
>
>> I presume it's this named semaphore and/or memory flags that you intend to
>> remove?
>
>
> I am definitely going to keep named semaphores (see step 3 in my
> original plan where a small change to the two-semaphores approach
> should change that approach from two unnamed semaphores to two named
> semaphores.) This variant of the two-semaphores method is necessary
> because some lame proprietary Unix systems (e.g., Mac OS X) only
> return a "not implemented" error when attempting to use unnamed
> semaphores.
>
> I still don't understand enough about the current code to answer the
> memory flags part of your question, but the only thing I intend to
> change is from the current circular buffer/one semaphore approach to
> the above two-semaphore approach for sending and receiving data with
> the plan being that -dev wxwidgets will both send and receive data and
> similarly for wxPLViewer.
>
> And, of course, I will leave the IPC approach used for the Windows case
> completely untouched.
>
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________

Re: [Plplot-devel] Recent progress with wxwidgets IPC

[Alan W. I. <ir...@be...>]

On 2017-02-06 23:52-0000 p.d...@gm... wrote:

> Hi Alan

> Not exactly sure what you mean by complex? It is not always possible
to send all data, as the shared memory is finite size and therefore
the data to be sent may be bigger than the shared memory.

Hi Phil:

To get a preview of what I mean by implementing an approach that is
simpler than the current one, I suggest you take a look at the code in
cmake/test_linux_ipc.  There, the shared memory buffer size is
relatively small, that buffer is _not_ a circular buffer, and
typically the amount of data to be transmitted >> shared memory buffer
size.  The data are split up into chunks that fit into the buffer on
the sending side and those chunks are assembled on the receiving side
under the control of two semaphores.  The result is an efficient
transfer of the entirety of what can be an extremely large amount of
data (25MB in one test) between sender and receiver with relatively
simple code and relatively small shared memory buffer.

My current plan is to have a generic function "send" for sending a
generic array of bytes and a generic function "receive" for receiving
that array where those functions contain all the details of the
two-semaphore method for transmitting and receiving a generic data
array.  Then higher level code would create an array to be sent or
received and then they would use this send/receive API to transfer
those arrays.  Of course, the usual case is that -dev wxwidgets
normally would call the "send" API and wxPlViewer normally would
normally call the "receive" API, but when those roles are reversed
(e.g., when transmitting back the physical size of displayed text
strings), then wxPlViewer will be calling the "send" API and -dev
wxwidgets will be using the "receive" API.

> I presume it's this named semaphore and/or memory flags that you intend to remove?

I am definitely going to keep named semaphores (see step 3 in my
original plan where a small change to the two-semaphores approach
should change that approach from two unnamed semaphores to two named
semaphores.) This variant of the two-semaphores method is necessary
because some lame proprietary Unix systems (e.g., Mac OS X) only
return a "not implemented" error when attempting to use unnamed
semaphores.

I still don't understand enough about the current code to answer the
memory flags part of your question, but the only thing I intend to
change is from the current circular buffer/one semaphore approach to
the above two-semaphore approach for sending and receiving data with
the plan being that -dev wxwidgets will both send and receive data and
similarly for wxPLViewer.

And, of course, I will leave the IPC approach used for the Windows case
completely untouched.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Recent progress with wxwidgets IPC

[<p.d...@gm...>]

Hi Alan
Not exactly sure what you mean by complex? It is not always possible to send all data, as the shared memory is finite size and therefore the data to be sent may be bigger than the shared memory. The reading and writing is therefore set up as a ring buffer, so when the writing gets to the end of the memory it begins writing again at the beginning. Of course if we write faster than we read (entirely likely because as we read we may be rendering, which is expensive) then the writing will eventually catch up with the reading and would “lap it”. There is therefore extra logic to avoid that. Then there are some features that require two way comms such as getting text size and getting mouse positions and key presses for pllocate so there is extra logic for these. All this is set up with a series of flags in the memory. And a named semaphore if memory serves correctly.

I presume it's this named semaphore and/or memory flags that you intend to remove?

Phil

Sent from my Windows 10 phone

From: Alan W. Irwin
Sent: 06 February 2017 21:31
To: Phil Rosenberg; PLplot development list
Subject: Re: [Plplot-devel] Recent progress with wxwidgets IPC

On 2017-02-04 19:41-0800 Alan W. Irwin wrote:

> So in the ideal case (where all steps below ultimately work and there
> are no efficiency reductions due to any of these steps) the remaining
> planned steps in this development effort for the
> -DPL_HAVE_UNNAMED_POSIX_SEMAPHORES=ON case are as follows:
>
> 1. Make the current one-semaphore approach work with unnamed
> semaphores. The last commit put essentially all the infrastructure in
> place to support this further change so this change should be a simple
> matter of replacing (for this case) the calls to sem_open and
> sem_close with sem_init and sem_destroy, and the rest of the current
> one-semaphore code should work as is along with the mutex and sleep
> API calls required by this method.

Hi Phil:

Thanks to your quick response to my C++ questions I have now (commit e705f7a)
completed step 1 above (by simply passing the address of wsem as
appropriate).  As expected (since the same fundamental
one-semaphore method is used in both cases) there is essentially no
difference between results using -DPL_HAVE_UNNAMED_POSIX_SEMAPHORES=ON
and -DPL_HAVE_UNNAMED_POSIX_SEMAPHORES=OFF.

So it is on to step 2 (below) for me.

> 2. Move from the present one-semaphore approach (which necessarily
> includes mutex and sleep activity) to the mutex- and sleep-free
> two-semaphore model following the cmake/test_linux_ipc proof-of-concept
> project.

With regard to step 2. I am still trying to figure out why
wxPLDevice::TransmitBuffer and wxPlFrame::ReadTransmission are
currently so complex, and I hope they will be vastly simplified with
the two-semaphore approach that was used in the proof-of-concept
project cmake/test_linux_ipc.  In that project there was a simple
write loop to send _all_ the data using just the two semaphores to
control the progress of the data transfer and a simple read loop to
read all the data that was sent (again using the two semaphores to
control progress of the data transfer).  So that simple approach
is what I hope to implement for step 2.

More later as I get deeper into step 2.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] Recent progress with wxwidgets IPC

[Alan W. I. <ir...@be...>]

On 2017-02-04 19:41-0800 Alan W. Irwin wrote:

> So in the ideal case (where all steps below ultimately work and there
> are no efficiency reductions due to any of these steps) the remaining
> planned steps in this development effort for the
> -DPL_HAVE_UNNAMED_POSIX_SEMAPHORES=ON case are as follows:
>
> 1. Make the current one-semaphore approach work with unnamed
> semaphores. The last commit put essentially all the infrastructure in
> place to support this further change so this change should be a simple
> matter of replacing (for this case) the calls to sem_open and
> sem_close with sem_init and sem_destroy, and the rest of the current
> one-semaphore code should work as is along with the mutex and sleep
> API calls required by this method.

Hi Phil:

Thanks to your quick response to my C++ questions I have now (commit e705f7a)
completed step 1 above (by simply passing the address of wsem as
appropriate).  As expected (since the same fundamental
one-semaphore method is used in both cases) there is essentially no
difference between results using -DPL_HAVE_UNNAMED_POSIX_SEMAPHORES=ON
and -DPL_HAVE_UNNAMED_POSIX_SEMAPHORES=OFF.

So it is on to step 2 (below) for me.

> 2. Move from the present one-semaphore approach (which necessarily
> includes mutex and sleep activity) to the mutex- and sleep-free
> two-semaphore model following the cmake/test_linux_ipc proof-of-concept
> project.

With regard to step 2. I am still trying to figure out why
wxPLDevice::TransmitBuffer and wxPlFrame::ReadTransmission are
currently so complex, and I hope they will be vastly simplified with
the two-semaphore approach that was used in the proof-of-concept
project cmake/test_linux_ipc.  In that project there was a simple
write loop to send _all_ the data using just the two semaphores to
control the progress of the data transfer and a simple read loop to
read all the data that was sent (again using the two semaphores to
control progress of the data transfer).  So that simple approach
is what I hope to implement for step 2.

More later as I get deeper into step 2.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

Re: [Plplot-devel] C++ help requested for step 1. (fwd)

[<p.d...@gm...>]

Hi Alan
I'm not sure how exactly the two classes interact, but if the parameters for the constructor will be the same every time you can pass them in as constant values. I.e, I presume the size will be the same each time and you could use a static member to hold a name that you increment after each use and pass in on next construction.

Phil

Sent from my Windows 10 phone

From: Alan W. Irwin
Sent: 05 February 2017 22:35
To: Phil Rosenberg
Cc: PLplot development list
Subject: Re: [Plplot-devel] C++ help requested for step 1. (fwd)

On 2017-02-05 21:16-0000 Phil Rosenberg wrote:

[...]
> As it stands I don't see in your source code where you have called
> m_unamed_semaphore_MemoryMap.create( <add params here> );
>
> I think what is happening is that you construct your PLnamedMutex
> which constructs all the member variables using their default
> constructors (the ones which take no variables). The default
> constructor for PLMemoryMap just sets m_buffer to NULL. Your call to
> getwsem then takes m_buffer, which is NULL and tries to access some of
> it's data and I imagine that is the point that everything goes wrong.
>

OK. That completely explains my gdb results. All the members of the
shmbuf struct appear to well aligned so there is no padding.  So the
address of wsem (the first member) is exactly the same as the value of
m_buffer (the address of the struct) which is NULL according to your
above reasoning.  And on Linux, sem_t consumes 32 bytes for 64-bit
systems and size_t consumes 8 bytes.  So buf is offset from the start
of the struct by 2*32 + 8 = 72 bytes = 0x48 which is the (bad) gdb
result I got for the address of buf.  So it appears I am getting
consistently wrong results in the PLNamedMutex case for anything to do
with PLMemoryMap because of the default constructor used for
PLMemoryMap.

> Anyway, the fix is to either call m_unamed_semaphore_MemoryMap.create
> just before getwsem, or call this in the constructor of PLNamedMutex,
> or tell PLNamedMutext to use the non-default constructor for
> m_unamed_semaphore_MemoryMap. To do the last of these the syntax would
> be as follows in wxWidgets_comms.cpp
>
> PLMemoryMap::PLMemoryMap( const char *name, PLINT size, bool
> mustExist, bool mustNotExist )
> : m_unamed_semaphore_MemoryMap(<arguments for the constructor you want to use>)
> {
>

The issue with all these potential solutions is PLNamedMutex::create
needs access to name, size, etc. for the non-default PLMemoryMap
constructor.  So it appears that in all cases these solutions of this
inter-class communication issue involves passing additional arguments
to the PLNamedMutex so the non-default PLMemoryMap constructor can be
invoked with the right name, etc.  In which case I think the better
approach is not to invoke the PLMemoryMap constructor at all from
PLNamedMutex and instead pass the wsem address that is determined
(correctly according to gdb) from a call to m_outputMemoryMap.getswm()
in wxPLDevice::SetupMemoryMap.

Anyhow, thanks for your help pointing out the default constructor
issue for PLMemoryMap for my present code, and more later about
whether the alternative idea of passing the wsem address works out or
not.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________