Re: [Plplot-devel] Important new evidence concerning the remaining Linux inefficiencyissues for wxwidgets

[<p.d...@gm...>]

Hi Alan
Good detective work. I'll have a look as soon as I can

Sent from my Windows 10 phone

From: Alan W. Irwin
Sent: 02 February 2017 01:33
To: Phil Rosenberg; PLplot development list
Subject: Important new evidence concerning the remaining Linux inefficiencyissues for wxwidgets

Hi Phil:

The wxwidgets IPC (inter process communication) improvements I have
just started working on are suddenly less urgent because some new
evidence (see below) indicates the cause of the Linux inefficiency
issue has nothing to do with IPC!

To remind you of what that (remaining Linux inefficiency) problem is, here
are timing results for example 8 for current master.

I first built test_c_xcairo and test_c_wxwidgets to get all
prerequisites built properly.  Then

software@raven> time(examples/c/x08c -dev xcairo -np)

real    0m2.128s
user    0m1.896s
sys     0m0.040s
software@raven> time(examples/c/x08c -dev wxwidgets -np)

real    0m13.423s
user    0m0.224s
sys     0m0.024s

Note the ~13 seconds of time when my cpus were completely idle for the
-dev wxwidgets -np test, and my cpumeter confirmed that result (i.e.,
it barely twitched from dead idle through most of this time). Of
course, the above timing results just measures examples/c/x08c and
-dev wxwidgets and ignores wxPLViewer, but that widget is quite
inefficient for this example as well because it remained on my screen
for ~20
seconds after example 8 and -dev wxwidgets had completed.

My initial (naive) view was this order of magnitude inefficiency in both
-dev wxwidgets and wxPLViewer likely had to do with IPC, but that
extra 20 seconds where wxPLViewer continues on after examples/c/x08c
(and therefore -dev wxwidgets) completes indicates there is at least
one big inefficiency that must not have anything to do with IPC.
Furthermore if you drop the -np option you get the following result:

software@raven> time(examples/c/x08c -dev wxwidgets)

real    0m1.067s
user    0m0.236s
sys     0m0.020s

That is, without the -np (nopause) option -dev wxwidgets is faster (!)
than xcairo (as it should be since the rendering done by wxPLViewer is
not counted as part of this time).  That still leaves the order of
magnitude inefficiency of wxPLViewer for this case (it required 30
seconds or so to get this example rendered if I hit the "enter" key as
soon as the plot of one page was completed so this inefficiency
appears to be the same as in the -np case).  But this wxPLViewer
inefficiency _cannot_ have anything to do with IPC since
examples/c/x08c and -dev wxwidgets are completely done and dusted
after ~1 second when no -np option is used.

So it appears likely to me that what is going on for the -np case is
some of the large wxPLViewer inefficieny is slowing down -dev
wxwidgets, i.e., the real culprit is wxPLViewer inefficiency for the
-np case.  Note also that Jim's split of the wait and eop states that
I reinstated recently for -dev xcairo completely changed how the -np
option was handled for that device so addressing that split of wait
and eop for wxwidgets might also make a big difference to the above
-np timing results for -dev wxwidgets (although such a change should
have no effect on the order of magnitude inefficiency of wxPLViewer
itself).

@Phil:

As a result of the above new test results, I hope you agree your own
wxwidgets development plans should include the following two steps as
a matter of some urgency:

1. Implement the wait/eop split for wxwidgets following Jim's work
that was recently reinstated for xcairo.  I would do that myself, but
I don't understand the plbuf/xcairo nuances used by Jim (e.g., why
explicit nopause processing could be removed in xcairo yet the -np
option still clearly works properly for that device), and I don't
understand enough of the wxwidgets device driver to figure out exactly
what part of the current eop processing should be split off into the
new wait routine that must be implemented for -dev wxwidgets.

Anyhow, my feeling is dealing with this issue is a matter of some
urgency since it could potentially kill 3 birds with one stone, i.e.,
(i) the wxwidgets eop issue you were concerned about during the last
few days of the previous release cycle (although ultimately our joint
decision was not to delay the 5.12.0 release due to that issue), (ii)
the background color issue for the first page of example 16, and (iii)
the above order of magnitude inefficieny in -dev wxwidgets only when
the -np option is used. But regardless of these speculations the most
important point is this current inconsistency between plbuf and
wxwidgets should be addressed soon since plbuf now distinguishes
between wait states and eop states and until wxwidgets also follows
that same model we won't know whether some wxwidgets issue we observe
(such as any of the 3 issues above) is due to this current mismatch
between plbuf and wxwidgets or something else.

2. Solve the inefficient rendering (already demonstrated above to have
nothing to do with IPC) currently shown by wxPLViewer for examples
such as example 8 that have many graphical elements to plot. I feel
solving this issue is a matter of urgency since this lack of
wxPLViewer rendering speed (at least when there is anything
complicated to plot as in example 8) will tend to discourage most user
interest in wxwidgets regardless of platform until it is fixed.

Assuming you solve these two important issues in a timely manner, I
promise to follow up with a 5.12.1 release so that wxwidgets users can
quickly benefit from your work.

I have already invested some time in improved IPC for wxwidgets.  I
doubt (from the above evidence) it will improve efficiency very much,
but I still plan to finish this effort simply because the efficient
IPC algorithm I implemented for the cmake/test_linux_ipc miniproject
is much simpler (no mutex, no sleeping) then the IPC algorithm used
now for wxwidgets.

So I will continue updating you on my wxwidgets IPC effort, and I hope
to have it completed in a matter of a week or so.  However, I believe
now your timely resolution of the above two wxwidgets issues is much
more important than my own ongoing work on simplifying the wxwidgets
IPC.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________