Re: [Plplot-devel] Correct implementation of key and button events in the interactive drivers

[Alan W. I. <ir...@be...>]

On 2016-01-11 19:41-0000 Arjen Markus wrote:

> Hi everyone,
>
>
>
> I am trying to get example x20f to work properly with the new Fortran bindings and the wingcc driver. The first part is working: I see the various fields of the structure PLGraphicsIn being filled with plausible values - so the Fortran bindings are working. However, there are two fields that are not set in line with the xwin driver:
>
> -        The button field: it seems the left button has value 1 and the right button should be value 3 - in the window the somewhat confusing message says to press "button 2". I guess this is due to mice classically used on UNIX/X Window systems have/had three buttons. That is a simple change to the wingcc driver.
>
> -        The state field: this is a bit more troublesome. I have no idea what it means. Nor what values it can assume. Can anyone enlighten me?

Hi Arjen:

You have opened up a whole lot of issues here, but I am glad you have
done so.  Furthermore, I suggest you keep the issues as separated
as possible from each other by, for example, debugging wingcc
interactive issues with C examples only, and debugging the interactive
issues of the new Fortran examples/binding using -dev xwin alone
since that is our only device that has implemented interactivity
correctly so far, i.e., it is the only interactive device
on Linux that operates C example 20 and C example 01 with -xor example
option correctly.

@Phil (wxwidgets), @Jim (wingcc and your new extended version of that
you are developing), @ Arjen (wingcc if Jim does not get to it), and
those volunteers willing to support interactivity properly on xcairo
and qtwidgets.  This is a long post because of many cut and pasted
results from modified example 1 which are quite informative
netherless.  But please especially notice where the device you are
interested in is mentioned below, and please fix the several issues I
have identified for all such devices (other than xwin where the issues
are fairly minor).

Here are the issues that need to be dealt with.

I. C example 20 and the -xor example option for C example 01 are not
working correctly for most of our interactive devices.

For current master tip (commit 3f05b0b which includes all of Phil's
latest wxwidgets efforts), I find that with any of the xcairo,
qtwidget, or wxwidgets devices that examples/c/x20c simply skips
through the interactive part of the example with no chance to drag and
drop to select a part of the image to view like you get with the xwin
device.  Similarly, I find the -xor example option for examples/c/x01c
appears to be a nop-op for the xcairo, qtwidget, or wxwidgets devices
while the xwin device provides some interactive results (a symbol
visibly moving along one of the lines plotted).  Both these cases
depend critically on whether an interactive device has implemented xor
capability or not, but that may not be all the story because of
the push and release and push, drag, and drop issues that I
also note below.

If you search all driver code this way

grep -i xor drivers/* |less

then it appears that some attempt has been made to support xor mode
for cairo, wingcc, wxwidgets, and xwin, and it appears this
functionality does work properly for xwin but not for cairo and
wxwidgets, and I have not been able to test wingcc or Jim's device.
Note also, there currently seems to be not even an attempt
to support xor with our qtwidget device.

So implementation of working xor mode needs to be addressed by those
wanting to support proper device interactivity for wxwidgets, wingcc
(likely), Jim's device (likely), xcairo, and qtwidget.

II. Metadata returned by interactive events being processed by our
various interactive devices.

I have just now (commit 406d8cf) modified the -locate mode of C
example 01 to return complete information about all the fields in the
PLGraphicsIn struct returned by plGetCursor. This should vastly
improve our ability to diagnose interactive issues.

The results below also verify and enhance what Phil has said
in response to Arjen's question about state.

Here are results for a simple button push and release (without
dragging) for my left mouse, middle mouse (where you depress the
scroll wheel on a generic mouse that uses the Microsoft protocol), and
right mouse buttons followed by a simple push and release of the a, B,
C, and d buttons. I have confirmed in every case that the correct
subwindow was identified by these results.

xwin push and release:

software@raven> ./x01c -dev xwin -locate
PLplot library version: 5.11.1
subwin = 0, wx = 2.155642,  wy = 18.765058, dx = 0.214087,  dy = 0.756677
keysym = 0x20, button = 0x01, string = '', type = 0x00, state = 0x00
subwin = 1, wx = 0.355375,  wy = 0.019289, dx = 0.712697,  dy = 0.770030
keysym = 0x20, button = 0x02, string = '', type = 0x00, state = 0x00
subwin = 2, wx = 2.085808,  wy = 0.313648, dx = 0.211307,  dy = 0.232938
keysym = 0x20, button = 0x03, string = '', type = 0x00, state = 0x00
subwin = 0, wx = 1.547419,  wy = 15.049218, dx = 0.177943,  dy = 0.724036
keysym = 0x61, button = 0x00, string = 'a', type = 0x00, state = 0x00
subwin = 0, wx = 1.547419,  wy = 15.049218, dx = 0.177943,  dy = 0.724036
keysym = 0x42, button = 0x00, string = 'B', type = 0x00, state = 0x01
subwin = 0, wx = 1.547419,  wy = 15.049218, dx = 0.177943,  dy = 0.724036
keysym = 0x43, button = 0x00, string = 'C', type = 0x00, state = 0x01
subwin = 0, wx = 1.547419,  wy = 15.049218, dx = 0.177943,  dy = 0.724036
keysym = 0x64, button = 0x00, string = 'd', type = 0x00, state = 0x00

Evidently, xwin only pays attention to one of button/key press or
button/key release events but not both, and does not pay any attention
to the actual event corresponding to the push or release of the shift
key.  The shift key being pressed while a letter is pressed does
modify both the state result (which is changed to PL_MASK_SHIFT), the
keysym, and the resulting string of the event associated with
the non-modifying key.  And similarly when capslock is on
you get a corresponding change in the state to PL_MASK_CAPS and
corresponding changes in keysym and string.  I think all these
constraints are a good model of what other devices should do since
enough information is delivered that xwin knows how to handle the
interactive aspects of example 01 and 20 while the other devices do
not because they deliver information about too many events (xcairo)
or too few events (qtwidget, wxwidgets).

However, that said, I would like to see a minor xwin change so that it
always fills in string and state for the button case like xcairo does
instead of the current result for that case where string is empty and
state is 0x00.

xcairo push and release:

software@raven> ./x01c -dev xcairo -locate
PLplot library version: 5.11.1
subwin = 0, wx = 2.292135,  wy = 20.790289, dx = 0.225000,  dy = 0.775926
keysym = 0x20, button = 0x01, string = 'button 1', type = 0x00, state = 0x00
subwin = 0, wx = 2.292135,  wy = 20.790289, dx = 0.225000,  dy = 0.775926
keysym = 0x20, button = 0x01, string = 'button 1', type = 0x00, state = 0x100
subwin = 1, wx = 0.329669,  wy = 0.019185, dx = 0.706944,  dy = 0.746296
keysym = 0x20, button = 0x02, string = 'button 2', type = 0x00, state = 0x00
subwin = 1, wx = 0.329669,  wy = 0.019185, dx = 0.706944,  dy = 0.746296
keysym = 0x20, button = 0x02, string = 'button 2', type = 0x00, state = 0x200
subwin = 2, wx = 5.603909,  wy = 0.612271, dx = 0.313889,  dy = 0.294444
keysym = 0x20, button = 0x03, string = 'button 3', type = 0x00, state = 0x00
subwin = 2, wx = 5.603909,  wy = 0.612271, dx = 0.313889,  dy = 0.294444
keysym = 0x20, button = 0x03, string = 'button 3', type = 0x00, state = 0x400
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x61, button = 0x26, string = 'a', type = 0x00, state = 0x00
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x61, button = 0x26, string = 'a', type = 0x00, state = 0x00
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0xffe1, button = 0x32, string = 'Shift_L', type = 0x00, state = 0x00
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x42, button = 0x38, string = 'B', type = 0x00, state = 0x01
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x42, button = 0x38, string = 'B', type = 0x00, state = 0x01
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0xffe1, button = 0x32, string = 'Shift_L', type = 0x00, state = 0x01
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0xffe1, button = 0x32, string = 'Shift_L', type = 0x00, state = 0x00
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x43, button = 0x36, string = 'C', type = 0x00, state = 0x01
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x43, button = 0x36, string = 'C', type = 0x00, state = 0x01
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0xffe1, button = 0x32, string = 'Shift_L', type = 0x00, state = 0x01
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x64, button = 0x28, string = 'd', type = 0x00, state = 0x00
subwin = 0, wx = 2.485441,  wy = 20.989239, dx = 0.236111,  dy = 0.777778
keysym = 0x64, button = 0x28, string = 'd', type = 0x00, state = 0x00

N.B. Evidently xcairo marks both push and release events for buttons,
character keys and even modifying keys like Shift_L, and the only
difference for buttons is the state which on release is set to the
correct PL_MASK_BUTTON[1-3].  There is no difference in push and
release events for both ordinary and modifying keys.  I believe there
are way too many events being reported here, and that may be confusing
interactive results for -dev xcairo.  Therefore, I would strongly
advise our volunteer supporter of xcairo that the noted events should
be masked down to just one of push or release (with state set to
PL_MASK_BUTTON[1-3] for events involving those mouse buttons) and
modifying character push or release should have no event reported for
it all. It is good that xcairo makes a special effort to fill in
string and state (at least on release) for button events, and I urge
that be done for all other devices as well.

qtwidget push and release:

software@raven> ./x01c -dev qtwidget -locate
PLplot library version: 5.11.1
subwin = 0, wx = 2.789786,  wy = 17.775337, dx = 0.252969,  dy = 0.747899
keysym = 0x20, button = 0x01, string = '', type = 0x00, state = 0x100
subwin = 1, wx = 0.464964,  wy = 0.019220, dx = 0.752969,  dy = 0.754622
keysym = 0x20, button = 0x02, string = '', type = 0x00, state = 0x200
subwin = 2, wx = 4.343026,  wy = 0.565404, dx = 0.277910,  dy = 0.284034
keysym = 0x20, button = 0x03, string = '', type = 0x00, state = 0x400
subwin = 0, wx = 2.973587,  wy = 12.998339, dx = 0.263658,  dy = 0.704202
keysym = 0x00, button = 0x00, string = '', type = 0x00, state = 0x00
subwin = 0, wx = 2.994009,  wy = 12.814608, dx = 0.264846,  dy = 0.702521
keysym = 0x00, button = 0x00, string = '', type = 0x00, state = 0x00
subwin = 0, wx = 3.014431,  wy = 12.630877, dx = 0.266033,  dy = 0.700840
keysym = 0x00, button = 0x00, string = '', type = 0x00, state = 0x00
subwin = 0, wx = 3.014431,  wy = 12.630877, dx = 0.266033,  dy = 0.700840
keysym = 0x00, button = 0x00, string = '', type = 0x00, state = 0x00

This does follow the same limits as xwin in terms of what events are
noted which is good, but the amount of information supplied for each
event needs to be increased to what is supplied by xwin (supplemented
by the extra information I have asked for in that case).

wxwidgets push and release:

software@raven> ./x01c -dev wxwidgets -locate
PLplot library version: 5.11.1
subwin = 0, wx = 1.110958,  wy = 29.386613, dx = 0.155242,  dy = 0.854008
keysym = 0x20, button = 0x01, string = '', type = 0x00, state = 0x100
subwin = 1, wx = 0.435386,  wy = 0.019594, dx = 0.742608,  dy = 0.842557
keysym = 0x20, button = 0x02, string = '', type = 0x00, state = 0x200
subwin = 2, wx = 0.948778,  wy = 0.218958, dx = 0.180780,  dy = 0.212786
keysym = 0x20, button = 0x03, string = '', type = 0x00, state = 0x400

Note, I started with a black window with no plots on it, but I guessed
at the first upper left location of the subwindow to push the left
mouse button and got the above good first event from that and so
forth.  By the last event the plot window was actually plotting.
Obviously, these symptoms indicate there is a rendering bug here in
locate mode that needs to be addressed.

That said, these limited button results are good except for the empty
string returned for these events (note that same comment for all other
devices than xcairo which does return a proper string for button
events).  However, key events (as opposed to button events) are masked
out for this device, and that substantial limitation should be
addressed.  (I have noted this deficiency before, but I keep
mentioning it each time I run into it for wxwidgets because
interactive astronomical research programmes I am familiar with do
heavily use key events as well as mouse button events, and I assume
there are a lot of other interactive programmes that do that as well.)

That terminates my push and release data for mouse buttons and a few
selected keys, but now I would like to follow up with push,
redacted drag and drop results for the left mouse button for each of the
interactive devices I have mentioned, where the redacted part is every
drag event after the first one which are identical to the first one
except for position.

xwin push, redacted drag and drop:

software@raven> ./x01c -dev xwin -locate
PLplot library version: 5.11.1
subwin = 0, wx = 1.282297,  wy = 23.832113, dx = 0.162187,  dy = 0.801187
keysym = 0x20, button = 0x01, string = '', type = 0x00, state = 0x00
subwin = 0, wx = 1.297892,  wy = 23.663212, dx = 0.163114,  dy = 0.799703
keysym = 0x20, button = 0x00, string = '', type = 0x00, state = 0x100
[....]

Evidently, xwin identifies the button during the initial push, but
sets the button result to 0 during the subsquent drag event, but does
signal that event is going on with button 1 (the left mouse button in
my case) by producing the state result of 0x100 = PL_MASK_BUTTON1 for
a drag event.  (Just to satisfy my curiosity, I have also confirmed
state is PL_MASK_BUTTON2, and PL_MASK_BUTTON3 if the dragging is done
with button 2 [by pressing the scroll wheel for my mouse downward
without rotating it] or button 3 [the right mouse button in my case].)
Note that the final drop event is masked out.  I think that is a mistake
and the xcairo result below that includes that is what we should aim
for here.

xcairo push, redacted drag, and drop:

software@raven> ./x01c -dev xcairo -locate
PLplot library version: 5.11.1
subwin = 0, wx = 1.470581,  wy = 25.565071, dx = 0.177778,  dy = 0.820370
keysym = 0x20, button = 0x01, string = 'button 1', type = 0x00, state = 0x00
subwin = 0, wx = 1.494744,  wy = 25.366122, dx = 0.179167,  dy = 0.818519
keysym = 0x20, button = 0x00, string = 'button 0', type = 0x00, state = 0x100
[....]
subwin = 0, wx = 1.591398,  wy = 24.570325, dx = 0.184722,  dy = 0.811111
keysym = 0x20, button = 0x01, string = 'button 1', type = 0x00, state = 0x100

The three kinds of events here are the push, the drag (repeated many
times but redacted down to one) and the drop (the last record above).

I actually like this better than the xwin result above because it does
include the final drop event.  But for just push and release in the
same place you just want one event.  So some care would be required to
make push and release and push, drag, and drop report correctly for
both -dev xwin and -dev xcairo.

qtwidget push, redacted drag, and drop:

software@raven> ./x01c -dev qtwidget -locate
PLplot library version: 5.11.1
subwin = 0, wx = 1.176425,  wy = 26.226949, dx = 0.159145,  dy = 0.825210
keysym = 0x20, button = 0x01, string = '', type = 0x00, state = 0x100

For this case, only the push event is shown because the drag part and
the final drop had all events masked out so the above was all the
output there was.  So clearly some work needs to be done here to allow
push, drag, and drop for -dev qtwidget to be reported similarly to
what is available for -dev xcairo.

wxwidgets push, redacted drag, and drop:

software@raven> ./x01c -dev wxwidgets -locate
PLplot library version: 5.11.1
subwin = 0, wx = 0.487317,  wy = 33.354515, dx = 0.118952,  dy = 0.890267
keysym = 0x20, button = 0x01, string = '', type = 0x00, state = 0x100

For this case, only the push event is shown because the drag part and
the final drop had all events masked out so the above was all the
output there was.  So clearly some work needs to be done here to allow
push, drag, and drop for -dev wxwidgets to be reported similarly to
what is available for -dev xcairo.

III. Button identification codes mentioned by example 20.

I think those are wrong since the second button mentioned in the
example 20 message is actually button 3 (the right button) on my
generic mouse with default mouse button identification according to
the results contained in the PLGraphicsIn struct returned by
plGetCursor.  And most people who think they only have a two button
mouse forget that depressing the scroll wheel without spinning
it constitutes button 2 while the right-hand button is button
3.

@ Phil, Arjen, and Jim:

To help explore the possibilities here, what does C example 1 in
-locate mode tell Cygwin users about the 3 mice buttons I have
described on most generic mice?  If the middle button (consisting of
depressing the scroll wheel) has no effect on Cygwin with -dev xwin,
then it means the Cygwin developers have not been able to extract any
information about the mouse button from the Windows system, and it is
possibly just not available there. On the other hand, if that
experiment shows that -dev xwin on Cygwin notices events from that
middle mouse button, then it means that information must be available
on bare Windows (for the Cygwin X developers to harvest and pass on to
-dev xwin), and so that means that same event information should be 
available to -dev wingcc for both the Cygwin and MSVC cases.

My apologies this post was so long, but there was a lot of cut and
paste information from C example 1 -locate mode to convey which I hope
will be a big help in implementing proper interactivity for all our
interactive devices.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________