gnuplot-beta

Re: An observation in animate demo.

[Hans-Bernhard B. <br...@ph...>]

On Fri, 7 May 2004, Daniel J Sebald wrote:

> gnuplot> load 'animate.dem'
[...]
> Then at that point hit return and the demo starts... and zooms by 
> quickly.  But if I hit return once again in the middle of the demo it 
> will slow down its redraw rate significantly.

What platform?  Unix/X11 I presume?

I think that's the same old X11 multiple-stream handling problem again.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: An observation in animate demo.

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>On Fri, 7 May 2004, Daniel J Sebald wrote:
>
>  
>
>>gnuplot> load 'animate.dem'
>>    
>>
>[...]
>  
>
>>Then at that point hit return and the demo starts... and zooms by 
>>quickly.  But if I hit return once again in the middle of the demo it 
>>will slow down its redraw rate significantly.
>>    
>>
>
>What platform?  Unix/X11 I presume?
>

Right.

>I think that's the same old X11 multiple-stream handling problem again.
>

Oh.  I wasn't aware of that one.  But that sounds about right.

Dan

Re: An observation in animate demo.

[Ethan A M. <merritt@u.washington.edu>]

On Sunday 09 May 2004 01:27 pm, Hans-Bernhard Broeker wrote:
> On Fri, 7 May 2004, Daniel J Sebald wrote:
>> gnuplot> load 'animate.dem'

>>  But if I hit return once again in the middle of the demo it
>> will slow down its redraw rate significantly.
>
> I think that's the same old X11 multiple-stream handling problem again.

Sort of.  I added a time delay deliberately to handle a different
problem, but this is an example of where it imposes a penalty.
Let me explain what is going on, and you all can speak up as
to how you want it handled.

The issue is this.

1) When a plot (or replot) is started, the x11 driver sends a query to
the X-server asking it some basic information like how big is the
window and what size is the font being used.  This allows much better
plot layout. Because of this the x11 layout is much nicer in version 4,
and the information is needed to support enhanced text mode for x11.

2) X11_waitforinput() now waits for the X-server to reply.

3) If terminal input comes in while it is waiting, X11_waitforinput() wakes
up, notices that it isn't ready to handle new terminal input yet, and then
loops back to the loop to wait again for the X-server to reply.

4) Problem #1 is that we don't want to read the terminal input at this
point because that would cause characters to be lost.  That's
what it used to do, and there were lots of complaints.

5) Problem #2 is that once there is terminal input pending, the select()
call returns immediately every time through the loop, so we sit there
burning CPU cycles like mad until the X-server finally responds.
There were complaints about this also.

6) So as a compromise, I put in a 1 second time delay before going back
to the top of the loop and the select() call.  

Digression: I also added a maximum number of times through the loop,
so that if the X-server never responds (should only happen in the case
of a serious error, perhaps  DISPLAY set to an incorrect value) there
is still a maximum time elapsed before gnuplot returns to accepting
terminal input.  Even more recently I figured out how to allow ctrl-C to
interrupt in the case of a jammed X-server, so this maximum loop count
is no longer strictly necessary but it doesn't by itself hurt anything.

7) In the case of a real X-11 error, like a bad network connection, the
1 second delay is a small price to pay for handling a bad connection
without locking up the CPU.  In the case of a loop, however, even with
a fully responsive X-server the unexpected terminal input causes a
1 second delay every time through the loop. I agree this is bad.

Possible fixes:

Make the time delay much smaller.  On platforms with usleep()
it could be some number of microseconds rather than a full second.
As I recall, I originally I had usleep(100) but I switched to sleep(1)
when I realized that not all platforms provide usleep().

Don't send the font+window size query if mousing is disabled.
Then in the case of an animation loop you could just disable the
mouse before entering the loop, and re-enable it afterwards if
needed.   I don't like this because if your original plot has 
mousing disabled, you will never get the info about window or
font size.  OTOH perhaps replot could handle this differently than
plot (I'd have to look to see whether X11_waitforinput() can
tell the difference). This would actually speed up the animation loop,
since even if the X-server is responding normally the query/response
cycle is kind of slow compared to everything else.

Introduce a new command to toggle the query/response feedback.
The internal mechanism is already in place, because some time 
back I added this as a command line option   'gnuplot -nofeedback'
in order to allow people with problems to turn it off altogether.
I don't like this because it's a command that would be specific to
X11, and would just clutter up the command set.

I tentatively propose to modify the time delay to be
#ifdef HAVE_USLEEP
	usleep(100);
#else
	sleep(1);
#endif

I am open to feedback about whether 'unset mouse' should
disable the size+font query altogether in the case of a replot,
or the question of having an explicit command to disable it.

	Ethan


-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: An observation in animate demo.

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

>On Sunday 09 May 2004 01:27 pm, Hans-Bernhard Broeker wrote:
>  
>
>>On Fri, 7 May 2004, Daniel J Sebald wrote:
>>    
>>
>>>gnuplot> load 'animate.dem'
>>>      
>>>
>
>  
>
>>> But if I hit return once again in the middle of the demo it
>>>will slow down its redraw rate significantly.
>>>      
>>>
>>I think that's the same old X11 multiple-stream handling problem again.
>>    
>>
>
>Sort of.  I added a time delay deliberately to handle a different
>problem, but this is an example of where it imposes a penalty.
>Let me explain what is going on, and you all can speak up as
>to how you want it handled.
>
>The issue is this.
>
>1) When a plot (or replot) is started, the x11 driver sends a query to
>the X-server asking it some basic information like how big is the
>window and what size is the font being used.  This allows much better
>plot layout. Because of this the x11 layout is much nicer in version 4,
>and the information is needed to support enhanced text mode for x11.
>
>2) X11_waitforinput() now waits for the X-server to reply.
>
>3) If terminal input comes in while it is waiting, X11_waitforinput() wakes
>up, notices that it isn't ready to handle new terminal input yet, and then
>loops back to the loop to wait again for the X-server to reply.
>
>4) Problem #1 is that we don't want to read the terminal input at this
>point because that would cause characters to be lost.  That's
>what it used to do, and there were lots of complaints.
>
>5) Problem #2 is that once there is terminal input pending, the select()
>call returns immediately every time through the loop, so we sit there
>burning CPU cycles like mad until the X-server finally responds.
>There were complaints about this also.
>
>6) So as a compromise, I put in a 1 second time delay before going back
>to the top of the loop and the select() call.  
>

Well, actually, this makes the likelihood of a character loss much 
smaller.  There could still be a problem if for some reason the x11 
queries take more than a second, say the system is off doing a lengthy 
hard drive access.  Is this correct?

>Digression: I also added a maximum number of times through the loop,
>so that if the X-server never responds (should only happen in the case
>of a serious error, perhaps  DISPLAY set to an incorrect value) there
>is still a maximum time elapsed before gnuplot returns to accepting
>terminal input.  Even more recently I figured out how to allow ctrl-C to
>interrupt in the case of a jammed X-server, so this maximum loop count
>is no longer strictly necessary but it doesn't by itself hurt anything.
>
>7) In the case of a real X-11 error, like a bad network connection, the
>1 second delay is a small price to pay for handling a bad connection
>without locking up the CPU.  In the case of a loop, however, even with
>a fully responsive X-server the unexpected terminal input causes a
>1 second delay every time through the loop. I agree this is bad.
>
>Possible fixes:
>
>Make the time delay much smaller.  On platforms with usleep()
>it could be some number of microseconds rather than a full second.
>As I recall, I originally I had usleep(100) but I switched to sleep(1)
>when I realized that not all platforms provide usleep().
>
 >

>Don't send the font+window size query if mousing is disabled.
>Then in the case of an animation loop you could just disable the
>mouse before entering the loop, and re-enable it afterwards if
>needed.   I don't like this because if your original plot has 
>mousing disabled, you will never get the info about window or
>font size.  OTOH perhaps replot could handle this differently than
>plot (I'd have to look to see whether X11_waitforinput() can
>tell the difference). This would actually speed up the animation loop,
>since even if the X-server is responding normally the query/response
>cycle is kind of slow compared to everything else.
>
 >

>Introduce a new command to toggle the query/response feedback.
>The internal mechanism is already in place, because some time 
>back I added this as a command line option   'gnuplot -nofeedback'
>in order to allow people with problems to turn it off altogether.
>I don't like this because it's a command that would be specific to
>X11, and would just clutter up the command set.
>

I know developers are against this.  But the issue of being X11 specific 
doesn't bother me.  I'd say trust your instinct for what to do, but wait 
if or until you have an nice solution in mind for which you won't mind 
breaking the current set up.

>I tentatively propose to modify the time delay to be
>#ifdef HAVE_USLEEP
>	usleep(100);
>#else
>	sleep(1);
>#endif
>

Not completely comfortable with that for the simple fact that it is 
increasing the likelihood of a discarded character.  However, at this 
point coming off the 4.0 release I think this would be a reasonable 
solution and then maybe put mouse mods on hold for a while because other 
things might be worth adding and testing.

Am I right in thinking that now that the developers have put a lot of 
work into a stable 4.0 after a long long duration of developments, it 
might be easier to get successive versions ready?  But not too often.  
:-)  So perhaps some smaller changes would be in order.  Coming up with 
a target date for a next release candidate, say 1 year from now, might 
be good.  Also, establishing and prioritizing some general goals for 
that release would help.  (I could offer some if people want to start 
discussing that.)

>I am open to feedback about whether 'unset mouse' should
>disable the size+font query altogether in the case of a replot,
>or the question of having an explicit command to disable it.
>

Eh.  A command the user has to know when and how to apply and it may 
behave differently on different platforms?  Nah.

Dan

-- 

Dan Sebald
email: daniel DOT sebald AT ieee DOT org
URL: http://acer-access DOT com/~dsebald AT acer-access DOT com/

Re: An observation in animate demo.

[Ethan A M. <merritt@u.washington.edu>]

On Sunday 09 May 2004 11:12 pm, Daniel J Sebald wrote:
> Ethan A Merritt wrote:
>
> >I tentatively propose to modify the time delay to be
> >#ifdef HAVE_USLEEP
> >	usleep(100);
> >#else
> >	sleep(1);
> >#endif
>
> Not completely comfortable with that for the simple fact that it is
> increasing the likelihood of a discarded character.

No, I think I haven't explained it well enough.  You will not lose
a character until waitforinput() gives up and leaves the loop
entirely. It is still possible to loop as many times as you like
before giving up.   Total wait time = time per loop X # of iterations.
If you still have no response from the X server after several
seconds, I think losing a character is the smaller part of your
problem at that point.

If things are functioning properly, then a loop like the one in
animate.dem will only go through the waitforinput() loop once
per replot even if there is unexpected terminal input. So making
the per-loop delay shorter should take care of the problem.


-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: An observation in animate demo.

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

>On Sunday 09 May 2004 11:12 pm, Daniel J Sebald wrote:
>  
>
>>Ethan A Merritt wrote:
>>
>>    
>>
>>>I tentatively propose to modify the time delay to be
>>>#ifdef HAVE_USLEEP
>>>	usleep(100);
>>>#else
>>>	sleep(1);
>>>#endif
>>>      
>>>
>>Not completely comfortable with that for the simple fact that it is
>>increasing the likelihood of a discarded character.
>>    
>>
>
>No, I think I haven't explained it well enough.  You will not lose
>a character until waitforinput() gives up and leaves the loop
>entirely. It is still possible to loop as many times as you like
>before giving up.   Total wait time = time per loop X # of iterations.
>If you still have no response from the X server after several
>seconds, I think losing a character is the smaller part of your
>problem at that point.
>
>If things are functioning properly, then a loop like the one in
>animate.dem will only go through the waitforinput() loop once
>per replot even if there is unexpected terminal input. So making
>the per-loop delay shorter should take care of the problem.
>

Oh, I see.  The delay is simply to prevent the consumption of all the 
CPU cycles, not to solve the character being discarded.  That problem 
was solved with the original wait protocol.  Yeah, the #ifdef 
HAVE_USLEEP should do.

Dan

Zoom peculiarities

[Daniel J S. <dan...@ie...>]

Here is something odd in the latest CVS version.  In the "using.dem" 
script, for the first plot, with the mouse active, zoom in on say a top 
middle third of the data.  There will be some lines extending below the 
bottom border.  The length of the lines seems to vary upon what area was 
zoomed.  (That is, do this again and the line lengths may be shorter or 
longer by random.)

With the first plot still zoomed, go to the command line, hit return and 
go to the second plot.  The second plot will be zoomed also.  (Should it 
stay zoomed?)  In the window, unzoom the second plot by typing 'u'.  The 
ranges don't seem to be very nice.  The image is small.  The second plot 
script issues a

set xrange [1:8]

Is there a problem with setting the range when in zoomed mode?

Dan

PS:  This next observation comes with a caveat.  I was working on some 
image code at the time so it may have been mea culpa.  But the image 
stuff has been fairly robust lately so I thought I'd mention it. 
 Anyway, I was resizing a window for a plot once and objects started to 
appear in strange random places on the plot.  For example, the text 
strings would be shifted, the key would be in the wrong place.  The next 
two or three plots showed the same thing, getting progressively worse. 
 The problem then corrected itself and things worked fine again.  I 
haven't been able to reproduce this.  But in all my times of messing 
about with Gnuplot I've not seen (or inadvertently caused) X11 behavior 
like that.  So, just a heads up for now.

Mouse coord lag (Re: An observation in animate demo.)

[Daniel J S. <dan...@ie...>]

Ethan,

I'm observing something with mousing now that may have been affected by 
changing the time delay in this discussion from about a month back:

Ethan A Merritt wrote:

>On Sunday 09 May 2004 11:12 pm, Daniel J Sebald wrote:
>  
>
>>Ethan A Merritt wrote:
>>
>>    
>>
>>>I tentatively propose to modify the time delay to be
>>>#ifdef HAVE_USLEEP
>>>	usleep(100);
>>>#else
>>>	sleep(1);
>>>#endif
>>>

In a surprising way, the faster response may have slowed down the 
mousing... but I don't think it is too critical.  What I'm seeing is 
that if one uses the mouse to select an area to zoom, a whole bunch of 
"position text redraws" get queued.  Then it can take a second or two 
for them all to finish drawing before the actual zooming action takes 
place.  To see this, try right-clicking the mouse to bring up the zoom 
box, then shake the mouse around a bit and quickly left-click the mouse. 
 It will take a few seconds for the positions to finish drawing.

It isn't too bothersome, but it is noticeably different from the 
seemingly instantaneous response of a couple months ago.

I haven't pinpointed where the lag is coming from to confirm it is 
associated with the above change, so it could be a different change as well.

Oh, something peculiar though.  By doing the above jostling of the 
mouse, the dashed box seems to be fast responding.  Why would it be that 
the dashed box gets to its final position so quickly while the text 
keeps drawing away?

Dan

Re: Mouse coord lag (Re: An observation in animate demo.)

[Petr M. <mi...@ph...>]

> In a surprising way, the faster response may have slowed down the
> mousing... but I don't think it is too critical.  What I'm seeing is
> that if one uses the mouse to select an area to zoom, a whole bunch of
> "position text redraws" get queued.  Then it can take a second or two
> for them all to finish drawing before the actual zooming action takes
> place.  To see this, try right-clicking the mouse to bring up the zoom
> box, then shake the mouse around a bit and quickly left-click the mouse.
>  It will take a few seconds for the positions to finish drawing.

I could't reproduce this.

--
PM

Re: Mouse coord lag (Re: An observation in animate demo.)

[Ethan M. <merritt@u.washington.edu>]

On Tuesday 15 June 2004 01:42 am, Daniel J Sebald wrote:
> I haven't pinpointed where the lag is coming from to confirm it is
> associated with the above change, so it could be a different change as
> well.

Do you have any rotated text in your figure?
That is the only case that I know for certain has slowed down.
The underlying problem here is the re-drawing algorithm.
I have had a fix queued up, but it's a large enough change that
I felt uneasy about putting it in before the version 4.0 release.

> Oh, something peculiar though.  By doing the above jostling of the
> mouse, the dashed box seems to be fast responding.  Why would it be that
> the dashed box gets to its final position so quickly while the text
> keeps drawing away?

Is it rotated text?

-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

Re: Mouse coord lag (Re: An observation in animate demo.)

[Daniel J S. <dan...@ie...>]

Ethan Merritt wrote:

>Is it rotated text?
>

It isn't rotated text.

I thought I had the date of the changed pinned down somewhat.  However, 
there is a month's time between the two version.  From what I can see, 
there is a version I downloaded on May 4, 2004 in which I see no slow 
text drawing.  Then there is a version I downloaded on May 31, 2004 in 
which I see slow text drawing.

I have attached a "diff" between the two directories... and there are a 
*lot* of changes, so I'm not sure how much this would help.  Perhaps you 
know better what to look for.

Before diving too much into this, let's make sure it is, in fact, a 
change between these two dates that is causing this.  I am fairly 
certain the versions I speak of above are just as downloaded from CVS. 
 (Before making any changes, I always duplicate the "gnuplot" directory 
to "gnuplot-cvs".)

Dan

Re: Mouse coord lag (Re: An observation in animate demo.)

[Daniel J S. <dan...@ie...>]

Daniel J Sebald wrote:

> Before diving too much into this, let's make sure it is, in fact, a 
> change between these two dates that is causing this.  I am fairly 
> certain the versions I speak of above are just as downloaded from CVS. 
> (Before making any changes, I always duplicate the "gnuplot" directory 
> to "gnuplot-cvs".) 


And no one else seems to see this, so that is making me wonder too.

Dan

Re: Mouse coord lag (Re: An observation in animate demo.)

[Ethan M. <merritt@u.washington.edu>]

On Tuesday 15 June 2004 08:34 am, Ethan Merritt wrote:
> On Tuesday 15 June 2004 01:42 am, Daniel J Sebald wrote:
> > I haven't pinpointed where the lag is coming from to confirm it is
> > associated with the above change, so it could be a different change as
> > well.
>
> The underlying problem here is the re-drawing algorithm.
> I have had a fix queued up, but it's a large enough change that
> I felt uneasy about putting it in before the version 4.0 release.

I have replicated your observation, and I can confirm that
the patch fixes it.  If it bothers you, you can apply the attached
patch to gplt_x11.c.   This patch is from Jay Painter, and I don't
fully understand it, which is why I was reluctant to insert it into
version 4.0 late in the game.



-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

Re: Mouse coord lag (Re: An observation in animate demo.)

[Daniel J S. <dan...@ie...>]

Ethan Merritt wrote:

>I have replicated your observation, and I can confirm that
>the patch fixes it.  If it bothers you, you can apply the attached
>patch to gplt_x11.c.   This patch is from Jay Painter, and I don't
>fully understand it, which is why I was reluctant to insert it into
>version 4.0 late in the game.
>
Well, I can't get the patch to work on the latest CVS version.  The diff file breaks up the hunks in a strange way where "#ifdef USE_MOUSE" and "#if 0" are used.

There are a number of modifications in the patch, but it seems to me the new core change within is the use of XCheckEvent, i.e.,

+    while (XCheckTypedWindowEvent(dpy, event->xany.window, ConfigureNotify, event));

What this does, I believe, is keep getting ConfigureNotify type events 
(and discarding after getting) until there are none left in the queue. 
 At that point the last one that was retrieved (XCheckEvent doesn't 
modify "event" unless there is a valid one present) is the one that is 
acted upon.  I'm assuming then with this change the mouse events that 
are queued, except for the most recent, are discarded.

As for all the other smaller changes, I can't tell what the 
ramifications are without being able to patch the file.

Dan

Re: Mouse coord lag (Re: An observation in animate demo.)

[Ethan M. <merritt@u.washington.edu>]

On Tuesday 15 June 2004 10:28 am, Daniel J Sebald wrote:
> I found the location of HAVE_USLEEP in the most recent CVS version and
> commented out the "usleep(100)" code.  That removes the "lag" in text
> drawing.

Apologies, Dan.  You are correct that the usleep() call is causing the
problem.  I went astray because I didn't realize that different linux kernels
implement the underlying operating system call differently, and I imagine
there is variation in other OS implementations as well.

I originally tested this code on a system that truly counts out the
requested number of microseconds.  But most out-of-the-box
linux systems guarantee only that *at least* that much time elapses,
and in fact return on the next clock tick (1/60 of a seconds).  This is
no good.

So it seems the cure can be worse than the disease in this case.
(Remember the original problem only manifested if the user 
suspended gnuplot without backgrounding it as well).  I will try to
find a different cure, but if I can't I will just back out the usleep() call
altogether.


-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195