Menu
▾
▴
gnuplot-beta
|
From: Jun T. <tak...@kb...> - 2016-07-16 06:05:44
Attachments:
alldem-aqua.patch
|
With terminal aqua, nonlinear3.dem takes *very* long time. It took about 8 minutes on my Mac. This is a problem of Aquaterm.app (or its API); it is designed for vector graphics (and works very well for vector graphics), but can't handle bitmap graphics properly (slow, and uses lots of memory). Since most Mac users would think gnuplot has gone into an infinite loop and try to kill gnuplot (or Aquaterm) before the demo finishes, I think it would better to skip this demo in all.dem, as in the attached patch. |
|
From: Daniel J S. <dan...@ie...> - 2016-07-16 06:20:24
|
On 07/16/2016 01:05 AM, Jun T. wrote: > With terminal aqua, nonlinear3.dem takes *very* long time. > It took about 8 minutes on my Mac. > > This is a problem of Aquaterm.app (or its API); it is designed > for vector graphics (and works very well for vector graphics), > but can't handle bitmap graphics properly (slow, and uses > lots of memory). Does the 'image.dem' example take so long as well? Could your computer be running low on memory causing your system to use disk swap? Dan |
|
From: Jun T. <tak...@kb...> - 2016-07-17 16:34:55
|
2016/07/16 15:20, Daniel J Sebald <dan...@ie...> wrote: > Does the 'image.dem' example take so long as well?, No, it works normally. I think use of the 'pixels' keyword in probably_tux.dem is causing the slowness. If I remove the 'pixels' from the last line of probably_tux.dem then nonlinear3.dem ends quickly, but the nonlinear axis scaling does not work anymore. > Could your computer be running low on memory causing your system to use disk swap? No, my Mac has enough free memory while running the demos. It seems gnuplot uses about 0.5GB of memory for nonlinear3.dem. |
|
From: Daniel J S. <dan...@ie...> - 2016-07-17 17:08:30
|
On 07/17/2016 11:34 AM, Jun T. wrote: > > 2016/07/16 15:20, Daniel J Sebald <dan...@ie...> wrote: > >> Does the 'image.dem' example take so long as well?, > > No, it works normally. > > I think use of the 'pixels' keyword in probably_tux.dem is causing > the slowness. If I remove the 'pixels' from the last line of > probably_tux.dem then nonlinear3.dem ends quickly, but the nonlinear > axis scaling does not work anymore. Ah, yes, the vector-based pixels, but that is contrary to what you had concluded. In that mode, gnuplot is not using bitmap graphics, but vector graphics (think of a pixel as being a rectangle...or parallelogram if rotated in space). But why so long? There is an example in image.dem that I think falls back to vector-based. Also 'image2.dem' has several of those. Anything where the image is projected non-parallel to the viewing plane will fall back to drawing polygons for pixels. >> Could your computer be running low on memory causing your system to use disk swap? > > No, my Mac has enough free memory while running the demos. > It seems gnuplot uses about 0.5GB of memory for nonlinear3.dem. Right. That isn't the issue. I'm still surprised that this takes so long. If none of the demos in 'image.dem' and 'image2.dem' are glacial-pace slow, then I wonder if something is happening in which the nonlinearity creates a rectangular with peculiar coordinates that causes an algorithm aqua is using to veer off course. What happens if you temporarily make this change in probably_tux.dem: #f(x) = norm((x-center)/sigma) #g(x) = sigma*invnorm(x)+center f(x) = x g(x) = x Dan |
|
From: Daniel J S. <dan...@ie...> - 2016-07-17 17:18:59
|
On 07/17/2016 12:08 PM, Daniel J Sebald wrote: > On 07/17/2016 11:34 AM, Jun T. wrote: >> [snip] > I'm still surprised that this takes so long. If none of the demos in > 'image.dem' and 'image2.dem' are glacial-pace slow, then I wonder if > something is happening in which the nonlinearity creates a rectangular > with peculiar coordinates that causes an algorithm aqua is using to veer > off course. > > What happens if you temporarily make this change in probably_tux.dem: > > #f(x) = norm((x-center)/sigma) > #g(x) = sigma*invnorm(x)+center > f(x) = x > g(x) = x I'm guessing that is pretty much the same as leaving off nonlinear x and nonlinear y. We should probably create a bug report for this Jun. Maybe a NaN or something odd is being sent to the Aqua terminal that it doesn't like. Dan |
|
From: Jun T. <tak...@kb...> - 2016-07-19 05:04:26
|
On 2016/07/18, at 2:08, Daniel J Sebald <dan...@ie...> wrote: > > Ah, yes, the vector-based pixels, but that is contrary to what you had concluded. In that mode, gnuplot is not using bitmap graphics, but vector graphics Yes, gnuplot creates the image as a vector graphics, but I've been thinking that other terminals, such as qt, internally convert it into a bitmap while aqua treats it as a vector graphics. But yes, even if this is the case, I don't know why aqua is so slow. >> It seems gnuplot uses about 0.5GB of memory for nonlinear3.dem. qt or wxt uses much smaller amount of memory. As for the CPU usage, Aquaterm.app uses about 95% of cpu while gnuplot uses 5 to 10% (100% means one cpu core is fully used). So it seems gnuplot creates large amount of data and send it to Aquaterm.app, while the latter uses large amount of cpu to process the data. I will do some profiling later (if I have some time). > I'm still surprised that this takes so long. If none of the demos in 'image.dem' and 'image2.dem' are glacial-pace slow, They don't use the 'pixels' option for plot. > What happens if you temporarily make this change in probably_tux.dem: > > #f(x) = norm((x-center)/sigma) > #g(x) = sigma*invnorm(x)+center > f(x) = x > g(x) = x Just the following command (without any scaling etc.) takes a long time: plot 'blutux.rgb' binary array=(128,128) flipy format='%uchar' with rgbimage pixels |
|
From: Daniel J S. <dan...@ie...> - 2016-07-19 06:30:14
|
On 07/19/2016 12:04 AM, Jun T. wrote: >>> It seems gnuplot uses about 0.5GB of memory for nonlinear3.dem. > > qt or wxt uses much smaller amount of memory. > > As for the CPU usage, Aquaterm.app uses about 95% of cpu while gnuplot uses 5 to 10% > (100% means one cpu core is fully used). So it seems gnuplot creates large amount > of data and send it to Aquaterm.app, while the latter uses large amount of cpu > to process the data. > > I will do some profiling later (if I have some time). > >> I'm still surprised that this takes so long. If none of the demos in 'image.dem' and 'image2.dem' are glacial-pace slow, > > They don't use the 'pixels' option for plot. That's true. But they do use the same polygon code ultimately. > Just the following command (without any scaling etc.) takes a long time: > > plot 'blutux.rgb' binary array=(128,128) flipy format='%uchar' with rgbimage pixels How long does the following plot take?: plot 'blutux.rgb' binary array=(128,128) flipy rotation=30d format='%uchar' with rgbimage Dan |
|
From: Daniel J S. <dan...@ie...> - 2016-07-19 07:28:35
|
On 07/19/2016 01:29 AM, Daniel J Sebald wrote: > On 07/19/2016 12:04 AM, Jun T. wrote: > >>>> It seems gnuplot uses about 0.5GB of memory for nonlinear3.dem. >> >> qt or wxt uses much smaller amount of memory. >> >> As for the CPU usage, Aquaterm.app uses about 95% of cpu while gnuplot uses 5 to 10% >> (100% means one cpu core is fully used). So it seems gnuplot creates large amount >> of data and send it to Aquaterm.app, while the latter uses large amount of cpu >> to process the data. >> >> I will do some profiling later (if I have some time). >> >>> I'm still surprised that this takes so long. If none of the demos in 'image.dem' and 'image2.dem' are glacial-pace slow, >> >> They don't use the 'pixels' option for plot. > > That's true. But they do use the same polygon code ultimately. > > >> Just the following command (without any scaling etc.) takes a long time: >> >> plot 'blutux.rgb' binary array=(128,128) flipy format='%uchar' with rgbimage pixels > > How long does the following plot take?: > > plot 'blutux.rgb' binary array=(128,128) flipy rotation=30d > format='%uchar' with rgbimage If the above isn't as slow as your example using "pixels" option, then the difference is that one is using parallelograms and the other rectangles. The example that uses rectangles (the slow example), calls (*term->fillbox). In the aquaterm.c file fillbox has a log message at the front that filled_polygon does not, i.e., LOG(@"\nstyle=%d\nstyle & 0xf = %d\nfillpar=%d\n", style, style & 0xf, style >> 4); Whenever some numbers are changed to ASCII it expands the data quite a bit. Could it be that for each pixel one of the log messages is being put in some type of AQUA debugger? And that debugger could be processing a lot of data, and it might have to dynamically build a list of log messages so could be inefficient. All a guess. Do you know if you have debugging active? http://www.roseindia.net/tutorial/iphone/examples/nslog/index.html Is there some way of undefining LOGGING and recompile?: #ifdef LOGGING #define LOG NSLog #else #define LOG NOOP_ #endif /* LOGGING */ Dan |
|
From: Jun T. <tak...@kb...> - 2016-07-19 13:18:19
|
On 2016/07/19, at 15:29, Daniel J Sebald <dan...@ie...> wrote: >> >> They don't use the 'pixels' option for plot. > > That's true. But they do use the same polygon code ultimately. If 'pixels' is not specified, then process_image() calls AQUA_image(), while with 'pixels' it calls AQUA_boxfill(), which is very slow. > How long does the following plot take?: > > plot 'blutux.rgb' binary array=(128,128) flipy rotation=30d format='%uchar' with rgbimage It takes just a few seconds. As you know, AQUA_filled_polygon() is called in this case. On 2016/07/19, at 16:28, Daniel J Sebald <dan...@ie...> wrote: > Is there some way of undefining LOGGING and recompile?: LOGGING is not defined by default. I did some profiling, and has found that gnuplot uses lots of CPU time at [adapter eraseRect:scaledRect]; aquaterm.trm, line 662 and Aquaterm.app spends most of the CPU time processing these eraseRect requests from gnuplot. AQUA_filled_polygon() does not (can not) have this eraseRect call, so it is not slow. If I comment out the line 662 of aquaterm.trm, then nonlinear3.dem takes about 8 to 10 seconds (instead of 8 minutes), and it *seems* to give the same plot. But I guess eraseRect has been added intentionally to get better results in some cases, and rather hesitate to remove this line. Another possibility is to include TERM_POLYGON_PIXELS in term->flags of aqua terminal. I also tried this, and it was virtually as fast as removing the line 662. Maybe this would be safe enough? TERM_POLYGON_PIXELS is not used other than at line 4977 of graphics.c. NOTE: I *guess* the eraseRect is slow due to the following reason. Suppose there is already a rectangle R0 with 4 corners at (0,0)-(0,100)-(100,100)-(100,0). If eraseRect is called with a rectangle Rx=(50,50)-(50,150)-(150,150)-(150,50) then it needs to modify R0 into a polygon with 6 vertices at (0,0)-(100,0)-(100,50)-(50,50)_(50,100)-(0,100). If there are many rectangles/polygons already in the plot, then eraseRect must find *all* the intersections of these pre-existing rectangles/polygons with the rectangle Rx. |
|
From: Ethan A M. <sf...@us...> - 2016-07-19 17:20:16
|
Brief summary: The command "plot <foo> with image pixels" tells gnuplot to render individual pixels rather than sending a bitmap of the entire image. This has a number of uses. The current bug report comes from demo nonlinear3.dem, in which pixels must be rendered individually because they are not of uniform size. Bug: This demo is horribly slow (8 minutes) using the aqua terminal On Tuesday, 19 July, 2016 22:18:09 Jun T. wrote: > > I did some profiling, and has found that gnuplot uses lots of CPU time at > > [adapter eraseRect:scaledRect]; aquaterm.trm, line 662 > > and Aquaterm.app spends most of the CPU time processing these eraseRect > requests from gnuplot. > > AQUA_filled_polygon() does not (can not) have this eraseRect call, so it > is not slow. > > If I comment out the line 662 of aquaterm.trm, then nonlinear3.dem takes > about 8 to 10 seconds (instead of 8 minutes), and it *seems* to give > the same plot. But I guess eraseRect has been added intentionally to get > better results in some cases, and rather hesitate to remove this line. > > Another possibility is to include TERM_POLYGON_PIXELS in term->flags of > aqua terminal. I also tried this, and it was virtually as fast as > removing the line 662. > Maybe this would be safe enough? TERM_POLYGON_PIXELS is not used other > than at line 4977 of graphics.c. It is fine to set set TERM_POLYGON_PIXELS. That flag is an advisory to the core code that the term->filled_polygon() routine is prefered to the term->boxfill() routine for whatever reason. In this case the reason is execution speed. I do not know why the aqua boxfill() routine calls eraseRect. Other terminals do not have an equivalent call. Perhaps the original author remembers why (cc-ed to Per Persson) For now I will add the TERM_POLYGON_PIXELS flag, but it would be nice to fix/improve AQUA_boxfill() also. > > NOTE: > I *guess* the eraseRect is slow due to the following reason. Suppose there > is already a rectangle R0 with 4 corners at (0,0)-(0,100)-(100,100)-(100,0). > If eraseRect is called with a rectangle Rx=(50,50)-(50,150)-(150,150)-(150,50) > then it needs to modify R0 into a polygon with 6 vertices at > (0,0)-(100,0)-(100,50)-(50,50)_(50,100)-(0,100). > If there are many rectangles/polygons already in the plot, then eraseRect > must find *all* the intersections of these pre-existing rectangles/polygons > with the rectangle Rx. Ethan |
|
From: Daniel J S. <dan...@ie...> - 2016-07-19 17:36:34
|
On 07/19/2016 08:18 AM, Jun T. wrote: >> How long does the following plot take?: >> >> plot 'blutux.rgb' binary array=(128,128) flipy rotation=30d format='%uchar' with rgbimage > > It takes just a few seconds. > As you know, AQUA_filled_polygon() is called in this case. > > > On 2016/07/19, at 16:28, Daniel J Sebald <dan...@ie...> wrote: >> Is there some way of undefining LOGGING and recompile?: > > LOGGING is not defined by default. > > > I did some profiling, and has found that gnuplot uses lots of CPU time at > > [adapter eraseRect:scaledRect]; aquaterm.trm, line 662 > > and Aquaterm.app spends most of the CPU time processing these eraseRect > requests from gnuplot. > > AQUA_filled_polygon() does not (can not) have this eraseRect call, so it > is not slow. > > If I comment out the line 662 of aquaterm.trm, then nonlinear3.dem takes > about 8 to 10 seconds (instead of 8 minutes), and it *seems* to give > the same plot. But I guess eraseRect has been added intentionally to get > better results in some cases, and rather hesitate to remove this line. > > Another possibility is to include TERM_POLYGON_PIXELS in term->flags of > aqua terminal. I also tried this, and it was virtually as fast as > removing the line 662. > Maybe this would be safe enough? TERM_POLYGON_PIXELS is not used other > than at line 4977 of graphics.c. > > NOTE: > I *guess* the eraseRect is slow due to the following reason. Suppose there > is already a rectangle R0 with 4 corners at (0,0)-(0,100)-(100,100)-(100,0). > If eraseRect is called with a rectangle Rx=(50,50)-(50,150)-(150,150)-(150,50) > then it needs to modify R0 into a polygon with 6 vertices at > (0,0)-(100,0)-(100,50)-(50,50)_(50,100)-(0,100). > If there are many rectangles/polygons already in the plot, then eraseRect > must find *all* the intersections of these pre-existing rectangles/polygons > with the rectangle Rx. OK, I think you've found the CPU drain. Erasing the rectangles is probably second order growth because for each rectangle drawn, the AQUA driver probably goes through the whole list of elements to see if there is overlap. I conclude that from this comment in the aquaterm code: https://sourceforge.net/p/aquaterm/mailman/aquaterm-commit/?viewmonth=200308 /*" Add a filled rectangle. Should normally be preceeded with #eraseRect: to remove any objects that will be covered by aRect."*/ For example, an N pixel image will take on the order SUM_{i=1}^N i * (i-1) For small N this probably isn't too bad, but for N = 128*128 = 16384 this probably gets to be a huge number, actually 1.4660e+12, evaluating the sum via loop in Octave. So there is probably a lot of comparison code that needs to be run that many times (and most of the time in this case it isn't doing anything). I think that eraseRect can be removed. We generally don't use this approach in other terminals, but instead just allow things to overlap and let the terminal's renderer deal with what is visible for a given output pixel and what is not. In the case of an image using rectangles for pixels, we at least know that individual pixels don't overlap. Of course, that doesn't account for anything else that has already been drawn from some other plot feature or will be drawn after, but if we want to use such an approach, gnuplot can probably do a more optimum job of it because of apriori knowledge concerning depth ordering in hidden line removal and so on. Interestingly, Qt has a similar type of routine, QPainter::eraseRect, but it approaches things slightly different. It simply draws a filled rectangle using the background color. That's not as computationally demanding. There is one other erase in aquaterm, but it is in the initialization routine AQUA_graphics(). That should stay because its role is to simply clear the whole screen of any drawing elements. Ethan can think this one over and make the change for you if it seems innocuous enough. Dan |
|
From: Daniel J S. <dan...@ie...> - 2016-07-19 20:01:37
|
On 07/19/2016 12:36 PM, Daniel J Sebald wrote:
> For example, an N pixel image will take on the order
>
> SUM_{i=1}^N i * (i-1)
Make that
SUM_{i=1}^N (i-1) = (N-1) * N/2
so that's 1.342e8. Still a large number.
Dan
|
|
From: Per P. <per...@ma...> - 2016-07-20 20:48:04
|
(Away from computer, decade old code, etc., ...) IIRC, I hesitantly added the erase rect code since some demo(?) relied on "overprinting" to erase previously drawn stuff, or possibly edge effect occurring when drawing multiple layers under anti-aliasing. My guess is that taking out the call to erase rect is safe for 99.99% of the use cases. That said, building an image using individual "pixels" is never going to be fast with the aqua terminal since it will create an object graph of all individual items so that the can be manipulated "post drawing". For speed, a better strategy is to create a bitmap image in the driver an push that as a unit to aquaterm. (Think that's how it's done elsewhere in the driver for som other command?) Hope this helps, Per > 19 juli 2016 kl. 19:17 skrev Ethan A Merritt <sf...@us...>: > > Brief summary: > > The command "plot <foo> with image pixels" tells gnuplot to render > individual pixels rather than sending a bitmap of the entire image. > This has a number of uses. The current bug report comes from demo > nonlinear3.dem, in which pixels must be rendered individually because > they are not of uniform size. > > Bug: This demo is horribly slow (8 minutes) using the aqua terminal > > >> On Tuesday, 19 July, 2016 22:18:09 Jun T. wrote: >> >> I did some profiling, and has found that gnuplot uses lots of CPU time at >> >> [adapter eraseRect:scaledRect]; aquaterm.trm, line 662 >> >> and Aquaterm.app spends most of the CPU time processing these eraseRect >> requests from gnuplot. >> >> AQUA_filled_polygon() does not (can not) have this eraseRect call, so it >> is not slow. >> >> If I comment out the line 662 of aquaterm.trm, then nonlinear3.dem takes >> about 8 to 10 seconds (instead of 8 minutes), and it *seems* to give >> the same plot. But I guess eraseRect has been added intentionally to get >> better results in some cases, and rather hesitate to remove this line. >> >> Another possibility is to include TERM_POLYGON_PIXELS in term->flags of >> aqua terminal. I also tried this, and it was virtually as fast as >> removing the line 662. >> Maybe this would be safe enough? TERM_POLYGON_PIXELS is not used other >> than at line 4977 of graphics.c. > > It is fine to set set TERM_POLYGON_PIXELS. That flag is an advisory to > the core code that the term->filled_polygon() routine is prefered to the > term->boxfill() routine for whatever reason. In this case the reason is > execution speed. > > I do not know why the aqua boxfill() routine calls eraseRect. > Other terminals do not have an equivalent call. > Perhaps the original author remembers why > (cc-ed to Per Persson) > > For now I will add the TERM_POLYGON_PIXELS flag, but it would be nice to > fix/improve AQUA_boxfill() also. > >> >> NOTE: >> I *guess* the eraseRect is slow due to the following reason. Suppose there >> is already a rectangle R0 with 4 corners at (0,0)-(0,100)-(100,100)-(100,0). >> If eraseRect is called with a rectangle Rx=(50,50)-(50,150)-(150,150)-(150,50) >> then it needs to modify R0 into a polygon with 6 vertices at >> (0,0)-(100,0)-(100,50)-(50,50)_(50,100)-(0,100). >> If there are many rectangles/polygons already in the plot, then eraseRect >> must find *all* the intersections of these pre-existing rectangles/polygons >> with the rectangle Rx. > > Ethan |
