gnuplot-beta — Developers and beta testers of new gnuplot code

Re: Postscript and PM3D file size.

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>On Thu, 4 Mar 2004, Daniel J Sebald wrote:
>
>[...]
>
>  
>
>>I must learn how to use this cscope.  (I've bookmarked the info page...  
>>hey, why the water jostling around in the magnifying glass guy's head?
>>Is he/she/it supposed to be half magnifying glass, half periscope, i.e.,
>>sea scope?)
>>    
>>
>
>I've no idea --- I didn't design the mascot, and the person who made it
>didn't leave an explanation of the intent.  But your guess appears quite
>likely to be true.
>

Well, I guess water should actually "slosh", not "jostle".  So anyway, 
why doesn't Gnuplot have a mascot?  :)

Dan

Re: Postscript and PM3D file size.

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>On Wed, 3 Mar 2004, Daniel J Sebald wrote:
>  
>
>>Oh yeah, I forgot about "matrix".  The issue there is that "matrix" is 
>>designed for 3d data, i.e., splot.  (List members, if anything I say 
>>here is incorrect, please correct me.)  
>>    
>>
>
>I was going to about 'matrix', but you seemed to be progressing so nicely,
>I though I'ld let you you find that one yourself ;->
>
>  
>
>>There is a routine in "datafile.c" called df_3dmatrix.  I think it only
>>get's called from "plot3d.c".  
>>    
>>
>
>Don't just think, check it.  Tools like "cscope" will let you answer such
>queries almost in an instant:
>
>gnuplot/src> cscope -L0 df_3dmatrix
>datafile.h <global> 100 int df_3dmatrix __PROTO((struct surface_points *, int ));
>datafile.c df_3dmatrix 1268 df_3dmatrix(this_plot, need_palette)
>plot3d.c get_3ddata 652 xdatum = df_3dmatrix(this_plot, NEED_PALETTE(this_plot));
>
>You can even let it show you only the calls to a function:
>
>gnuplot/src> cscope -L3 df_3dmatrix
>plot3d.c get_3ddata 652 xdatum = df_3dmatrix(this_plot, NEED_PALETTE(this_plot));
>

I must learn how to use this cscope.  (I've bookmarked the info page... 
hey, why the water jostling around in the magnifying glass guy's head? 
 Is he/she/it supposed to be half magnifying glass, half periscope, 
i.e., sea scope?)

>>not the mode is "splot" or "plot".  In other words, if I'm not mistaken,
>>Gnuplot should not allow "matrix" to be used in "plot" mode.  
>>    
>>
>
>The current official version probably shouldn't, right.  So let's try...
>
>gnuplot> p 'using.dat' matrix u ($0):2:3 w e
>
>No error, no warning, and a rather stupid plot. How's that for a rather
>surprising undocumented feature?
>

Yes...  As a temporary fix for 4.0, the following placed near the start 
of the eval_plots() routine inside plot2d.c will rule out using 'matrix' 
option with 'splot'.  (Patch attached.)

    /* Temporary hack to rule out 'matrix' for 2d plots.  'matrix' is 
currently
       tested for inside df_open() which doesn't know the option currently
       can't be used with 'plot'.  This should be cleaned up post 4.0 
and this
       chunk of code tossed out. */
    while (!END_OF_COMMAND) {
    if (almost_equals(c_token, "mat$rix"))
        int_error(c_token, "option 'matrix' not supported for 2d plots");
    c_token++;
    }
    c_token = begin_token;

    /* ** First Pass: Read through data files ***

>>Actually, in the image patch, I modified things so that a mode variable 
>>is passed into that routine so it is possible to identify whether 
>>"splot" or "plot" was issued.  I probably needed that information for 
>>something else, but it could be used to rule out "matrix" when issued 
>>with "plot".  
>>    
>>
>
>But it should only be ruled out if that was not a 'plot with pixels',
>right?
>

Correct.

>>in previous discussions, that 2d and 3d formats have diverged in some 
>>ways.  
>>    
>>
>
>I suspect they have never started off a common ground in the first place.
>After looking at all the code for prolonged times, you get the impression
>3D must have been grafted onto an existing 2D-only plotting program as an
>afterthought.
>

Yes, that is my impression as well.

>>I think the "datafile.c" and related routines could be cleaned up 
>>a great deal, focusing more on just getting data from a file into 
>>Gnuplot and not so much on what the eventually plotting mode will look like.
>>    
>>
>
>Agreed.  After 4.0 is out, we could split datafile.c into two: datafile.c
>and "using.c".  The latter would be in charge of all the translation from
>data records into gnuplot "points", i.e. treat options 'using', 'every',
>'index' and 'smooth' (which really should be renamed to 'filter' some
>time).
>

That should be good.

>>Do developers here agree that, for large files, binary data in "plot" 
>>mode is preferable to ascii matrix?
>>    
>>
>
>Preferrable: yes.  But not so much so that it would be a strong argument
>against allowing 'matrix' at least in those 2D plots that really are 3D
>(i.e. "plot with image").
>
>I think we actually should allow 'matrix' for such plots.  It's about
>equally sensible as allowing binary files for 2D plots.'
>

Sure.  I think 'matrix' should be valid whenever appropriate.  If 
someone has a small set of data, fine.  And if they have a large set of 
data, fine too.  But I think it is just a better habit to work with 
compact data and not 20 M ASCII files.

Dan

Re: default tic label format

[Hans-Bernhard B. <br...@ph...>]

On Wed, 3 Mar 2004, Pieter-Tjerk de Boer wrote:

> Hello,
> 
> While trying the 3.8k.1 version of gnuplot, I noticed that the default
> format for the labels on the axis is "% g".
> This has two (minor) disadvantages, compared to simple "%g" as it was
> in version 3.7:

Note: it's '% g' there, too, as of 3.7 patchlevel 3.0.

> - labels on horizontal axes are not centered nicely (the invisible
>   leading space is included in the centering calculation).
> - labels on the y2 axis (if used) start farther away from the axis
>   than the y1 axis labels do.

But note that both of these affect only *positive* ticks.  To stick
with the y2 axis as the most visibly affected one, the choice we have,
for a symmetric axis range, is one of:

%g:
5
0
-5

% g:
 5
 0
-5

%+g:
+5
+0
-5

Looking at these, I think "% g" indeed looks better than %g or %+g would.

In the end, this really is one of the personal preference issue that
can't be solved by the program itself.  Such personal default adjustments 
is what you have ~/.gnuplot or gnuplot.ini for (depending on your 
platform).

> Obviously, I would suggest changing to "%g" before the release of 4.0.

For the record: I suggest we keep '% g'.


-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: Postscript and PM3D file size.

[Hans-Bernhard B. <br...@ph...>]

On Wed, 3 Mar 2004, Daniel J Sebald wrote:
> 
> Oh yeah, I forgot about "matrix".  The issue there is that "matrix" is 
> designed for 3d data, i.e., splot.  (List members, if anything I say 
> here is incorrect, please correct me.)  

I was going to about 'matrix', but you seemed to be progressing so nicely,
I though I'ld let you you find that one yourself ;->

> There is a routine in "datafile.c" called df_3dmatrix.  I think it only
> get's called from "plot3d.c".  

Don't just think, check it.  Tools like "cscope" will let you answer such
queries almost in an instant:

gnuplot/src> cscope -L0 df_3dmatrix
datafile.h <global> 100 int df_3dmatrix __PROTO((struct surface_points *, int ));
datafile.c df_3dmatrix 1268 df_3dmatrix(this_plot, need_palette)
plot3d.c get_3ddata 652 xdatum = df_3dmatrix(this_plot, NEED_PALETTE(this_plot));

You can even let it show you only the calls to a function:

gnuplot/src> cscope -L3 df_3dmatrix
plot3d.c get_3ddata 652 xdatum = df_3dmatrix(this_plot, NEED_PALETTE(this_plot));

> not the mode is "splot" or "plot".  In other words, if I'm not mistaken,
> Gnuplot should not allow "matrix" to be used in "plot" mode.  

The current official version probably shouldn't, right.  So let's try...

gnuplot> p 'using.dat' matrix u ($0):2:3 w e

No error, no warning, and a rather stupid plot. How's that for a rather
surprising undocumented feature?

> Actually, in the image patch, I modified things so that a mode variable 
> is passed into that routine so it is possible to identify whether 
> "splot" or "plot" was issued.  I probably needed that information for 
> something else, but it could be used to rule out "matrix" when issued 
> with "plot".  

But it should only be ruled out if that was not a 'plot with pixels',
right?

> in previous discussions, that 2d and 3d formats have diverged in some 
> ways.  

I suspect they have never started off a common ground in the first place.
After looking at all the code for prolonged times, you get the impression
3D must have been grafted onto an existing 2D-only plotting program as an
afterthought.

> I think the "datafile.c" and related routines could be cleaned up 
> a great deal, focusing more on just getting data from a file into 
> Gnuplot and not so much on what the eventually plotting mode will look like.

Agreed.  After 4.0 is out, we could split datafile.c into two: datafile.c
and "using.c".  The latter would be in charge of all the translation from
data records into gnuplot "points", i.e. treat options 'using', 'every',
'index' and 'smooth' (which really should be renamed to 'filter' some
time).

> Do developers here agree that, for large files, binary data in "plot" 
> mode is preferable to ascii matrix?

Preferrable: yes.  But not so much so that it would be a strong argument
against allowing 'matrix' at least in those 2D plots that really are 3D
(i.e. "plot with image").

I think we actually should allow 'matrix' for such plots.  It's about
equally sensible as allowing binary files for 2D plots.

But this really had better be delayed until after the 4.0 release. We
might want to put this into the "TODO" file, though.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: Postscript and PM3D file size.

[Daniel J S. <dan...@ie...>]

Justace Clutter wrote:

>I have been using this ASCII grid format for quite a while now.  That is
>the definition from the help pages for the matrix datafile format.  Am I
>wrong with this?
>

Oh yeah, I forgot about "matrix".  The issue there is that "matrix" is 
designed for 3d data, i.e., splot.  (List members, if anything I say 
here is incorrect, please correct me.)  There is a routine in 
"datafile.c" called df_3dmatrix.  I think it only get's called from 
"plot3d.c".  There is no way to call it from "plot2d.c".  Inside 
"datafile.c" where the word "matrix" is tested for there seems to be no 
test for whether or not the mode is "splot" or "plot".  In other words, 
if I'm not mistaken, Gnuplot should not allow "matrix" to be used in 
"plot" mode.  In the documentation for "matrix" there is reference to a 
mesh, so I assume there is an "splot" restriction.

Actually, in the image patch, I modified things so that a mode variable 
is passed into that routine so it is possible to identify whether 
"splot" or "plot" was issued.  I probably needed that information for 
something else, but it could be used to rule out "matrix" when issued 
with "plot".  Or, for that matter plot2d.c could be modified to allow 
data files in "matrix" format.  To you Justace, this is all probably 
neither here nor there.  But it gets back to a point I've tried to make 
in previous discussions, that 2d and 3d formats have diverged in some 
ways.  I think the "datafile.c" and related routines could be cleaned up 
a great deal, focusing more on just getting data from a file into 
Gnuplot and not so much on what the eventually plotting mode will look like.

Suffice it to say that "matrix" is currently being ignored in "plot" 
mode.  Sorry.  I suppose it would be possible to hack a quick fix 
together in the image patch (I'm not sure how much work it would take), 
but I'd prefer to not do that.  Instead, I think it would be better for 
the group to discuss how best to clean up the datafile routines post 
4.0, then make the change.  I say that because there is the alternative 
of binary data, and I think it is a better habit to work with binary 
data for large files.

Do developers here agree that, for large files, binary data in "plot" 
mode is preferable to ascii matrix?

Dan




>Justace
>
>On Wed, 2004-03-03 at 10:46, Daniel J Sebald wrote:
>  
>
>>Justace Clutter wrote:
>>
>>    
>>
>>>I like the idea of this pixels based image.  The fact that the image
>>>could be 10 megs does not bother me.
>>>
>>>      
>>>
>>Well, in my case my web space is only 10 M.
>>
>>    
>>
>>> It is the idea that an image could
>>>be 120 megs bothers me.  So I got the patch and applyed.  It did not go
>>>clean as one hunk did not make it.  But I fixed it by hand.  I was
>>>patching against 3.8k.1.  So, I guess that I do not understand the
>>>syntax.  Here is the datafile that I have, or really a sample:
>>>
>>>0.999849 0.652204 0.678995 0.658985 0.585981 0.566624 0.547193 0.615902
>>>0.539023 0.556559 0.547578 0.53746 0.522146 0.535582 0.530735 0.531924
>>>0.554801 0.518362 0.479106 0.527362 0.534836 0.546124 0.522834 0.508155
>>>0.54706 0.469279 0.581292 0.468283 0.535739 0.557979 0.490128 0.536971
>>>0.530735 0.506848 0.523189 0.531732 0.550917 0.484002 0.57789 0.460027
>>>0.556111 0.535427 0.508878 0.558746 0.459494 0.578337 0.45805 0.595234 
>>>
>>>So I have a grid here, 8x6, that I would like displayed.  Each of these
>>>floats would need to be mapped onto a color.  PM3D did this great.  So
>>>when I try to use the matrix modifier with just plot it is not happy,
>>>then I tried to use splot, still not happy.  Here is what I was trying..
>>>
>>>gnuplot> plot 'tmp.data' matrix with image
>>>
>>>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>>>
>>>Warning: empty x range [0.999849:0.999849], adjusting to
>>>[0.989851:1.00985]
>>>
>>>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>>>
>>>gnuplot> splot 'tmp.data' matrix with image
>>>
>>>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>>>
>>>gnuplot> set view map
>>>gnuplot> splot 'tmp.data' matrix with image
>>>
>>>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>>>
>>>gnuplot>
>>>
>>>Can you give me some ideas on how I can use the image style with this
>>>setup?  I looked at the information on the image but it seems pretty
>>>biased toward images like the tux and not array'd data.  Thanks in
>>>advance.
>>>
>>>      
>>>
>>Well, here is the issue.  You are putting the data in ASCII format in 
>>what looks to you like a grid.  However, Gnuplot hasn't been set up to 
>>read ASCII data in such a way.  It is set up to read ordered tuples of 
>>data, such as
>>
>>x1  y1  z1
>>x2  y2  z2
>>...
>>
>>where the position is part of the data specification.
>>
>>Telling Gnuplot what the underlying grid is without specifying it is the 
>>problem.  That is why the "binary" file specifications have extra 
>>qualifiers to indicate grid dimensions, grid spacing, etc.
>>
>>Another alternative for binary is "Gnuplot binary" where the first line 
>>and row of the matrix give the x and y spacing information.  I've 
>>generally not used that for images, but I think it should work.  (Type 
>>"help splot binary" at Gnuplot's command line for more info on that format.)
>>
>>The reason you may want to avoid putting your image in ASCII format is 
>>that you are playing the numbers game again.  Your example above has 9 
>>characters for each pixel element.  1800 x 1800 x 9 or roughly 27 M at a 
>>minimum, and if when you add the (x,y) coordinates it goes to 81 M. 
>> That's too big.  If in the program or software you are using to 
>>generate the data you could multiply by 2^16 and then save the values in 
>>binary as unsigned 16 bit ints you would be better off.  Can you do that?
>>
>>Dan
>>
>>    
>>
>>>Justace
>>>
>>>On Wed, 2004-03-03 at 01:00, Daniel J Sebald wrote:
>>> 
>>>
>>>      
>>>
>>>>Justace Clutter wrote:
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>So,
>>>>>
>>>>>	I have this project where I have computed a bunch of channel
>>>>>correlations.  The entire matrix is like 1800x1800 with one side of the
>>>>>diaganal set to NaN since it is symetric.  Now to the root of the
>>>>>problem.  I use the pm3d portion of gnuplot to create a 2D map of the
>>>>>matrix.  Here is the snipplit of code that I use...
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>><snip>
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>I used the first set size command to try to bring the file size down, of
>>>>>course it did not work.  The resultant file is 120Meg.  This is crazy.
>>>>>Am I doing somehting wrong?
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>Justace,
>>>>
>>>>Am I understanding correctly that you are essentially attempting to 
>>>>display an 1800x1800 matrix as an image?  Ethan's comment is correct 
>>>>that each element, i.e., pixel, will be drawn as a rectangle, i.e., 
>>>>polygon.  There is a lot of ASCII text associated with each rectangle. 
>>>>You might try the image patch on SourceForge (something originally 
>>>>titled "with pixels").  That will put the image into a PostScript file 
>>>>in a compact form.
>>>>
>>>>Realize though that 1800x1800 is a lot of data.  Saved as bytes, that 
>>>>alone would be roughly 3M.  I've tried 1000x1000 images with no 
>>>>problems, but things really do slow down.  So, unless it is necessary to 
>>>>keep such high resolution for the image, your first step might be to 
>>>>down sample the data appropriately.  Keep in mind what your eventual 
>>>>display will be.  (For example, people often put high res photos on 
>>>>their web pages which take close to a minute to download.  However, the 
>>>>display is usually low resolution so a condensed image would have served 
>>>>equally well and, in fact, would save space on their server.)  Ethan's 
>>>>suggestion of PNG is a good idea because that inherently brings down the 
>>>>resolution.
>>>>
>>>>Dan
>>>>   
>>>>
>>>>        
>>>>
>>
>>
>>-------------------------------------------------------
>>SF.Net is sponsored by: Speed Start Your Linux Apps Now.
>>Build and deploy apps & Web services for Linux with
>>a free DVD software kit from IBM. Click Now!
>>http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
>>_______________________________________________
>>gnuplot-beta mailing list
>>gnu...@li...
>>https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>>    
>>
>
>
>
>  
>

-- 

Dan Sebald
email: daniel DOT sebald AT ieee DOT org
URL: http://acer-access DOT com/~dsebald AT acer-access DOT com/

default tic label format

[Pieter-Tjerk de B. <ptd...@cs...>]

Hello,

While trying the 3.8k.1 version of gnuplot, I noticed that the default
format for the labels on the axis is "% g".
This has two (minor) disadvantages, compared to simple "%g" as it was
in version 3.7:
- labels on horizontal axes are not centered nicely (the invisible
  leading space is included in the centering calculation).
- labels on the y2 axis (if used) start farther away from the axis
  than the y1 axis labels do.

Actually, I remember already briefly discussing this with Lars a few
years ago; he then wrote:

>  I remember that change, and that there was a reason for it ...
> 
> Yep, it was suggested to me by Lucas Hart:
> 
> |  BTW, I find DEF_FORMAT "% g" more legible for tables than "%g"; any 
> |  effect on labels that would preclude a change?  Also, since you are 
> 
>  I suppose your observation qualifies for Lucas' condition ...

Obviously, I would suggest changing to "%g" before the release of 4.0.

Best regards,
  Pieter-Tjerk

Re: Postscript and PM3D file size.

[Daniel J S. <dan...@ie...>]

Daniel J Sebald wrote:

>   My experience with angiography images and the sort is that maybe 50% 
> lossless compression is the best on average.

(That's not RLE encoding, but something else.)

Dan

Re: Postscript and PM3D file size.

[Daniel J S. <dan...@ie...>]

Justace Clutter wrote:

>I like the idea of this pixels based image.  The fact that the image
>could be 10 megs does not bother me.
>

Well, in my case my web space is only 10 M.

>  It is the idea that an image could
>be 120 megs bothers me.  So I got the patch and applyed.  It did not go
>clean as one hunk did not make it.  But I fixed it by hand.  I was
>patching against 3.8k.1.  So, I guess that I do not understand the
>syntax.  Here is the datafile that I have, or really a sample:
>
>0.999849 0.652204 0.678995 0.658985 0.585981 0.566624 0.547193 0.615902
>0.539023 0.556559 0.547578 0.53746 0.522146 0.535582 0.530735 0.531924
>0.554801 0.518362 0.479106 0.527362 0.534836 0.546124 0.522834 0.508155
>0.54706 0.469279 0.581292 0.468283 0.535739 0.557979 0.490128 0.536971
>0.530735 0.506848 0.523189 0.531732 0.550917 0.484002 0.57789 0.460027
>0.556111 0.535427 0.508878 0.558746 0.459494 0.578337 0.45805 0.595234 
>
>So I have a grid here, 8x6, that I would like displayed.  Each of these
>floats would need to be mapped onto a color.  PM3D did this great.  So
>when I try to use the matrix modifier with just plot it is not happy,
>then I tried to use splot, still not happy.  Here is what I was trying..
>
>gnuplot> plot 'tmp.data' matrix with image
>
>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>
>Warning: empty x range [0.999849:0.999849], adjusting to
>[0.989851:1.00985]
>
>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>
>gnuplot> splot 'tmp.data' matrix with image
>
>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>
>gnuplot> set view map
>gnuplot> splot 'tmp.data' matrix with image
>
>GNUPLOT (plot_image):  Image grid must be at least 2 x 2.
>
>gnuplot>
>
>Can you give me some ideas on how I can use the image style with this
>setup?  I looked at the information on the image but it seems pretty
>biased toward images like the tux and not array'd data.  Thanks in
>advance.
>

Well, here is the issue.  You are putting the data in ASCII format in 
what looks to you like a grid.  However, Gnuplot hasn't been set up to 
read ASCII data in such a way.  It is set up to read ordered tuples of 
data, such as

x1  y1  z1
x2  y2  z2
...

where the position is part of the data specification.

Telling Gnuplot what the underlying grid is without specifying it is the 
problem.  That is why the "binary" file specifications have extra 
qualifiers to indicate grid dimensions, grid spacing, etc.

Another alternative for binary is "Gnuplot binary" where the first line 
and row of the matrix give the x and y spacing information.  I've 
generally not used that for images, but I think it should work.  (Type 
"help splot binary" at Gnuplot's command line for more info on that format.)

The reason you may want to avoid putting your image in ASCII format is 
that you are playing the numbers game again.  Your example above has 9 
characters for each pixel element.  1800 x 1800 x 9 or roughly 27 M at a 
minimum, and if when you add the (x,y) coordinates it goes to 81 M. 
 That's too big.  If in the program or software you are using to 
generate the data you could multiply by 2^16 and then save the values in 
binary as unsigned 16 bit ints you would be better off.  Can you do that?

Dan

>Justace
>
>On Wed, 2004-03-03 at 01:00, Daniel J Sebald wrote:
>  
>
>>Justace Clutter wrote:
>>
>>    
>>
>>>So,
>>>
>>>	I have this project where I have computed a bunch of channel
>>>correlations.  The entire matrix is like 1800x1800 with one side of the
>>>diaganal set to NaN since it is symetric.  Now to the root of the
>>>problem.  I use the pm3d portion of gnuplot to create a 2D map of the
>>>matrix.  Here is the snipplit of code that I use...
>>>
>>>      
>>>
>><snip>
>>
>>    
>>
>>>I used the first set size command to try to bring the file size down, of
>>>course it did not work.  The resultant file is 120Meg.  This is crazy.
>>>Am I doing somehting wrong?
>>>
>>>      
>>>
>>Justace,
>>
>>Am I understanding correctly that you are essentially attempting to 
>>display an 1800x1800 matrix as an image?  Ethan's comment is correct 
>>that each element, i.e., pixel, will be drawn as a rectangle, i.e., 
>>polygon.  There is a lot of ASCII text associated with each rectangle. 
>> You might try the image patch on SourceForge (something originally 
>>titled "with pixels").  That will put the image into a PostScript file 
>>in a compact form.
>>
>>Realize though that 1800x1800 is a lot of data.  Saved as bytes, that 
>>alone would be roughly 3M.  I've tried 1000x1000 images with no 
>>problems, but things really do slow down.  So, unless it is necessary to 
>>keep such high resolution for the image, your first step might be to 
>>down sample the data appropriately.  Keep in mind what your eventual 
>>display will be.  (For example, people often put high res photos on 
>>their web pages which take close to a minute to download.  However, the 
>>display is usually low resolution so a condensed image would have served 
>>equally well and, in fact, would save space on their server.)  Ethan's 
>>suggestion of PNG is a good idea because that inherently brings down the 
>>resolution.
>>
>>Dan
>>    
>>

Re: Postscript and PM3D file size.

[Daniel J S. <dan...@ie...>]

Arnd Baecker wrote:

>On Wed, 3 Mar 2004, Petr Mikulik wrote:
>
>[...]
>
>  
>
>>2. Use "with pixels style" patch by Daniel at the "Patches" section on
>>sourceforge.
>>    
>>
>
>And in that case I think using RLE instead of the present scheme
>should reduce the file size by another relevant factor.
>(However, though this seems pretty straight forward,
>someone has to code this and at the moment I am completely
>busy with other stuff ;-)
>

Well, run length encoding is great for applications like fax data and 
the sort, i.e., a single bit image depth.  That's because there are 
typically long stretches of white space where the run length really 
achieves compression.  However, I'm not sure what RLE will get you in 
color or gray scale images.  My experience with angiography images and 
the sort is that maybe 50% lossless compression is the best on average. 
 There are forms of lossy compression, but this takes consideration.  If 
someone on the list has experience with compact representation of 
various image types, that would be helpful...

I feel that coding is half the problem.  The other half is simply making 
a decision what to do, i.e., agreeing on what the format or syntax 
should be.  It can in fact be a fun little project to implement some of 
these things.  I just went to the library and looked at the Adobe 
PostScript Standard book and picked the most flexible beginning format, 
which to me seemed to be ASCII85.  The reason is that it would be 
lossless compression (keeping everyone fairly happy from the start), 
various PostScript viewers would likely be able to read it, and there 
would be no question of whether we should use, say, and MPEG compression 
library routine.  That is, I didn't see any sense in going full tilt 
into writing, say, an MPEG routine if it wasn't certain it would get 
used or if the developers may decided using some existing library would 
be the smartest maintenance alternative.

Dan

Re: Postscript and PM3D file size.

[Arnd B. <arn...@we...>]

On Wed, 3 Mar 2004, Petr Mikulik wrote:

[...]

> 2. Use "with pixels style" patch by Daniel at the "Patches" section on
> sourceforge.

And in that case I think using RLE instead of the present scheme
should reduce the file size by another relevant factor.
(However, though this seems pretty straight forward,
someone has to code this and at the moment I am completely
busy with other stuff ;-)

Best,

Arnd

Re: Postscript and PM3D file size.

[Petr M. <mi...@ph...>]

> 	I have this project where I have computed a bunch of channel
> correlations.  The entire matrix is like 1800x1800 with one side of the
> diaganal set to NaN since it is symetric.  Now to the root of the
> problem.  I use the pm3d portion of gnuplot to create a 2D map of the
> matrix.  Here is the snipplit of code that I use...
>
> I used the first set size command to try to bring the file size down, of
> course it did not work.  The resultant file is 120Meg.  This is crazy.

Ethan has explained why the ps file does not depend on "set size".

Decrease size of pm3d map in ps file can be done by:

1. Postprocessing your ps file -- see scripts pm3dCompress.awk and
pm3dConvertToImage.awk in gnuplot sources or in
http://gnuplot.sourceforge.net/scripts/

2. Use "with pixels style" patch by Daniel at the "Patches" section on
sourceforge.

---
Petr Mikulik

Re: Postscript and PM3D file size.

[Ethan A M. <merritt@u.washington.edu>]

On Tuesday 02 March 2004 08:20 pm, Justace Clutter wrote:
>
> 	I have this project where I have computed a bunch of channel
> correlations.  The entire matrix is like 1800x1800 with one side of the
> diaganal set to NaN since it is symetric.  Now to the root of the
> problem.  I use the pm3d portion of gnuplot to create a 2D map of the
> matrix.  Here is the snipplit of code that I use...
>       set terminal postscript eps color palfuncparam 500

This is a case where PostScript is probably the wrong format to choose
for output.   The PostScript output file will contain a separate command
for every point (or box or line or...) plotted, even if they all fall on top
of each other.  For very large datasets, where there are more data
points than the resolution of your output device, you are better off
using a pixel-based output format like PNG.   That way the output
size is fixed in advance.  No matter how many data points map to
a single pixel, it's still only one pixel.


-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Postscript and PM3D file size.

[Justace C. <pro...@co...>]

So,

	I have this project where I have computed a bunch of channel
correlations.  The entire matrix is like 1800x1800 with one side of the
diaganal set to NaN since it is symetric.  Now to the root of the
problem.  I use the pm3d portion of gnuplot to create a 2D map of the
matrix.  Here is the snipplit of code that I use...

-----------------------SNIP
  if($save) {

    print GNUPLOT "

      set terminal postscript eps color palfuncparam 500
      set output \"${saveFilename}\"

      set size 3/10,3/7

    ";
  } else {

    GNUPLOT->autoflush(1);

  }

  print GNUPLOT "

    set xrange [1:1792]
    set yrange [1:1792]
    set cbrange [-1:1]

#    unset colorbox

    unset ytics

    set xtics (\"129\" 129, \"257\" 257, \"385\" 385, \"513\" 513,
\"641\" 641, \"0\" 769, \"129\" 897, \"257\" 1025, \"385\" 1153, \"513\"
1281, \"641\" 1409, \"769\" 1537, \"897\" 1665)
    set y2tics (\"129\" 129, \"257\" 257, \"385\" 385, \"513\" 513,
\"641\" 641, \"0\" 769, \"129\" 897, \"257\" 1025, \"385\" 1153, \"513\"
1281, \"641\" 1409, \"769\" 1537, \"897\" 1665)

    set format y \"%4.0f\"
    set format x \"%4.0f\"

    set size ratio 1

    set title \"${titleClean}\"

    set y2label \"Channel\"
    set xlabel \"Channel\"

    set pm3d map

    set palette defined (0 \"white\", 1 \"black\", 2 \"blue\")
#    set palette defined (-1 \"red\", -0.5 \"black\", 0.5 \"black\", 1
\"blue\")

    splot '${tmpFilename}' matrix title \"\"

  ";

  if(! $save) {

    print GNUPLOT "

      pause -1;

    ";

    getc();

  }

  close(GNUPLOT);
--------------------------END SNIP

I used the first set size command to try to bring the file size down, of
course it did not work.  The resultant file is 120Meg.  This is crazy.
Am I doing somehting wrong?

Justace

Supporting 'set style fill' when plotting filledcurves

[Ethan M. <merritt@u.washington.edu>]

There have been a couple of requests that 'plot with filledcurves'
should honor the setting of 'set style fill'.  I have placed a patch on
SourceForge that adds a style parameter to the term_api definition
of term->filled_polygon for all terminal types that support it.
I went ahead and implemented it for x11 and gd, but not for any
other terminal types as yet.  This is *not* intended for inclusion
in the upcoming version 4.0 release (at least not unless the release
gets substantially delayed for other reasons).

Remarks:

Implementing this for the postscript-like drivers
could be ugly. It would largely duplicate code and
macros already in place for post_boxfill(). I am
inclined to think that the code should be revised to
support only post_filled_polygon(), and post_boxfill()
re-implemented as a special-case request for a 4-vertex
polygon. Note that this is already done for the
equivalent calls in cgm.trm. The problem here is that
currently boxfill() code is compiled conditional on
USE_ULIG_BOXES, while filled_polygon() code is
conditional on PM3D. We would have to disentangle these
two different configuration options, at least with
regard to their overlapping requirement for
term->filled_polygon() support.

Request:

Please feel free to contribute support for other
terminal drivers. A clean PostScript implementation and
windows code are probably both needed before this could
go into cvs.



-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center    (206)543-1421
Mailstop 357742
University of Washington, Seattle, WA 98195

Re: PostScript prologue

[Hans-Bernhard B. <br...@ph...>]

On Mon, 1 Mar 2004, Dave Denholm wrote:

> Aren't string literals merely a convenience feature. You can implement
> things explicitly as
> 
> static const char label1[] = "hello world\n";

Sure.  The core problem is with large arrays of string literals...
Technically, that makes them 2D arrays, and for those, you can't
have the "inner" dimension (i.e. the lengths of individual strings)
variable.  That would mean we either

	static const char PS_header[][60] = {
	   "line 1\n",
	   "line2\n",
	   /*... */
	};

and waste even more space than we currently do, because of variations in
line lengths, or declare each line as an individual, named string
variable, and construct the array from pointers to those strings:

	static const char line1[]="line1\n";
	static const char line2[]="line2\n";
	/*...*/

	static const char *PS_header[] = {
	   line1,
	   line2,
	   /* ... */
	};

The problem at hand is not with string literals as such, but with large
collections of them, and with keeping the source code at least barely
legible and extensible.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: PostScript prologue [was: 16-bit versions of gnuplot doomed?]

[Petr M. <mi...@ph...>]

Instead of forcing to compile gnuplot 4.0 under 16bit with crippled
functionality, what about to extend gnuplot 3.7.3 to 3.7.4 so that it
supports "unset blabla" and "set style xxx" instead of the previous?


> every output document is stored as a separate file in some appropriate
> directory, and then copied into the output stream when a new

I'm like this way -- people are used to have a single (or two)  executables
and will be happening that under crazy installations, they won't be able to
create postscript files, and thus cry a lot.

> 3) It is possible to have multiple, printer- or language- specific, prologues.

There could be a new option for the postscript driver to load such personal
prologues. But that's a new feature never needed until now.

> separate prologue file might get lost if the installation process is botched.
> But this shouldn't be any worse than having to correctly find the outboard
> gnuplot_x11 driver, or the help file.

Unixish people do mostly "make install", but all others (using "*.exe"
binaries) will definitely loose this file frequently.

---
Petr Mikulik

Re: PostScript prologue

[Dave D. <dde...@es...>]

Hans-Bernhard Broeker <br...@ph...> writes:

> On Fri, 27 Feb 2004, Ethan Merritt wrote:
>
>> It occurs to me that we might use the approach of Adobe products
>> like FrameMaker.  The long PostScript prolog that is constant for
>> every output document is stored as a separate file in some appropriate
>> directory, and then copied into the output stream when a new 
>> document is being generated.
>
> Yeah, that idea had been creeping around at the back of my mind, too.
>
>> 1) It doesn't take up any space in the source code module, which is
>>    what your DOS compiler doesn't handle well
>
> At least part of the blame for that goes to the language itself.  There's
> no syntax in C to influence where string constants are to be stored.  
> If you look at the (slightly modified) definition of PS_header:
>
> 	static const char GPFAR * const GPFAR PS_header[] = {
> 	   "line of text\n",
> 	   "line of text\n",
> 	   /*...*/
> 	  "last line\n"
> 	};
>
> you'll find that there are two GPFAR markers for the array of pointers,
> i.e. it's a "far array" of "far pointers".  But I can't tell the compiler
> to put the *strings* themselves into a "far data" or "far code" segment.
>



Aren't string literals merely a convenience feature. You can implement
things explicitly as

static const char label1[] = "hello world\n";

main()
{
  printf(label1);
}

which probably generates identical code (apart from pollution of name space)


So, while tedious, it should be possible to force the string data
to go anywhere you want ..?


dd
-- 
Dave Denholm              <dde...@es...>       http://www.esmertec.com

Re: PostScript prologue [was: 16-bit versions of gnuplot doomed?]

[Ethan A M. <merritt@u.washington.edu>]

On Saturday 28 February 2004 08:12 am, Hans-Bernhard Broeker wrote:
> On Fri, 27 Feb 2004, Ethan Merritt wrote:
> > 2) It allows local customization of the prologue without having to
> >    recompile the main program.
>
> That would complicate bug report handling, though --- it's one more thing
> exposed to be broken by users who don't quite know what they're doing.

People can always break things if they try hard enough.

The normal unix way would be to install the default prologue file read-only
in some system directory, but have the program check first for a 
possibly modified copy in ~/.gnuplot/prologue.ps  or some such.
The first line of defence in the FAQ and in screening bug reports then
becomes:  "If you have a locally customized version of prologue.ps
in your home directory, please remove it and see if the problem persists."

-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: PostScript prologue [was: 16-bit versions of gnuplot doomed?]

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>On Fri, 27 Feb 2004, Ethan Merritt wrote:
>
>  
>
>>It occurs to me that we might use the approach of Adobe products
>>like FrameMaker.  The long PostScript prolog that is constant for
>>every output document is stored as a separate file in some appropriate
>>directory, and then copied into the output stream when a new 
>>document is being generated.
>>    
>>
>
>Yeah, that idea had been creeping around at the back of my mind, too.
>
>  
>
>>1) It doesn't take up any space in the source code module, which is
>>   what your DOS compiler doesn't handle well
>>    
>>
>
>At least part of the blame for that goes to the language itself.  There's
>no syntax in C to influence where string constants are to be stored.  
>If you look at the (slightly modified) definition of PS_header:
>
>	static const char GPFAR * const GPFAR PS_header[] = {
>	   "line of text\n",
>	   "line of text\n",
>	   /*...*/
>	  "last line\n"
>	};
>
>you'll find that there are two GPFAR markers for the array of pointers,
>i.e. it's a "far array" of "far pointers".  But I can't tell the compiler
>to put the *strings* themselves into a "far data" or "far code" segment.
>

Yes, that seems like an assembler controlled directive.

>I'm not quite through trying all possible tricks to get this through
>the compiler's limitations, though.
>  
>

Can you issue an assembler directive using the "asm()" command depending 
upon if it is DOS?  How many different assemblers are there for DOS? 
 (Borland, Microsoft)   And how different are there assembler commands 
for control memory location?  But that might still not do it, because 
controlling the string placement is the issue, right?

This is odd, I just grabbed an old Microsoft C book by Jamsa.  He has an 
example:

main()
{
   char far *str = "Message to display";
   str_display(str);
}

where the subroutine has the command

   printf("%c %p\n", str[i], &str[i]);

which ends up printing the memory values as

6CB2:0282 M
6CB2:0283 e
...

and so on.  So that seems to suggest to me that assemblers should be 
programmed smart enough to recognize that the strings declared with 
"far" pointers can be placed in far memory blocks.

Perhaps the C syntax above is conflicting.  Maybe by also declaring the 
pointer as "const", the assembler is forced to place the strings in the 
segment where constant memory is located; and that may be in the code 
segment.  As a wild guess, you might try removing the "static" and/or 
"const" qualifiers.

Dan

Re: PostScript prologue [was: 16-bit versions of gnuplot doomed?]

[Hans-Bernhard B. <br...@ph...>]

On Fri, 27 Feb 2004, Ethan Merritt wrote:

> It occurs to me that we might use the approach of Adobe products
> like FrameMaker.  The long PostScript prolog that is constant for
> every output document is stored as a separate file in some appropriate
> directory, and then copied into the output stream when a new 
> document is being generated.

Yeah, that idea had been creeping around at the back of my mind, too.

> 1) It doesn't take up any space in the source code module, which is
>    what your DOS compiler doesn't handle well

At least part of the blame for that goes to the language itself.  There's
no syntax in C to influence where string constants are to be stored.  
If you look at the (slightly modified) definition of PS_header:

	static const char GPFAR * const GPFAR PS_header[] = {
	   "line of text\n",
	   "line of text\n",
	   /*...*/
	  "last line\n"
	};

you'll find that there are two GPFAR markers for the array of pointers,
i.e. it's a "far array" of "far pointers".  But I can't tell the compiler
to put the *strings* themselves into a "far data" or "far code" segment.

I'm not quite through trying all possible tricks to get this through
the compiler's limitations, though.

> 2) It allows local customization of the prologue without having to
>    recompile the main program. 

That would complicate bug report handling, though --- it's one more thing
exposed to be broken by users who don't quite know what they're doing.

> 3) It is possible to have multiple, printer- or language- specific,
> prologues.

I don't really see a need for that...  The prologue we have seems to work
reasonably well for everyone.

> The only possible down-side I can think of at the moment is that this
> separate prologue file might get lost if the installation process is botched.

That's indeed a potentially serious problem, mainly because there's no
"standardized" directory layout for DOS and Windows programs.  Win16
doesn't even recognize the "c:\program files\package" scheme.

Even now, it may take an environment variable just to allow DOS and 
Windows gnuplot to reliably find their help files.

I'll include this option in my further considerations ...

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: cgm.trm [FILLEDBOXES && !PM3D] breaks

[Ethan M. <merritt@u.washington.edu>]

On Thursday 26 February 2004 09:22 am, Ethan Merritt wrote:
> On Thursday 26 February 2004 04:39 am, Hans-Bernhard Broeker wrote:
> > cgm.trm can't be built if FILLEDBOXES was turned on, but PM3D wasn't,
> > because CGM_fillbox uses the PM3D type gpiPoint internally, and calls
> > CGM_filled_polygon() to do the actual job.
>
> If necessary I can probably hack at it to make it compile properly given
> the various conditional compilation options, but I would need to have
> someone else to confirm the validity of the output in each case.

Simple patch attached.  It just dummies up one typedef so that
cgm.trm can use the same code it would have used if PM3D were present.
Someone please confirm that this works on a system with "real" cgm.


-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center    (206)543-1421
Mailstop 357742
University of Washington, Seattle, WA 98195

PostScript prologue [was: 16-bit versions of gnuplot doomed?]

[Ethan M. <merritt@u.washington.edu>]

On Wednesday 25 February 2004 01:03 am, Hans-Bernhard Broeker wrote:
>
> DOS-16bit works, sort of.  But that's after I disabled most of the bit
> optional features (filled boxes, PM3D) *and* removed pretty much all
> terminal drivers besides dospc (the interactive graphics driver), table
> and cgm.  Yes, that does mean I had to throw out post.trm ;-( The reason
> post.trm had to go is the PS_header[] string array.  This is simply too
> large, ...

It occurs to me that we might use the approach of Adobe products
like FrameMaker.  The long PostScript prolog that is constant for
every output document is stored as a separate file in some appropriate
directory, and then copied into the output stream when a new 
document is being generated.

This has a number of advantages:

1) It doesn't take up any space in the source code module, which is
   what your DOS compiler doesn't handle well

2) It allows local customization of the prologue without having to
   recompile the main program. 
   This would allow customization of A4/letter page size, selection of colors,
   dash-dot styles, and alternate character encodings.

3) It is possible to have multiple, printer- or language- specific, prologues.

The only possible down-side I can think of at the moment is that this
separate prologue file might get lost if the installation process is botched.
But this shouldn't be any worse than having to correctly find the outboard
gnuplot_x11 driver, or the help file.





-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center    (206)543-1421
Mailstop 357742
University of Washington, Seattle, WA 98195

Re: Going over bugs before release...

[Daniel J S. <dan...@ie...>]

Ethan Merritt wrote:

>On Thursday 26 February 2004 09:57 am, Daniel J Sebald wrote:
>  
>
>>Oh yeah, what about this issue with the arrowhead tips extending past
>>the point specified in the data?  I guess I'm in the camp that says that
>>should be fixed so that the arrowhead tip ends right at the data point
>>specified.  It would be nice to not let people get into the habit of
>>compensating for that themselves and then to later have it changes.
>>    
>>
>
>I am opposed to touching this before a version 4 release.
>It falls into the area of aesthetics and philosophy rather than
>bug-fixing.
>

It is a big change.  Not a show-stopper.

Dan

Re: Going over bugs before release...

[Ethan M. <merritt@u.washington.edu>]

On Thursday 26 February 2004 09:57 am, Daniel J Sebald wrote:
> Oh yeah, what about this issue with the arrowhead tips extending past
> the point specified in the data?  I guess I'm in the camp that says that
> should be fixed so that the arrowhead tip ends right at the data point
> specified.  It would be nice to not let people get into the habit of
> compensating for that themselves and then to later have it changes.

I am opposed to touching this before a version 4 release.
It falls into the area of aesthetics and philosophy rather than
bug-fixing.

I don't want to rehash the whole discussion now, but I suggest
that long-term the best fix is to have an arrow drawing mode
that works by building the arrow out of filled rectangles and filled
triangles.  That way you could specify thin lines but a solid fill
and I think everyone would be happy with the resulting plot.

-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center    (206)543-1421
Mailstop 357742
University of Washington, Seattle, WA 98195

Re: Going over bugs before release...

[Daniel J S. <dan...@ie...>]

Oh yeah, what about this issue with the arrowhead tips extending past 
the point specified in the data?  I guess I'm in the camp that says that 
should be fixed so that the arrowhead tip ends right at the data point 
specified.  It would be nice to not let people get into the habit of 
compensating for that themselves and then to later have it changes.  I 
wrote out the formula and was going to write a patch, but unfortunately 
there was nowhere that I could get the line thickness from.

Could it be kludged in some way?  That is, could we add a terminal 
routine (dread!) that returns the width of the line and then in the 
terminal function just do some trial and error guesstimating about the 
line thickness?  That is, the formula for the vector to compensate is

v = - SE * w / (2 |SE| sin(alpha))

where SE is the vector in terms of start and end position, alpha is the 
angle of the arrow command, and w is the width of the current line. 
 Since line widths are a unitless number, I assume that each terminal 
needs a different routine, e.g.,

w = term->linewidth( unitless_width );

Where w is the width in appropriate graph units.  Or maybe the linewidth 
could be an array entry in other terminal characteristics.  For now, 
maybe the values could be fudged by trial and error.  E.g.,

float PS_linewidth( unitless_width ) {
  /* Do a crude linear approximation for now. */
  return unitless_width * TRIAL_AND_ERROR_PARAMETER;
}

Any thoughts on what to do here?

Dan