gnuplot-beta

Simplified processing of image and rgbimage plots

[Ethan A M. <merritt@u.washington.edu>]

Yesterday I committed to CVS a patch that Dan Sebald and I worked out.
It removes many restrictions on the use of RGBIMAGE in 3D plots, and the 
use of RGB color in polygons derived from image data. Now all terminal 
drivers that can handle RGB colors and filled polygons can automatically 
also handle image and rgbimage plot styles.

This extends the capabilities of svg, xfig, and maybe other terminals.

The specialized code for image handling could be removed from gd.trm, for 
instance. Also x11. Is this worth doing?

The specialized image handling code in post.trm is still useful because it 
drastically reduces the output file size, however.

An interesting note:  
The patch removed as much code as it added. 
Net change: +2 lines of code.
And this is aside from the possibility of now removing the special-case
term->image code from gd.trm, x11, and maybe other terminal drivers.



-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

Re: Simplified processing of image and rgbimage plots

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:
> Yesterday I committed to CVS a patch that Dan Sebald and I worked out.
> It removes many restrictions on the use of RGBIMAGE in 3D plots, and the 
> use of RGB color in polygons derived from image data. Now all terminal 
> drivers that can handle RGB colors and filled polygons can automatically 
> also handle image and rgbimage plot styles.
> 
> This extends the capabilities of svg, xfig, and maybe other terminals.
> 
> The specialized code for image handling could be removed from gd.trm, for 
> instance. Also x11. Is this worth doing?
> 
> The specialized image handling code in post.trm is still useful because it 
> drastically reduces the output file size, however.

I like this improvement, but I also suggest leaving the the broader image handling in place with the fall back to the polygon-based approach.  For x11, the image code is preferred from the standpoint of memory size and refresh speed.  For an image there is M x N pixel values and the location of its points and borders.  The same image as polygons is M x N pixel values and 4 x M x N pixel corner values.  More memory, more processing for the polygons (even though the code for polygon processing may be more straightforward).  The other thing that having all the image data in the form of one hunk is that if ever someone wants to improve anti-aliasing with the image, it would be possible.  With individual rectangular elements, signal processing on the image would be cumbersome.

For PostScript, there is the capability to place images at viewing angles.  (However, that terminal function doesn't have variables for that right now...  Hindsight is 20/20 I guess.)

For the bitmapped outputs like PNG and so on, it wouldn't matter if both the image routines and polygon methods produced the same output.  We could test that with a simple file diff, I think.

Dan

Re: Simplified processing of image and rgbimage plots

[Ethan A M. <merritt@u.washington.edu>]

On Thursday 16 November 2006 07:05 pm, Daniel J Sebald wrote:
> Ethan A Merritt wrote:
> > The specialized code for image handling could be removed from gd.trm,
> > for instance. Also x11. Is this worth doing?
>
> For the bitmapped outputs like PNG and so on, it wouldn't matter if 
> both the image routines and polygon methods produced the same output.

There seems to be a very slight difference in the scaling of the whole
plot.  I guess there must be some subtle difference in the coordinates
fed to the auto-scaling routine?  But they both look equally good to me.

I don't notice any difference at all between the two modes when
output to x11.

-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

Re: Simplified processing of image and rgbimage plots

[Petr M. <mi...@ph...>]

>> This extends the capabilities of svg, xfig, and maybe other terminals.
>>
>> The specialized code for image handling could be removed from gd.trm, for
>> instance. Also x11. Is this worth doing?

Do you really mean to remove the term->image() routine??? Definitely no, the 
processing is several orders of magnitude slower ... try to draw a 1024x1024 
image with pm3d and with image. Terminal-specific term->image() is the 
proper way.

However, your generic fall-back is OK.

---
PM

Re: Simplified processing of image and rgbimage plots

[Ethan A M. <merritt@u.washington.edu>]

On Sunday 19 November 2006 05:47 am, Petr Mikulik wrote:
> 
> However, your generic fall-back is OK.

I want to clarify something.  This is not just a fall-back.
It is currently the only way to place rgb data in 3D.  

So not only does it allow additional drivers (svg, fig, emf) to 
handle rgbimage in 2D, it also extends the capability of ALL drivers
to work in 3D.

However, as you noted it is rather expensive to describe an image
this way, which causes slow data transfer in x11 and large file
size in PostScript and PDF.

We could do better.

PostScript supports the specification of a general transformation
matrix.  Rather than using individual filled_polygons
to draw a rgbimage in 3D, we could simply include a projection of
the 3x3 rotation matrix.  This would drastically reduce the 
PostScript file size for such plots.   We already calculate this
same matrix in the core code in order to place the polygons; in
the case of PostScript it would be sufficient to provide the matrix
to the driver along with the original 2D array of rgb pixels.

Anyone want to have a go a coding this?

I believe that PDF and SVG also benefit from this approach, but
I have not investigated fully.


Hand-worke example:

Here is the start of the PostScript image code produced by the
first plot in "image.dem".  I have added one line that inserts
a transformation matrix.  Try it yourself.

 %%%%BeginImage
InterpretLevel1 { ...
} {
  gsave
  1149 3738 translate
  5490 -3219 scale
  [1 -.05 0.1 0.7 0.05 -0.15] concat %%% general transformation matrix %%%
  128 128 8
  [ 128 0 0 128 0 0 ]
  currentfile /ASCII85Decode filter
  false 3
  colorimage
} ifelse





-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

Re: Simplified processing of image and rgbimage plots

[<tim...@en...>]

> On Sunday 19 November 2006 05:47 am, Petr Mikulik wrote:
>>
>> However, your generic fall-back is OK.
>
> I want to clarify something.  This is not just a fall-back.
> It is currently the only way to place rgb data in 3D.
>
> So not only does it allow additional drivers (svg, fig, emf) to
> handle rgbimage in 2D, it also extends the capability of ALL drivers
> to work in 3D.
>
> However, as you noted it is rather expensive to describe an image
> this way, which causes slow data transfer in x11 and large file
> size in PostScript and PDF.
>
> We could do better.
>
> PostScript supports the specification of a general transformation
> matrix.  Rather than using individual filled_polygons
> to draw a rgbimage in 3D, we could simply include a projection of
> the 3x3 rotation matrix.  This would drastically reduce the
> PostScript file size for such plots.   We already calculate this
> same matrix in the core code in order to place the polygons; in
> the case of PostScript it would be sufficient to provide the matrix
> to the driver along with the original 2D array of rgb pixels.
>
> Anyone want to have a go a coding this?
>
> I believe that PDF and SVG also benefit from this approach, but
> I have not investigated fully.

For your information, cairo supports this, so the wxt terminal will profi=
t
from it too.

Best regards,

Timoth=E9e

Re: Simplified processing of image and rgbimage plots

[Ethan A M. <merritt@u.washington.edu>]

On Sunday 19 November 2006 05:47 am, Petr Mikulik wrote:
> >> This extends the capabilities of svg, xfig, and maybe other terminals.
> >>
> >> The specialized code for image handling could be removed from gd.trm, for
> >> instance. Also x11. Is this worth doing?
> 
> Do you really mean to remove the term->image() routine??? Definitely no, the 
> processing is several orders of magnitude slower ... try to draw a 1024x1024 
> image with pm3d and with image. Terminal-specific term->image() is the 
> proper way.

It may seem counter-intuitive, but in fact the RGB polygon processing is
just as fast as the term->image() processing, at least for gd.trm.
Here is a timed benchmark run using a 800x600 image for the test script

  set term png truecolor size 1024,768
  set output 'junk.png'
  plot 'bench.avs' binary filetype=avs with rgbimage
  replot
  [... 10 plots total]
  replot

time ./gnuplot bench.gnu  (average of 5 runs)
  16.785u 0.509s 0:17.31 99.9%    0+0k 0+0io 0pf+0w

same thing except "with rgbimage generic"
  16.672u 0.572s 0:17.18 99.4%    0+0k 0+0io 0pf+0w



The generic code is significantly slower (4-5X) for x11, but I think that
is a matter of the transfer time through the pipe for a description that is
less compact (30MB vs 2.9 MB).  This difference could be reduced by switching
to a binary protocol for transmitting RGB color + polygon down the pipe.




x11 with rgbimage
  6.331u 0.675s 0:07.05 99.2%     0+0k 0+0io 0pf+0w
x11 with rgbimage generic
  28.418u 17.907s 6:37.59 11.6%   0+0k 0+0io 0pf+0w


-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

Re: Simplified processing of image and rgbimage plots

[Petr M. <mi...@ph...>]

>> Do you really mean to remove the term->image() routine??? Definitely no, the
>> processing is several orders of magnitude slower ... try to draw a 1024x1024
>> image with pm3d and with image. Terminal-specific term->image() is the
>> proper way.
>
> It may seem counter-intuitive, but in fact the RGB polygon processing is
> just as fast as the term->image() processing, at least for gd.trm.

Are not ther any "aliasing" or "Moire" problems?

> And rendering speed.  gv is very slow with the filled polygons.

For performance reasons, I guess term->image() is a must. It was implemented 
for exactly this reason, after all.

BTW, there are some other screen terminals (OS/2, Windows) which may or may 
not be double-buffered, and where many-polygon approach instead of image 
makes them very very slow -- and even for a window redraw by the window 
manager.

---
PM

Re: Simplified processing of image and rgbimage plots

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

> We could do better.
> 
> PostScript supports the specification of a general transformation
> matrix.  Rather than using individual filled_polygons
> to draw a rgbimage in 3D, we could simply include a projection of
> the 3x3 rotation matrix.  This would drastically reduce the 
> PostScript file size for such plots.

And rendering speed.  gv is very slow with the filled polygons.  (The aliasing effect would probably go away too with the proper translation matrix.)

   We already calculate this
> same matrix in the core code in order to place the polygons; in
> the case of PostScript it would be sufficient to provide the matrix
> to the driver along with the original 2D array of rgb pixels.
> 
> Anyone want to have a go a coding this?

Not too difficult.  First step would be to add the necessary translation matrix info to the terminal image routine.

> I believe that PDF and SVG also benefit from this approach, but
> I have not investigated fully.

But does the PDF library support translation matrices?

Dan

Re: Simplified processing of image and rgbimage plots

[Ethan A M. <merritt@u.washington.edu>]

On Sunday 19 November 2006 09:11 pm, Daniel J Sebald wrote:
> Ethan A Merritt wrote:
> 
> > We could do better.
> > 
> > PostScript supports the specification of a general transformation
> > matrix.  Rather than using individual filled_polygons
> > to draw a rgbimage in 3D, we could simply include a projection of
> > the 3x3 rotation matrix.
> 
> > I believe that PDF and SVG also benefit from this approach, but
> > I have not investigated fully.
> 
> But does the PDF library support translation matrices?

Yes. Exactly the same as PostScript:

void PDF_concat(PDF *p, double a, double b, double c, double d, double e, double f)
  Concatenate a matrix to the current transformation matrix.
  a, b, c, d, e, f Elements of a transformation matrix. 
  The six values make up a matrix in the same way as in PostScript and PDF 
  (see references). In order to avoid degenerate transformations, 
  a*d must not be equal to b*c.



-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

Re: Simplified processing of image and rgbimage plots

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

>>But does the PDF library support translation matrices?
> 
> 
> Yes. Exactly the same as PostScript:
> 
> void PDF_concat(PDF *p, double a, double b, double c, double d, double e, double f)
>   Concatenate a matrix to the current transformation matrix.
>   a, b, c, d, e, f Elements of a transformation matrix. 
>   The six values make up a matrix in the same way as in PostScript and PDF 
>   (see references). In order to avoid degenerate transformations, 
>   a*d must not be equal to b*c.

Ah!  I didn't see that:

http://us2.php.net/manual/en/function.pdf-concat.php

I was looking for something that might be part of an image command.

Dan

Re: Simplified processing of image and rgbimage plots

[Daniel J S. <dan...@ie...>]

Timothée Lecomte wrote:

>>I believe that PDF and SVG also benefit from this approach, but
>>I have not investigated fully.
> 
> 
> For your information, cairo supports this, so the wxt terminal will profit
> from it too.

Already seems worth doing then.

Would it mean altering

    void (*image) __PROTO((unsigned, unsigned, coordval *, gpiPoint *, t_imagecolor));

to include a pointer to a set of values representing a translation matrix?

Or would a new terminal driver routine for a translation matrix that applies more generally be in order?

Dan

Re: Simplified processing of image and rgbimage plots

[Ethan A M. <merritt@u.washington.edu>]

On Monday 20 November 2006 10:01 pm, Daniel J Sebald wrote:
> 
> Would it mean altering
> 
>     void (*image) __PROTO((unsigned, unsigned, coordval *, gpiPoint *, t_imagecolor));
> 
> to include a pointer to a set of values representing a translation matrix?
> 
> Or would a new terminal driver routine for a translation matrix 
> that applies more generally be in order? 

I think it wants to be a new terminal entry (the core code needs to be able
to test for it). But in some of the terminals (post, pdf) this will 
essentially replace the current routine, which in turn becomes a one line
wrapper:

PS_image_rotate(M, N, *image, *corner, color_mode, double *matrix) 
{ old code with a new twist }

PS_image(M, N, *image, *corner, color_mode)
{
static double identity[9] = {1,0,0, 0,1,0, 0,0,1};
    PS_image_rotate(M, N, *image, *corner, color_mode, identity);
}

-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742