Re: Excessive memory usage with the pdfcairo terminal?

[Ethan M. <merritt@u.washington.edu>]

On Friday 08 October 2010 02:07:42 am Christoph Bersch wrote:
> Ethan Merritt schrieb:
> > 
> > The long-standing admonition still holds. Vector formats like PostScript,
> > PDF, SVG are bad choices for creating a plot with a million points.
> > A bitmap format like PNG does much better.  "Ah", you say, 
> > "but I asked cairo to produce a PNG file." Well yes, but that's not the way
> > cairo works.  First it creates the requested graphic in a standard internal
> > representation that maintains all the necessary information for vector output.
> > Then when you are ready to dump it to a file, it converts the internal
> > representation to the final output representation.
> > Or at least that's how I understand it.
> > 
> > Time to revisit the idea of a pdflatex terminal?
> 
> How do you think would a pdflatex terminal solve this problem and what 
> configuration do you have in mind for a pdflatex terminal?

I assume that the goal is to end up with a PDF file that contains
a plot constructed from an unreasonably large number of points.

This can be addressed by wrapping a bitmap image in a PDF file so that
the text is handled at full resolution while the bitmap image is handled
at a fixed maximum resolution that caps the total file size.

We already have a model for doing exactly this in gnuplot - the epslatex
terminal.  It splits the task into a text part and a non-text part, letting
latex handle the text and the postscript terminal handle the rest. 
I think it would be relatively easy to modify this so that the non-text
part is handled by the existing png driver and the latex part is tweaked
slightly for use with pdflatex rather than latex.

Obviously it is not strictly necessary to involve latex in this process.
I think the same split-and-recombine approach could be implemented by
importing a bitmap image generated by gd.trm into a complete document
created by pdfcairo.  But this would involve more work, as the code would
need to be written from scratch rather than being trivially modified from
the existing epslatex terminal driver. 

I can see arguments in favor of either path.
The argument in favor of png+pdflatex is that it could be supported
without requiring that gnuplot be built with the cairo libraries, and
the amount of new coding and debugging is expected to be small.
The arguments in favor of png+pdfcairo are that gnuplot itself would
emit the final pdf document rather than requiring a separate run of
pdflatex.

> In my eyes there are many different possibilities to realize a pdflatex 
> terminal, but I don't see any advantages of this compared to the cairo 
> terminals (apart from the mathematical typesetting).

Either way, someone has to code up the new terminal.   My estimate is
that basing it on the existing epslatex terminal would be relatively simple.
But if the hypothetical "someone" prefers to work with cairo and is willing
to take on the more complicated job, that's great also.

Either way there's already a patch on SourceForge that modifies gd.trm
so that it produces only the non-text portion of a plot.

	Ethan