#1820 Object order is lost in epslatex terminal
In Octave, a white background rectangle is placed below the axes, plot contents, legends, etc. The problem arises when creating an epslatex document in which the eps contents appears in a separate file. Order is lost because all EPS graphics elements are lumped into one file.
If one looks at the contents of the LaTeX file they'll see:
\gplbacktext
\put(0,0){\includegraphics{plot-inc}}%
\gplfronttext
The axes annotations all end up in the \gplbacktext definition while the key contents ends up in the \gplfronttext definition. Sincere there is a plot size rectangle in the background that ends up within plot-inc.eps, whatever is within \gplbacktext is no longer visible.
The best way to address this is to lump graphics into an EPS file until the next LaTeX text element comes along. So, for example, in the scenario I described, the contents might be something like
\begin{picture}(8000.00,4800.00)%
\put(0,0){\includegraphics{plot1-inc}}%
\colorrgb{0.00,0.00,0.00}%
\put(660,400){\makebox(0,0)[r]{\strut{}-1}}%
\colorrgb{0.00,0.00,0.00}%
\put(660,1440){\makebox(0,0)[r]{\strut{}-0.5}}%
\colorrgb{0.00,0.00,0.00}%
\put(660,2480){\makebox(0,0)[r]{\strut{}0}}%
\colorrgb{0.00,0.00,0.00}%
\put(660,3519){\makebox(0,0)[r]{\strut{}0.5}}%
\colorrgb{0.00,0.00,0.00}%
\put(660,4559){\makebox(0,0)[r]{\strut{}1}}%
\colorrgb{0.00,0.00,0.00}%
\put(780,200){\makebox(0,0){\strut{}0}}%
\colorrgb{0.00,0.00,0.00}%
\put(1923,200){\makebox(0,0){\strut{}2}}%
\colorrgb{0.00,0.00,0.00}%
\put(3066,200){\makebox(0,0){\strut{}4}}%
\colorrgb{0.00,0.00,0.00}%
\put(4210,200){\makebox(0,0){\strut{}6}}%
\colorrgb{0.00,0.00,0.00}%
\put(5353,200){\makebox(0,0){\strut{}8}}%
\colorrgb{0.00,0.00,0.00}%
\put(6496,200){\makebox(0,0){\strut{}10}}%
\colorrgb{0.00,0.00,0.00}%
\put(7639,200){\makebox(0,0){\strut{}12}}%
\put(0,0){\includegraphics{plot2-inc}}%
\colorrgb{0.00,0.00,0.00}%
\put(7279,4433){\makebox(0,0)[l]{\strut{}$\sin(t)$}}%
\end{picture}%
where plot1-inc.eps is the background and plot2-inc.eps is the graph line/symbols, axes lines, tic markes, etc. There is no use of \gplbacktext and \gplbacktext. So, there could be multiple EPS files.
Does this seem worth trying? It can't be too difficult to program something which ends an EPS file with the occurrence of a new text element and starts a new EPS file with the occurrence of a new graphic element.
I suspect you are doing something wrong in Octave. Here is a hand-constructed plot using the epslatex terminal that shows this is not a problem in general:
Last edit: Ethan Merritt 2016-06-21
sinc(x) is undefined for me. Is that something you defined prior to running the above script? Or is it something having to do with my build?
In any case, when I run your example and change sinc(x) to sin(x) it exhibits the same problem I reported prior. I does not look like your PNG example. How did you get the PNG example? Here's a screen capture of ghostview on the file created from
(called screenshot_current_gnuplot_backing1.png). I then went and manually altered the order of objects in the generated LaTeX file as follows:
\put(0,0){\includegraphics{backing-inc}}% \gplbacktext \gplfronttextwhich produces a result more along the lines of what is wanted, but of course doing what I did would take any foreground graphics objects and move them to the back.
I meant to select the screenshots...
Ah, "dvi". You used xdvi to view the DVI file. That doesn't work for me in either case, but xdvi viewer is known to be non-robust as far as PostScript integration. I can't recall exactly how PostScript is worked in, but essential xdvi leaves some space where the bounding box is and lets (maybe?) some other application draw the PostScript. That is why PostScript elements don't appear when using the zoom window in xdvi.
Actually I was using okular, not xdvi.
I suspect a bug elsewhere in your toolchain.
Here is my session, and my output
himeji [301] gnuplot_5.0.3 G N U P L O T Version 5.0 patchlevel 3 last modified 2016-02-21 Copyright (C) 1986-1993, 1998, 2004, 2007-2016 Thomas Williams, Colin Kelley and many others gnuplot home: http://www.gnuplot.info faq, bugs, etc: type "help FAQ" immediate help: type "help" (plot window: hit 'h') Terminal type set to 'qt' gnuplot> set tics front gnuplot> set obj 99 rect from screen 0,screen 0 to screen 1, screen 1 fs solid fc rgb "#ffffaa" gnuplot> set obj 99 behind gnuplot> set label 99 at 0,0 "This label is in front" front gnuplot> set term epslatex standalone Terminal type set to 'epslatex' Options are ' leveldefault monochrome blacktext \ dashlength 1.0 linewidth 1.0 butt noclip \ nobackground \ palfuncparam 2000,0.003 \ standalone "" 11 fontscale 1.0 ' gnuplot> set output 'backing.tex' gnuplot> plot sinc(x) gnuplot> set out gnuplot> !pslatex backing gnuplot> !dvips -o backing.ps backing.dvi gnuplot> qLast edit: Ethan Merritt 2016-06-21
By the way, another mechanism that bypasses this question of layers is to use
"set term epslatex background rgb '#aaffff'"
This places the background color in the LaTeX file rather than in the included eps file.
That may work, but it isn't generic. Octave can select all gnuplot output terminals so it is using a rectangle for backing in all cases. So, at a fairly high level I'd have to check the terminal type and then do or don't draw the rectangle.
Are there any relevant gnuplot terminals that don't already support "set term foo background xxx"? I'm pretty sure it works for all the cairo, postscript and libgd terminals, as well as svg and emf.
After experimenting a bit, it seems that it is the "set tics front" command that makes this work. Not the "set obj # behind". What happens is that the tick text appears in both gplbacktext and gplfronttext. (Should it be doing that? Wouldn't the text in the gplfronttext be sufficient?)
I still wonder about this single EPS file though. Say someone wants a rectangle in back, the axis decorations including text, and then symbols on the top layer so the data points are not obscured by decorations?
The program itself tracks multiple layers
Currently only the BACK/FRONT boundary is passed down to the terminals via term->layer() but if we need to this could be made more fine-grained.
At some point I think you decide that the epslatex approach is too much of a hack and switch to using tikz or conTeXt or some other more modern pathway to generating latex documents.
Here's a proof-of-concept patch that creates multiple EPS files rather than the text-layer elements. Experiment with it a bit; I think it has potential. Some comments:
1) This approach will maintain the natural order in which objects are layed out in the core. So maybe we can get rid of the TERM_LAYER_BACKTEXT, TERM_LAYER_FRONTTEXT and TERM_LAYER_END_TEXT enumerations. Other than EPSLATEX, it is only LUA terminal that uses those. It seems to me that the core is the natural place to handle this. Text layers within the drivers is cumbersome and probably only makes things worse from the core's perspective. Just always plotting things in order seems most simple.
2) The example I used to test this resulted in 14 EPS files. That still works of course, but it does make for some file bulkiness and slow rendering in GhostView. The reason is that the axis tics routine plots a couple tics, then a label, couple tics, label, etc. We should be able to make the core routine a little more friendly in this sense by laying out all the tics first, then all the labels. Three or four EPS files is better than 14. As an example, in graphics.c instead of
/* label first y axis tics */ axis_output_tics(FIRST_Y_AXIS, &ytic_x, FIRST_X_AXIS, ytick2d_callback); /* label first x axis tics */ axis_output_tics(FIRST_X_AXIS, &xtic_y, FIRST_Y_AXIS, xtick2d_callback);we could do
/* label first x and y axis tics */ axis_output_tics(FIRST_Y_AXIS, &ytic_x, FIRST_X_AXIS, ytick2d_justtic_callback); axis_output_tics(FIRST_X_AXIS, &xtic_y, FIRST_Y_AXIS, xtick2d_justtic_callback); axis_output_tics(FIRST_Y_AXIS, &ytic_x, FIRST_X_AXIS, ytick2d_justlabel_callback); axis_output_tics(FIRST_X_AXIS, &xtic_y, FIRST_Y_AXIS, xtick2d_justlabel_callback);without too much work.
3) The "set tics front" front puts the tics and labels in the file twice. So, with that option I'd get 26 EPS files. I investigated a bit and see why. There is a feature to draw the grid and tic/labels separately. However, while it is the case that the tic/labels can be drawn without the grid, the inverse isn't true. It would be a lot of work to draw just the grid and not the tic/labels give the intertwined code. So, someone just felt it easy enough to let the tics/labels be drawn twice. Of course, that creates duplication in terminals which save to a file. (For screen terminals it is simply a case of overwriting video memory.) That is more an issue of inflexibility on part of the core that should eventually be corrected.
I really do not like the sound of this idea. 26 included files?
What problem exactly is this solving?
I'm always happier to start with demonstration of a problem, so that there is a benchmark against which to evaluate proposed solutions. Can you provide a script that produces an acceptable plot in, say, the pdf (cairo) terminal but produces an unacceptable plot with epslatex?
The original bug report is here:
https://savannah.gnu.org/bugs/?48200
I'm attaching the gnuplot code that Octave generates in sinewave.gp, but I commented out the terminal/output commands so that the code can be used with other terminals. Attached is also the result for a few terminals. All of the examples have the tic labels except epslatex.
The problem that the patch is meant to solve is one of maintaining the order in which objects are displayed or layered on the output device. The inherent issue with the existing epslatex approach is that all graphics objects are flattened into a single layer; no LaTeX text can be layered inbetween. This is slightly different than this particular set of gnuplot code--the existing bug can likely be solved by placing the labels in the right category, either "backtext" or "fronttext". But one could come up with example where text lies between graphics objects without too much effort, say place a text box or key on top of the plot and which partially obscures the axes.
I forgot one, the patched epslatex result... As I pointed out, the problem of having so many files can be solved with a bit of optimization on part of the code: draw the font strings in as long a stretch of entries as possible. That can be done in simple ways with programming techniques (draw all the tics, then draw all the labels), or a very sophisticated hidden drawing routine (which I'm not suggesting) could bunch strings together so long as it doesn't disrupt the way in which plot objects obscure one another.
You recently fixed the SVG terminal so that it was not necessary to generate, uniquely name, and track multiple files PNG files associated with a single SVG figure. Here is seems you would be creating the very same problem, multiple files for a single figure, for the epslatex terminal. It is already non-ideal that one needs to track two files, FOO.tex and FOO-inc.eps. It seems crazy to increase this to a potentially unlimited number of auxiliary files.
If the current epslatex mechanism of backtext/image/fronttext is inadequate for some class of real figures, I think the better answer is to switch to a different LaTeX terminal. This problem doesn't exist for tikz, right?
Good point, but I think the situation is slightly different.
For the SVG case it wasn't an issue of losing the layer order, but more of self-containment of image files (not PostScript files). In some sense, the non-standalone version of the SVG terminal is creating true auxiliary files, and such files are the end product.
But in the case of epslatex, these PostScript files and the latex file itself are more like intermediate files (although we've probably always refered to them as auxiliary). There is further processing to be done on these files, via LaTeX, and after running LaTeX on a standalone version of the plot to create the EPS file (having the embedded LaTeX fonts) one can discard all those intermediate files and just use the end-product EPS file on its own.
Ah, so lua terminal is tikz, and that's the other terminal that has the text layer. Let's see how that one works... I'm having some trouble compiling the output in LaTeX. I've generated the files for the example, then tried compiling. But the "tikz" package could not be found. I went to the Mint package manager and found a qtikz package that required "pgf" package and I think the LaTeX style was in there. So now I can compile, and then run dvips to create the EPS file. The result I get is attached, but it generates all sorts of PostScript stack errors. Similarly, viewing the DVI file with xdvi creates a lot of postscript error messages at the command line while what is displayed is only a small blob of all the text strings plotted one on top of the other. Let me know if you can get Tikz processing to work and, if so, list the commands you used.
Anyway, the more I think about this I'd prefer to maintain layer order all the way through LaTeX processing and get rid of the special text-layer terminal commands in the core code. Note that pstex and pslatex also have a similar issue. Although they don't have associated intermediate PS files, they store up all of the text strings to be layered after a single embedded PostScript file. I don't think it would be too difficult to use the same strategy for those two terminals as what the patch does for epslatex terminal.
I think that is a misunderstanding of why people use the terminal. If the desired end-product is an EPS file, why use a LaTeX terminal? I'm pretty sure that people using epslatex are doing so because the end-product is a latex document that includes this gnuplot-generated figure. To do that you need to keep both the FOO.tex and FOO_inc.eps files together so that including FOO.tex into your final document can in its turn pull in the eps part.
The situation is exactly analogous to generating an svg file for inclusion on a web page.
tikz is intended for use with pdflatex, so no eps or dvi involved.
If you create a plot with "set term tikz standalone; set output 'FOO.tex'" then you can produce a pdf version of the plot using the command "pdflatex FOO". If the idea is to include it in a larger LaTeX document you can either refer to the processed FOO.pdf file or you can omit the standalone flag from the 'set term tikz' command and include the FOO.tex file directly in your larger document without needing to track any auxilliary files.
OK, this works using "pdflatex" as opposed to "latex" to process. (Why shouldn't "latex" work as well?) Looks very nice. It is a PDF file though rather than an EPS file. Perhaps there is now some way of importing a PDF file in LaTeX or other word processors, but that is beside the point.
Note though, that all this layering stuff in the Lua terminal
doesn't seem to have any affect. I look at the generated foo.tex file and the drawing order is just as it is with all the other terminals (except epslatex). Where in all the processing is a "backtext" or "fronttext" pushed string interpretted? I've a feeling that the layer description is just used for classifying and not really for changing the order in which LUA handles the text strings.
So, LUA is fine as to maintaining layer order consistent with almost all terminals, works nicely, and in fact that "fronttext", "backtext", etc. can be tossed, as I see it. That's good.
However, epslatex still has it's uses. The main reason for someone to mix latex and PostScript is to get LaTeX fonts and math formatting in a figure with nice graphics elements. The standard TeX/LaTeX drawing elements (not tikz/pgf) are crude and inflexible. I'm often creating plots that have formatted math formulas, etc. on them somewhere. You've described an alternative to using PS/LaTeX files, as though they are simply input files. Anyone who's worked with a manuscript or lengthy paper knows there can be a lot of various LaTeX files. On the other hand, the "standalone" option of epslatex lets one create a PS/LaTeX EPS file that stands on its own, having the fonts and formatting. The advantage there is that file can then be used in something other than LaTeX (say some researcher wants to create a poster to go along with their paper, but he or she doesn't want to use LaTeX to generate the poster). Or, the person can import the EPS file back into a LaTeX file--sounds weird, but the advantage is that standalone EPS file can be scaled within a floating figure or group of figures. By scaling I mean that the fonts get scaled along with it. If one tries to scale the figure that has unprocessed PS/LaTeX, it may be the case that the fonts don't get scaled down the way that the PostScript does.
One other circumstance where the EPS/LaTeX standalone is of benefit is submitting papers to journals. Often the journal wants the figures separate from the LaTeX code so that they can do the formatting within a collective journal. The editor can't do that so easily if the figures are still comprised of an intermingling of PostScript and LaTeX. They'll sometimes convert the figures to a different format. One never knows exactly how the editing staff does things, but it's just a better feeling to submit a complete EPS file that has the LaTeX fonts within.
Regarding "latex" not processing the output correctly, I submitted a new ticket here:
https://sourceforge.net/p/gnuplot/bugs/1822/
According to this description:
http://tex.stackexchange.com/a/100694
the tikz package doesn't sound too much different than what the pslatex and pstex terminals are doing. Perhaps the advantage of the tikz/pgf is that it can be edited with something like qtikz. (And I suppose perhaps some PDF-specific tooltips or links can be used somehow.) I'm not yet familiar enough with tikz.
OK, I looked around a bit in the code in ~/src for where, if any, the TERM_LAYER_BACKTEXT, TERM_LAYER_FRONTTEXT enumerations are used. The epslatex bug is originating from the following hunks in graphics.c:
/* Sync point for epslatex text positioning */ (term->layer)(TERM_LAYER_BACKTEXT); /* DRAW TICS AND GRID */ if (grid_layer == LAYER_BACK || grid_layer == LAYER_BEHIND) place_grid(grid_layer);and graph3d.c:
/* Sync point for epslatex text positioning */ (term->layer)(TERM_LAYER_BACKTEXT); /* now compute boundary for plot */ boundary3d(plots, pcount);Above is a special call, within the core, that is meant to solve an issue within the driver. I.e., driver-specific code within the core. We've always tried to not do such a thing, and that is what I attempted to do with the proof-of-concept patch. That is, get rid of this special code for a particular terminal.
The bug for this particular case is that the "place_grid" should be run in the TERM_LAYER_FRONTTEXT... but, again, it's better to just allow the natural plot order to play out as with other terminals.
Something else that comes to mind is that the PostScript generated by the PS class of terminals has a lot of definitions in the header of which only a portion ever get used. All of that needs to be processed by the PostScript interpretter. So another way to cut down on the amount of PostScript code for multiple files is to only include necessary definitions. That isn't as difficult as it sounds. Just carry out the drawing code as usual and keep a record of what needs inclusion. When doing a reset then construct the smaller header for the front and meld the two together.
Please keep thinking about this one. At first I din't consider this one too critical, but after seeing the special lines of layer code for epslatex it made me wonder. The pstex family of terminals are the only ones that use the internal layer mechanism, and I think removing that is more beneficial than having multiple EPS files. I can reduce the number of files to just a few with some small mods. All other terminals maintain the layer order.
I still haven't seen a compelling example of a plot that is badly handled by the current mechanism. Yes, you could construct some pathological case with overlapping objects each with it's own text (that you want obscured?!), but isn't that a case of "don't do that"? Simple overlapping text labels are handled correctly, even if they have boxes around them.