gnuplot development

In preparation for eventual release of gnuplot version 4.4, existing bugs
and patchsets are being re-prioritized.

I have chosen to use the following categories

9 - should be included in 4.4.rc1 if possible
7 - looks reasonable to me, but not high priority for 4.4.rc1
5 - default priority for newly submitted patches
2 - not under active consideration


These are my personal ratings. If you want to argue for a different
priority on one or more patches - go right ahead.

Ethan Merritt (sfeam)

Regarding

-------------
I still see a warning message on one machine (but not on the other):
src/Makefile.am:64: variable `gnuplot_lua_SOURCES' is defined but no
program or
src/Makefile.am:64: library has `gnuplot_lua' as canonic name (possible
typo)
-------------

I assume the later is not resolved?

Regarding the former, why not remove the "gnuplot_lua_SOURCES" line from
src/Makefile.am?

The lines you quote would be those immediately _after_ whatever caused
latex to glitch, so they are not informative on their own. You may be able
to find more complete information in the *.log file from the latex run.

Yes, you could upload the problematic *.tex file, but if you do then
please also upload the gnuplot script that created it.

I'm seeing errors with the cvs sources. My latex is complaining about the
lines ...

--------------
/hpt_ 31.5 def
/vpt_ 31.5 def
--------------

the error is

--------------
! Missing $ inserted.
<inserted text>
$
l.39 /hpt_
31.5 def
?
--------------

Say the word, and I'll upload the latex files.

By the way, I've updated term/lua/TODO to note that alpha-channel
transparency already works, so far as I can tell from running the demos.

However, while use of the character \mu works, use of \sigma generates an
error. I am not certain where the error comes from. Both mu and sigma are
in the transparent.dem demo script as UTF-8 characters.

I've manually added gnuplot-tikz.help and gnuplot-tikz.lua to my local copy
of the cvs-sources. I obtained those files from the posted patch.

With those two files added, I'm able to build.

Ben:
You report
> make[3]: *** No rule to make target `../term/lua/gnuplot-tikz.lua',
needed by `gnuplot-tikz.lua'. Stop.

Sigh. I can never remember to check whether a "cvs add foo" is needed
prior to the "cvs commit foo". In this case the files were renamed;
that's enough to make them require an "add" step.
Fixed now.

I still see a warning message on one machine (but not on the other):
src/Makefile.am:64: variable `gnuplot_lua_SOURCES' is defined but no
program or
src/Makefile.am:64: library has `gnuplot_lua' as canonic name (possible
typo)

I think it's harmless, but I'm not sure why it has just started appearing
now.

I'm in unfamiliar territory, but in src/Makefile.am I see

63 if BUILD_LUA
64 pkglibexec_PROGRAMS += gnuplot-tikz.lua
65 gnuplot_lua_SOURCES =
66 gnuplot-tikz.lua: $(top_srcdir)/term/lua/gnuplot-tikz.lua
67 cp $(top_srcdir)/term/lua/gnuplot-tikz.lua .
68 endif

Which does not seem to match what is in term/lua

$ ls -l term/lua
total 48
drwxr-xr-x 5 bpabbott bpabbott 170 2009-05-26 01:04 CVS
-rw-r--r-- 1 bpabbott bpabbott 8961 2009-05-26 01:04 ChangeLog
-rw-r--r-- 1 bpabbott bpabbott 4354 2009-05-26 01:04 INSTALL
-rw-r--r-- 1 bpabbott bpabbott 4497 2009-05-26 01:04 NEWS
-rw-r--r-- 1 bpabbott bpabbott 3123 2009-05-26 01:04 README
-rw-r--r-- 1 bpabbott bpabbott 596 2009-05-26 01:04 TODO
-rw-r--r-- 1 bpabbott bpabbott 11416 2009-05-26 01:04 gnuplot-lua-tikz.sty
-rw-r--r-- 1 bpabbott bpabbott 0 2009-05-26 01:04 gnuplot.lua

It appears (to me) that "gnuplot.lua" was added to the cvs, instead of
"gnuplot-tikz.lua".

I just tried to build, but it failed with the error below. For reference, I
had been able to apply the lua-tikz patch and build.

make[3]: *** No rule to make target `../term/lua/gnuplot-tikz.lua', needed
by `gnuplot-tikz.lua'. Stop.

I've added the 29-Mar-2009 update to CVS after modifying it very slightly
in accordance with the discussion about terminal names as used by Octave.
I left in place the fully general syntax that Peter provided, so 'set term
lua scriptname ...' should work for new variants without having to
recompile gnuplot. But I added a dummy terminal dispatch table for 'tikz'
and added a check so that 'set term tikz' is recognized as short for 'set
term lua tikz'. Because of the dummy dispatch table, 'tikz' appears as a
separate entry in GPVAL_TERMINALS.

I'd still like to see a modification that allows "show term" to work with
the lua terminal. As of now you still can't get confirmation of the
options you tried to set.

You're correct, we have been talking past eachother. Your propsal addresses
all my concerns. Sorry for the confusion.

Somehow we seem to be talking past each other. I propose that the command
is "set term tikz lua", in which case GPVAL_TERMINAL will report "tikz".
And if Octave uses this to decide tikz support is available, that will be
correct. Why would it have to know anything about lua?

Same for the cairo terminals. The command "set term png cairo" would get
you a png terminal, which is all that Octave cares about (I would think,
anyhow). Why does it care whether the undelying library is cairo or gd?
And "set term pdf cairo" gets you pdf output, regardless of whether cairo
or PDFlib is used underneath.

Octave uses the GPVAL_TERMINALS variable to determine if a requested
terminal is available before usiing it. If, for example, "pdf" is not
available, then the postscript terminal is used and ghostscript is used to
convert it (speaking of the developers sources).

At the present time, the list of terminals in GPVAL_TERMINALS is explicit
and unique, and delimited by spaces. If "lua" is listed it is not explicit
or unique (assuming other lua terminals are to be added). If "lua tikz" is
listed it will look like two differnet terminals.

Regarding pngcairo, you meant "cairo png" correct? If not then by analogy,
gnuplot would use "tikz lua" correct? In which case there would be no
problem for me.

I guess I'm failing to see where the problem arises for Octave. If the
syntax is "set term tikz lua", how is that any different from "set term
post eps" or "set term png truecolor"? In all three cases the output file
type is in the 3rd word, and the 4th word further modifies how the output
is formatted or produced. If you omit the 4th word you will still get
useful output. If you omit the "lua" keyword, then you will still get tikz
output, although maybe in some future version it will not have been
produced via lua by default.

I suppose for consistency we would then change "set term pngcairo" to "set
term png cairo", although the two variants are functionally the same so far
as I can see.

As Octave presently uses the GPVAL_TERMINALS variable to determine what
terminals are available, and I don't see how a two word terminal would be
properly listed, I encourage a one word terminal. If two words are to be
used, please ensure that each lua terminal is described in GPVAL_TERMINALS
so that Octave (and others) may determine what terminals are available.

I think that the user cares more about the final output format (tikz) than
the mechanism that the developers used to get there (lua). So if there is
to be either a single short name or a longer name that can be abbreviated,
then I think it should be "tikz" or "tikzlua".

> Well, my personal favorite is still the split "lua tikz", because I like
the idea of simply changing
> the "target name" and having the according script loaded ... still
waiting for more Lua target scripts ;-)

Could it be "set term <foo> lua" instead? With "tikz" being the only
useful value of <foo> so far.

If we decide to go with a single name I would like to suggest "luatikz" or
simply "tikz" because most probably there won't be much compatibility
between this and an alternative implementation. IMHO the ambiguity of
"tikzlua" might annoy the users more than it would help.

Well, my personal favorite is still the split "lua tikz", because I like
the idea of simply changing the "target name" and having the according
script loaded ... still waiting for more Lua target scripts ;-)

But as I said below I am not dogmatic about that and I will provide a new
patch with a single name "luatikz", "tikzlua" or "tikz" ...

If there are no further objections I will rename it to "luatikz".

Sorry for being slow to comment on this.
Would it be possible to use a naming scheme analogous to png/pngcairo
pdf/pdfcairo?
That is, "set term pdf" will find the PDFLib-based terminal if that is the
only one present in the current gnuplot executable; it will find the
cairo-based terminal if that is the only one present. If both are present,
then the PDFLib version is the default and "set term pdfcairo" is required
in order to select the cairo version.

In this case, the full name of the terminal would be tikzlua. "set term
tikz" would find as a short form. If some other tikz terminal is
hypothetically added in the future, then the full name might be required in
order to distinguish which terminal is wanted (assuming both are in the
same executable).

Using an alias is possible in principle, but it would require some
"hardwiring" in the code (see "set_terminal()" in set.c) since there are no
general mechanisms for this. One could argue that this would mess up the
code...


-Peter

Would it be possible to keep "set terminal lua tikz" for backward
compatible/the chance of easily adding new outputs AND having "set terminal
tikz" (or tikzlua), which basically just links to the first.

I am thinking of how "set term table" has been changed to "set table" in
4.3, but the old syntax still works.

AFAICS, going with combined names without a white space would need some
parsing of the terminal names before invoking them. And gnuplot would still
not add them to the list, because the real name is still "lua".
Another, maybe more feasible way, could be to fill a new gnuplot variable
e.g. GPVAL_LUA_TARGETS the same way like GPVAL_TERMINALS. Though, this
seems to be only possible _after_ invoking the lua terminal at least once,
because there are no calls of any terminal functions before :-(

What do you think?

-Peter

Thanks for the explaination.

Can your goal of keeping the generic Lua terminal separated from the
script be managed with terminal names such as "tikzlua"? ... (perhaps
"tikz-lua", or "tikz_lua", or "tikz:lua" would be more clear?) ... and can
gnuplot be made aware of the scripts so that the list of terminals
(GPVAL_TERMINALS) includes all the Lua terminals?

Ben

Yes, you are right about the new syntax.

The idea behind the naming is to keep the generic Lua terminal separated
from the script to allow easy addition of new "target" scripts without
adding new C-code and recompiling gnuplot.

But I see your point regarding Octave and I am not dogmatic about the
naming scheme. So if there are more cons then pros I will provide a new
patch.

Here is a short summary of ideas discussed before:
1) call the terminal "tikz"
2) call it "tikzlua" similar to "pdfcairo"
3) call it "tikz <target name>" and maybe let the Lua terminal show a
list of available target scripts
4) ...

-Peter

opps ... "nobody" is me (forgot to verify I was still logged in).

Ben

Peter/others,

As I understand the terminal syntax, to select the Lua/TikZ ...

"set terminal lua tikz ..."

Correct?

Also, Octave' gnuplot backend presently uses GPVAL_TERMINALS to verify
that a terminal is available before trying to use it. This was motivated by
linux distributions which do not include the pdf terminal (due to licensing
reasons?).

For the present implementation, we can verify "lua" is in the list and
assume TikZ is supported. However, our implemenation will run into some
difficulty as additional terminals are implemented via Lua.

Is there a reason the name of the terminal cannot be called TikZ?

Attached is an updated patch (lua_terminal_rev96_20090329.patch) to the
last one, that also adds the TikZ target's help to the standard
documentation.

The help file for inclusion by "lua.trm" can be generated by calling
lua gnuplot-tikz.lua termhelp > gnuplot-tikz.help

The patch also "renames" the script to "gnuplot-tikz.lua", so just
applying it should do.

Feedback is most welcome!
-Peter

There is a new patch attached (lua_terminal_rev92_20080222.patch) for
testing
against the current CVS version, that mainly changes the terminal naming
scheme
as suggested on the dev-list a few weeks ago.

Here is the list of visible changes:

* Providing a script or target name is now mandatory. The new syntax:
set terminal lua <target name> | "<file name>"
{<script_args> ...}
This will look for a script named `gnuplot-<target name>.lua' or
a script named "<file name>" (in quotes!).
* Support for script names via the environmental variable
GNUPLOT_LUA_SCRIPT is dropped
* The option "script" is obsolete and also removed.
* Via the environmental variable GNUPLOT_DRIVER_DIR the default search
directory for driver scripts can be changed.


The patch does not include the renaming of the Lua script to
'gnuplot-tikz.lua', please do this manually after applying the patch.

Any feedback and bug reports are most welcome.

-Peter

File Added: lua_terminal_rev92_20080222.patch

Please check "[ 2614496 ] gnuplot-lua-tikz.sty generated by gnuplot.lua is
faulty" in the bug section.
Link:
https://sourceforge.net/tracker/index.php?func=detail&aid=2614496&group_id=2055&atid=102055

There is a problem when creating the gnuplot-lua-tikz.sty LaTeX style file
automatically (set term lua createstyle), which stops it from compiling in
LaTeX.

Here is a copy of my last post on the topic including the fix:
---------------------------------------------------------------
The current gnuplot.lua script (rev. 90) looks like that (line 77-78):
pgf.REVISION_DATE = string.gsub("$Date: 2008/12/22 21:44:39 $",
"$Date: ([0-9]+)-([0-9]+)-([0-9]+)
.*","%1/%2/%3")

This causes the gnuplot-lua-tikz.sty to look like (line 8-9):
\ProvidesPackage{gnuplot-lua-tikz}%
[$Date: 2008/12/22 21:44:39 $ (rev. 90) GNUPLOT Lua terminal
style]

This causes itself an error when trying to run it through latex. I am not
quite sure why. I guess the $ signs causes problems as it sets it in Math
mode?! Or the Date: as it looks for the numerical date?! Well someone
would
have to check the PdfLaTeX manual ...

THE SOLUTION:
-------------
So obviously the parsing of the "pgf.REVISION_DATE" variable is wrong.
Commparing with the cvs of gnuplot-lua-tikz.sty yields the requierred
format. So the gnuplot.lua should look like (line 77-78 again):
pgf.REVISION_DATE = string.gsub("$Date: 2008/12/22 21:44:39 $",
"$Date: ([0-9]+)/([0-9]+)/([0-9]+)
.*","%1/%2/%3")

As you can see the hyohens have been replaced by slashes so that the date
is parsed propperly. The gnuplot-lua-tikz.sty style file generated using
"set term lua createstyle" will then look like (line 8-9 again):
\ProvidesPackage{gnuplot-lua-tikz}%
[2008/12/22 (rev. 90) GNUPLOT Lua terminal style]

Which will be compiled by LaTeX without problems!!!
---------------------------------------------------------------

Would be great if someone can update it asap.

Thanks,
Stephan

The complete set of files is now in CVS for 4.3.

I am not familiar with Debian package name conventions, but I do know that
they evaluate and modify the build scripts if necessary. So I expect the
Debian gnuplot package will contain a modified configure.in that correctly
looks for the libraries by their Debian names.

The devel package is installed. The problem is, that the package is not
following the common naming scheme lib<name>.so.<major>.<minor> but is
calling the lib "lua5.1". pkg-config therefore also expects lua5.1 as the
package name. The installed files on Hardy:
/usr/lib/liblua5.1.la
/usr/lib/liblua5.1.a
/usr/lib/liblua5.1.so.0.0.0
/usr/lib/liblua5.1.so.0 -> liblua5.1.so.0.0.0
/usr/lib/liblua5.1.so -> liblua5.1.so.0.0.0

> I had some trouble with it on my Ubuntu Hardy. On Debian based
> distributions
> the Lua package is called "lua5.1" :-(
> I had to change "configure.in"
> - AC_CHECK_LIB(lua,luaL_openlibs,
> + AC_SEARCH_LIBS(luaL_openlibs,lua5.1 lua,

No, this is not correct. It simply means that you have not installed the
lua-devel package (or maybe it's called liblua-devel). The devel package
will create a symlink with the generic name "liblua.so" that points to the
specific version currently installed "liblua5.1.so". Or you could create
such a symlink yourself, but the -devel package may also contain needed
header files. This mechanism allows the build script to work independent
of the current version of the library.

Thank you for the patch!

I had some trouble with it on my Ubuntu Hardy. On Debian based
distributions
the Lua package is called "lua5.1" :-(
I had to change "configure.in"
- AC_CHECK_LIB(lua,luaL_openlibs,
+ AC_SEARCH_LIBS(luaL_openlibs,lua5.1 lua,
and had to use the following configure options
LUA_CFLAGS=-I/usr/include/lua5.1 \
LUA_LIBS=-llua5.1 \
CPPFLAGS=-I/usr/include/lua5.1


There seems to be a compatibility issue with the used preview package and
TikZ. With TeXLive 2008 I get the right bounding box with 'dvips -E -i'
but
cannot see anything inside. dvipdf works like expected. But both fail
with
the CVS version of TikZ.
I did not want to use the geometry package, because of the terminal's
"plotsize"
option, that changes the canvas size.

I am currently working on the other mentioned issues and together with
Mojca on
better compatibility with plain TeX and ConTeXt.
I think it will be easier to send you patches against the CVS version
than
continuing on the patch tracker. And hopefully others will be encouraged
to
send patches, too :-)

Nonetheless I uploaded the most recent development version.


-Peter

File Added: gnuplot_lua_terminal.tgz

Sorry - make that 'tested under TeXLive and TeTeX'

Here's a more complete patch. It places the current contents of your
tarball into .../term/lua/ and .../term/ so that the autoconf tools can
configure and install with no additional intervention.

Comments:

- ./configure currently installs gnuplot-lua-tikz.sty to
/usr/local/share/gnuplot/<version>/ but of course TeX doesn't know to look
for it there. Where is the proper place to install this style file?

- Tested using TeXLive:
pdflatex works great. dvi output is still useless.

- dvips partially works, and may be fixable. It places 90% of the plot off

the bottom of the page, and then clips it so you can't recover even by
editing the BoundingBox.

- Tested using texmf:
Works fine using pdflatex except that I had to hunt for a copy of
ifxetex.sty

- I have not touched the code in lua.trm, but it needs a pass to remove
these warnings:
../term/lua.trm:132: warning: ISO C90 forbids mixed declarations and
code
../term/lua.trm:190: warning: ISO C90 forbids mixed declarations and
code
../term/lua.trm:532: warning: implicit declaration of function
‘luaL_openlibs’
../term/lua.trm:533: warning: implicit declaration of function
‘luaopen_debug’
../term/lua.trm:561: warning: ISO C90 forbids mixed declarations and
code
../term/lua.trm:1023: warning: ISO C90 forbids mixed declarations and
code
../term/lua.trm:125: warning: ‘LUA_stack_dump’ defined but not used

- set term lua should accept at least the following options:
size XX, YY # units of inches or cm
dashed/solid
color/monochrome
and it needs to echo back the options string to the user to confirm it
was accepted

But that's all minor stuff, and can be fixed up after the code goes into
cvs.
What do you think - should the whole lot go into cvs more or less
immediately?
File Added: lua_config_17dec2008.patch

Thanks for the patch.

As for your problem: That's because the tikz driver is not a pure latex
one. It just works with the most important output drivers (i.e. with
dvips, dvipdfm and pdftex). Therefor, run dvips and look at the postscript
output. Or run pdflatex in the first place.

Juergen

I have uploaded a patch that modifies configure.in and term.h.
The revised ./configure will recognize if you have a development package
for lua installed, and if so it builds the lua driver automatically. The
patch by itself does not include the files from your tarball, however. You
still need to copy those to appropriate places.

However, I am a bit lost as to what happens then. I tried running the
suggested test script:
set term lua
set output 'test.tex'
set title "Test of lua terminal"
plot x**2

I then wrap test.tex into a latex document:
\documentclass[10pt]{article}
\usepackage[T1]{fontenc}
\usepackage{textcomp}
\usepackage[utf8x]{inputenc}
\usepackage{gnuplot-lua-tikz}
\begin{document}
\include{test}
\end{document}

I can run "latex wrapper".
It gives no obvious errors.
It creates a *.dvi file.
But the *.dvi doesn't contain a recognizable plot, just some garbled text
near the center of the page that seems to be pieces of my title.

Am I missing some crucial step?
I am running:
lua-5.1.2-4mdv2008.0
texlive-2007-13mdv2008.0
Texlive contains a tikz that claims to be v 1.87 2007/06/07 07:41:10
tantau
File Added: lua_config.patch

I changed the copyright notice to be compatible with gnuplot.

-Peter
File Added: gnuplot_lua_terminal.tgz

I updated the Lua script and put in the changes suggested by Juergen
Wieferink.
Doing this I also straightened the canvas calculation code. Changing the
defaults is now much easier.

From the NEWS file:

* Changed default canvas size to 12.5cm x 8.75cm and use the preview
package to clip the plot in 'fulldoc' mode (suggested by Juergen
Wieferink).

* New option 'charsize'. In conjunction with the gnuplottex.sty the
font size can now be determined automatically. (look at the style file
for
example code).

* Removed the default font setting (previously set to "\small").


-Peter
File Added: gnuplot_lua_terminal.tgz

I have just PMed a patch to change the default document size to
12.5cm x 8.75cm, which is more common among the other terminals.
Additionally the patch changes the paper size of the fulldoc
document. I think the main use case is similar to the
"standalone" option of epslatex: To be able to generate a self
contained PDF which could even be used as graphic. While this
may not be the optimum, it can still be useful for tuning the
output.

Obviously, the change of the default size could be done on the
command line, too. It's just a convenience for the more common case.
The other changes should probably be optional...

Juergen

Juergen, sorry you are right. The memory settings in the texmf.conf have to
be increased. The values I am using are
main_memory = 5000000
extra_mem_top = 2000000
extra_mem_bot = 4000000

The "problem" of the rotated image is, that it is not drawn as a PDF or PS
inline graphic, but of many polygons. The huge amount of drawing operations
then hits the memory limit of TeX.

I am a quite new sourceforge user and did not know that only ticket
creators and admins are allowed to attach files. Anyway, please feel free
to send me files per email if necessary.

-Peter

I use

gnuplot -e 'set term lua fulldoc' -e 'set output "t.tex"' image.dem

in the demo/ subdirectory for generating the output.
Then I edit it by hand to quote all dollar and
percent signs (this is OK, the epslatex terminal behaves the same).

The problem seems to be the rotated figure and the fact that there
are more than one on one page (due to the page size of A4).

====================================8<========================================
Overfull \hbox (236.26631pt too wide) in paragraph at lines 24836--24837
[][]
Runaway argument?
{e17700e07600e07500df7300df7200df7100de6f00de6e00dd6d00dd6c00dd6b00dc\ETC.
! TeX capacity exceeded, sorry [main memory size=3500000].
\pgfsysprotocol@temp ...d00be2c00be2b00bd2a00bd2a0

0bc2900bc2800bb2800bb2700b...
l.25917 ...{128}{7.506}{6.607}{\gprawrgbimagedata}

! ==> Fatal error occurred, no output PDF file produced!
====================================8<========================================



As I am neither a dev nor the creator of this ticket, I cannot attach
a file. I use the TeX version shipped with SuSE with 10.3, named
texlive-latex-2007-69, but I'm not sure if SuSE really has all tools
up to date...

Juergen

Juergen, thanks for testing! Would you mind sending me your gnuplot file so
that I can have a look into it. I am using TeXLive 2008 and at least the
penguin demo works flawlessly.

I am not much into this autotool world, so yes, I need someone to do this
;-)

-Peter

I've had some deeper look into it. It seems to have quite a clean design.

The image demo did not work, though, because the TeX capacity was too
small.
I don't think this is this terminal's fault.

You will need someone to autoconfiscate the lua dependence and the
installation
of gnuplot.lua.

Juergen

Happy you like it!

AFAIK there is no "standard" for the canvas size among the terminals.
That's why you always need to change the size or scaling by hand if you are
not satisfied with the default settings. Actually I don't remember why I
settled with a square... maybe because it is more convenient to use the
"scale" option with a quadratic canvas. That does not mean that it couldn't
be changed.

To the standalone mode: One nice thing about using Lua is that you can
easily change things without recompiling. If you look at the Lua script you
will find

208 pgf.write_doc_begin = function(preamble)
209 gp.write("\\documentclass[10pt,a4paper]{scrartcl}\n"
210 .."\\usepackage[T1]{fontenc}\n"
211 .."\\usepackage{textcomp}\n\n"
212 .."\\usepackage[utf8x]{inputenc}\n\n"
213 .."\\usepackage{"..pgf.LATEX_STYLE_FILE.."}\n"
214 ..preamble.."\n\n"
215 .."\\begin{document}\n")
216 end


You can also use a local copy of the script by using the "script" option.

Concerning the license: I don't have any objections in changing the
license if necessary :-)


-Peter

Thank you for finally submitting this
very promising terminal. The resulting
LaTeX code seems to be very clean and even
more editable and shorter than that of the
postscript terminal. I would really
like to see this in.

That said: Why does the size default to 10cm
times 10cm. Landscape seems quite more
common to me. The standalone (or fulldoc) mode
does not set the page size (and \pagestyle{empty}).
Could this be done easily?

Are you aware of the gnuplot license? I don't think
it is compatible with the GPL.

Juergen