gnuplot-beta — Developers and beta testers of new gnuplot code

Re: Key documentation consistency

[Daniel J S. <dan...@ie...>]

On 07/08/2016 04:23 PM, Ethan A Merritt wrote:
> On Friday, 08 July, 2016 15:17:25 Daniel J Sebald wrote:
>> The documentation for key indicates:
>>
>>    The defaults for `set key` are `on`, `right`, `top`, `vertical`, `Right`,
>>    `noreverse`, `noinvert`, `samplen 4`, `spacing 1.25`, `title ""`, and
>>    `nobox`.  The default <linetype> is the same as that used for the plot
>>    borders.  Entering `set key default` returns the key to its default
>>    configuration.
>>
>> and when I do the following:
>>
>> gnuplot> plot x, -x
>> gnuplot> show key
>>
>> 	key is ON, position: top right vertical inside
>> 	key is right justified, not reversed, not inverted, enhanced and not boxed
>> 	sample length is 4 characters
>> 	vertical spacing is 1 characters
>> 	width adjustment is 0 characters
>> 	height adjustment is 0 characters
>> 	curves are automatically titled with filename
>> 	maximum number of columns is calculated automatically
>> 	maximum number of rows is calculated automatically
>>
>> 	key title is ""
>>
>> The vertical spacing says 1 character, not 1.25.  Mistake?  Change in
>> code without change in documentation?
>
> The default vertical spacing is 1 * (1.25 characters)

That's 1.25 characters.  The printout from show key indicates 1 
characters.  The lack of consistency is

default                 value in character units
-------                 ------------------------
`samplen 4`    => 'sample length is 4 characters'
`spacing 1.25` => 'vertical spacing is 1 characters'

If the printout indicated 'vertical space is 1', i.e., unitless, then it 
could be a scale factor.  But then it would make sense that samplen 
should be a scale factor as well.

Dan

Re: Key documentation consistency

[Ethan A M. <sf...@us...>]

On Friday, 08 July, 2016 15:17:25 Daniel J Sebald wrote:
> The documentation for key indicates:
> 
>   The defaults for `set key` are `on`, `right`, `top`, `vertical`, `Right`,
>   `noreverse`, `noinvert`, `samplen 4`, `spacing 1.25`, `title ""`, and
>   `nobox`.  The default <linetype> is the same as that used for the plot
>   borders.  Entering `set key default` returns the key to its default
>   configuration.
> 
> and when I do the following:
> 
> gnuplot> plot x, -x
> gnuplot> show key
> 
> 	key is ON, position: top right vertical inside
> 	key is right justified, not reversed, not inverted, enhanced and not boxed
> 	sample length is 4 characters
> 	vertical spacing is 1 characters
> 	width adjustment is 0 characters
> 	height adjustment is 0 characters
> 	curves are automatically titled with filename
> 	maximum number of columns is calculated automatically
> 	maximum number of rows is calculated automatically
> 
> 	key title is ""
> 
> The vertical spacing says 1 character, not 1.25.  Mistake?  Change in 
> code without change in documentation?  

The default vertical spacing is 1 * (1.25 characters)

Key documentation consistency

[Daniel J S. <dan...@ie...>]

The documentation for key indicates:

  The defaults for `set key` are `on`, `right`, `top`, `vertical`, `Right`,
  `noreverse`, `noinvert`, `samplen 4`, `spacing 1.25`, `title ""`, and
  `nobox`.  The default <linetype> is the same as that used for the plot
  borders.  Entering `set key default` returns the key to its default
  configuration.

and when I do the following:

gnuplot> plot x, -x
gnuplot> show key

	key is ON, position: top right vertical inside
	key is right justified, not reversed, not inverted, enhanced and not boxed
	sample length is 4 characters
	vertical spacing is 1 characters
	width adjustment is 0 characters
	height adjustment is 0 characters
	curves are automatically titled with filename
	maximum number of columns is calculated automatically
	maximum number of rows is calculated automatically

	key title is ""

The vertical spacing says 1 character, not 1.25.  Mistake?  Change in 
code without change in documentation?  Also, that space between max rows 
and key setting isn't needed.

Dan

Re: A key sample for pm3d surfaces?

[Daniel J S. <dan...@ie...>]

On 07/07/2016 02:33 PM, Daniel J Sebald wrote:
> On 07/07/2016 01:53 PM, Ethan A Merritt wrote:
>> On Thursday, 07 July, 2016 12:19:21 Daniel J Sebald wrote:
>>> I'm trying to add a key sample for a pm3d surface.  I know there is the
>>> color box and all, but it seems odd that there can be a title entry in
>>> the key but no sample.  I can sort of kludge a solution by leaving the
>>> surface title empty and then create a bogus line with NaN's that will
>>> contain the sample and title, linewidth 8.  For the Qt terminal, the
>>> default is rounded line ends so the key sample at linewidth 8 is round.
>>>
>>> Basically, it would be nice to have a sample entry for surface similar
>>> in appearance to the filledcurve or candlestick of 2D plots.  I wonder
>>> if it would make sense to have a "sample" descriptor in plot/splot lines
>>> similar to the "title" descriptor.
>>
>> The problem with using a colored key sample for anything with pm3d
>> coloring (surfaces or anything else) is that it is not clear how to
>> choose a color for the sample.   You might for example have 3 plots
>> in a single figure using the same pm3d palette, one plot predominantly
>> green, one predominantly blue, and one predominantly yellow.
>> You would like the key to match, but how would the program know this?
>
> That's a good example of the use.  As a default, key sample is sort of
> meaningless for surfaces--I suppose something like average color might
> show predominant color and somehow make surfaces distinct.  The default
> could be blank, but I was thinking something that the user could
> specify, e.g.,
>
>     plot 'foo.dat' with pm3d title 'green component' sample rgb 'green'
>
> where sample for pm3d would be have FILLSTYLE bit set so that it comes
> out as a patch rather than a line.  I've kind of achieved this with an
> extra bogus line of width 11 to contain the title.  It's roughly the
> same size as a key sample patch.

Actually, the most general key sample has a fill color/pattern and an 
outline.  For example the third demo plot in 'violinplot.dem' (titled 
"Same data - kernel density") has samples where the fill color and 
outline match that of the filled curves on the plot.  Right now a 
mesh/pm3d are separate entities even though a lot of the example plots 
mimic a mesh will fill by plotting the same function with different 
styles.  So there is no way of achieving the same sort of key sample in 
3D that filledcurves is doing in 2D... unless having a manual key sample 
control (short term), or generalizing "lines" in splot to "mesh" (long 
term).

Dan

Re: A key sample for pm3d surfaces?

[Daniel J S. <dan...@ie...>]

On 07/07/2016 01:53 PM, Ethan A Merritt wrote:
> On Thursday, 07 July, 2016 12:19:21 Daniel J Sebald wrote:
>> I'm trying to add a key sample for a pm3d surface.  I know there is the
>> color box and all, but it seems odd that there can be a title entry in
>> the key but no sample.  I can sort of kludge a solution by leaving the
>> surface title empty and then create a bogus line with NaN's that will
>> contain the sample and title, linewidth 8.  For the Qt terminal, the
>> default is rounded line ends so the key sample at linewidth 8 is round.
>>
>> Basically, it would be nice to have a sample entry for surface similar
>> in appearance to the filledcurve or candlestick of 2D plots.  I wonder
>> if it would make sense to have a "sample" descriptor in plot/splot lines
>> similar to the "title" descriptor.
>
> The problem with using a colored key sample for anything with pm3d
> coloring (surfaces or anything else) is that it is not clear how to
> choose a color for the sample.   You might for example have 3 plots
> in a single figure using the same pm3d palette, one plot predominantly
> green, one predominantly blue, and one predominantly yellow.
> You would like the key to match, but how would the program know this?

That's a good example of the use.  As a default, key sample is sort of 
meaningless for surfaces--I suppose something like average color might 
show predominant color and somehow make surfaces distinct.  The default 
could be blank, but I was thinking something that the user could 
specify, e.g.,

   plot 'foo.dat' with pm3d title 'green component' sample rgb 'green'

where sample for pm3d would be have FILLSTYLE bit set so that it comes 
out as a patch rather than a line.  I've kind of achieved this with an 
extra bogus line of width 11 to contain the title.  It's roughly the 
same size as a key sample patch.

Dan

Re: Alpha channel transparency value consistency

[Daniel J S. <dan...@ie...>]

On 07/07/2016 01:44 PM, Ethan A Merritt wrote:
> On Thursday, 07 July, 2016 12:01:27 Daniel J Sebald wrote:
>> I'm working on something with alpha channel and noticed that the alpha
>> channel 0 to 1 range behaves differently for color specification and for
>> input data.  For color spec (help colorspec):
>>
>>    "#AARRGGBB" represents an RGB color with an alpha channel (transparency)
>>    value in the high bits. An alpha value of 0 represents a fully opaque
>> color;
>>    i.e., "#00RRGGBB" is the same as "#RRGGBB".
>>
>> and for rgbalpha (help alpha):
>>
>>    The `rgbalpha` plotting style assumes that each pixel of input data
>> contains
>>    an alpha value in the range [0:255].  A pixel with alpha = 0 is purely
>>    transparent and does not alter the underlying contents of the plot. A
>> pixel
>>    with alpha = 255 is purely opaque.
>>
>> Was there a reason for the opposite correlation of the opacity/transparency?
>
> Removing the inconsistency between line colors and image pixel colors
> was on the list of possible major changes in gnuplot version 5.
> There was a bit of discussion at the time, but no one argued strongly
> enough for unification to outweigh the clear disadvantages:
> - breaking backward compatibility
> - inherent conflict of obvious defaults:
>    line color #00RRGGBB "obviously" should be the same as #RRGGBB
>      but
>    image processing has historically used alpha=0 as fully transparent
>
> This same inconsistency of whether "0" is fully transparent or fully
> opaque is present in real-world image formats and in other program
> implementations.

Yeah, I just had an example where the software's documentation referred 
to "transparency", but 1 in the range [0,1] is opaque.  In some sense, 
the solution is to correlate the range with a description.  In this case 
"alpha" is nebulous and takes on two meanings.  There could be

   rgbtransparent
   rgbopaque

rather than rgbalpha.  Can't really do such a thing with "line color 
#[]RRGGBB" though.

Unless, perhaps a meaning were to be given to alpha, e.g.,

   set alpha {transparent|opaque}

Dan

Re: A key sample for pm3d surfaces?

[Ethan A M. <sf...@us...>]

On Thursday, 07 July, 2016 12:19:21 Daniel J Sebald wrote:
> I'm trying to add a key sample for a pm3d surface.  I know there is the 
> color box and all, but it seems odd that there can be a title entry in 
> the key but no sample.  I can sort of kludge a solution by leaving the 
> surface title empty and then create a bogus line with NaN's that will 
> contain the sample and title, linewidth 8.  For the Qt terminal, the 
> default is rounded line ends so the key sample at linewidth 8 is round.
> 
> Basically, it would be nice to have a sample entry for surface similar 
> in appearance to the filledcurve or candlestick of 2D plots.  I wonder 
> if it would make sense to have a "sample" descriptor in plot/splot lines 
> similar to the "title" descriptor.

The problem with using a colored key sample for anything with pm3d
coloring (surfaces or anything else) is that it is not clear how to
choose a color for the sample.   You might for example have 3 plots
in a single figure using the same pm3d palette, one plot predominantly 
green, one predominantly blue, and one predominantly yellow.
You would like the key to match, but how would the program know this?

	Ethan

Re: Alpha channel transparency value consistency

[Ethan A M. <sf...@us...>]

On Thursday, 07 July, 2016 12:01:27 Daniel J Sebald wrote:
> I'm working on something with alpha channel and noticed that the alpha 
> channel 0 to 1 range behaves differently for color specification and for 
> input data.  For color spec (help colorspec):
> 
>   "#AARRGGBB" represents an RGB color with an alpha channel (transparency)
>   value in the high bits. An alpha value of 0 represents a fully opaque 
> color;
>   i.e., "#00RRGGBB" is the same as "#RRGGBB".
> 
> and for rgbalpha (help alpha):
> 
>   The `rgbalpha` plotting style assumes that each pixel of input data 
> contains
>   an alpha value in the range [0:255].  A pixel with alpha = 0 is purely
>   transparent and does not alter the underlying contents of the plot. A 
> pixel
>   with alpha = 255 is purely opaque.
> 
> Was there a reason for the opposite correlation of the opacity/transparency?

Removing the inconsistency between line colors and image pixel colors 
was on the list of possible major changes in gnuplot version 5.
There was a bit of discussion at the time, but no one argued strongly
enough for unification to outweigh the clear disadvantages:
- breaking backward compatibility
- inherent conflict of obvious defaults:
  line color #00RRGGBB "obviously" should be the same as #RRGGBB
    but
  image processing has historically used alpha=0 as fully transparent

This same inconsistency of whether "0" is fully transparent or fully
opaque is present in real-world image formats and in other program 
implementations.

	Ethan

A key sample for pm3d surfaces?

[Daniel J S. <dan...@ie...>]

I'm trying to add a key sample for a pm3d surface.  I know there is the 
color box and all, but it seems odd that there can be a title entry in 
the key but no sample.  I can sort of kludge a solution by leaving the 
surface title empty and then create a bogus line with NaN's that will 
contain the sample and title, linewidth 8.  For the Qt terminal, the 
default is rounded line ends so the key sample at linewidth 8 is round.

Basically, it would be nice to have a sample entry for surface similar 
in appearance to the filledcurve or candlestick of 2D plots.  I wonder 
if it would make sense to have a "sample" descriptor in plot/splot lines 
similar to the "title" descriptor.

Dan

Alpha channel transparency value consistency

[Daniel J S. <dan...@ie...>]

I'm working on something with alpha channel and noticed that the alpha 
channel 0 to 1 range behaves differently for color specification and for 
input data.  For color spec (help colorspec):

  "#AARRGGBB" represents an RGB color with an alpha channel (transparency)
  value in the high bits. An alpha value of 0 represents a fully opaque 
color;
  i.e., "#00RRGGBB" is the same as "#RRGGBB".

and for rgbalpha (help alpha):

  The `rgbalpha` plotting style assumes that each pixel of input data 
contains
  an alpha value in the range [0:255].  A pixel with alpha = 0 is purely
  transparent and does not alter the underlying contents of the plot. A 
pixel
  with alpha = 255 is purely opaque.

Was there a reason for the opposite correlation of the opacity/transparency?

Dan

Bad grammar in "key" documentation

[Daniel J S. <dan...@ie...>]

In the documentation for 'key' is the following phrase: "can only 
estimate the correctly exact width" which is confusing because of 
conflicting adjectives and adverbs.  Attached is a short diff rewriting 
portions of that paragraph.

Dan

Re: --with-readline=DIR doesn't work with CVS HEAD

[Bastian M. <bma...@we...>]

Am 21.06.2016 um 10:12 schrieb Jun T.:
> "--with-readline=/usr/local" does not work on Mac OS X after the
> following patch (configure.ac, revision 1.21):
> ----------------------------
> revision 1.21
> date: 2016/05/23 19:02:18;  author: markisch;  state: Exp;  lines: +34 -36
> Detect if --with-readline=DIR refers to GNU readline or NetBSD editline.
> ----------------------------
>
> On Mac OS X, Apple installs editline (libedit) in /usr/{include,lib}
> but I want to use GNU readline instead. So I've installed GNU readline
> in /usr/local/{include,lib}, and was configuring gnuplot as:
>
> ./configure --with-readline=/usr/local ...
>
> This was working fine, but after the above patch configure started to use
> /usr/lib/libedit.dylib even if "--with-readline=/usr/local" is specified.
> I want to use /usr/local/lib/libreadline.dylib instead.
>
> The problem is at line 447 of configure.ac:
>
>       AC_CHECK_HEADERS(editline/readline.h,[with_readline=bsd],,)
>
> This finds /usr/include/editline/readline.h and set with_readline=bsd
> even if -I/usr/local/include is added to CPPFLAGS (configure.ac:441).
>
> A possible patch is attached.
>
> Jun

Thank you very much for the report and fix.  Now in CVS.

   Bastian

Re: Lua / Tikz / pgf works for pdflatex only ?

[Daniel J S. <dan...@ie...>]

On 06/29/2016 01:44 AM, Mojca Miklavec wrote:
> On 29 June 2016 at 04:19, Daniel J Sebald wrote:
>> On 06/28/2016 08:06 PM, Mojca Miklavec wrote:
>>>
>>> On 28 June 2016 at 23:38, Daniel J Sebald wrote:
>>>>
>>>> Does anyone familiar with Tikz/pgf know why the output from pdflatex
>>>> terminal or "lua tikz" terminal should not work with the normal "latex"
>>>> command line processing as opposed to requiring "pdflatex"?
>>>>
>>>> https://sourceforge.net/p/gnuplot/bugs/1822/
>>>
>>>
>>> The "plot sin(x)" as opposed to "test" works for me. I would suspect
>>> fill patterns, but there might of course be any other reason for the
>>> failure.
>>
>> That sounds slightly different from what I'm experiencing (and should be
>> fixed, as I think the TikZ shouldn't have too much trouble with patterns,
>> judging from how extensive the manual is).
>
> Officially not. But bugs keep popping up and some combinations of
> features and engines are simply too rarely used and tested. In ConTeXt
> not even things like "color=red!20!yellow" work properly and there are
> other problems.
>
> But again, I didn't claim that this was in fact the problem, it was
> just a quick thought because simple graphs work just fine for me.
>
>> It sounds like you are at least
>> able to get the TikZ code to work with "latex".  I'm not getting that far.
>> The PostScript files that "latex" is generating from TikZ/PGF targeted code
>> is missing the definition "pgfo".  That definition is in the file
>>
>> /usr/share/texmf/tex/generic/pgf/systemlayer/pgfsys-dvips.def
>>
>> Right, I suppose that is what Chapter 10 of the TikZ/PGF manual addresses.
>> The manual does suggest that dvips should work, by somehow including the
>> pgfsys-dvips.def file.  (It contains PostScript definitions required for
>> dvips.)  But I can't seem to include those definitions no matter what I
>> include or redefine in the TikZ-targeted LaTeX code.
>
> The official way to do that is supposed to be
>      \def\pgfsysdriver{pgfsys-dvips.def}
> before including TikZ, but dvips *is* already the default driver when
> you compile with "latex", redefining the sysdriver only makes sense if
> you plan to use dvipdfmx or anything like that. So you won't achieve
> anything by including pgfsys-dvips.def.

I found that out.  My best guess is that the preview LaTeX package is 
undoing some of what the pgfsys-dvips.def definitions did.


>> On what system are you running "latex", and what version?  I have
>
> I don't believe the system makes any difference, but it's an old OS X
> version with the latest TeX Live.
>
>> latex --version
> pdfTeX 3.14159265-2.6-1.40.17 (TeX Live 2016)
> kpathsea version 6.2.2
> ...
> Compiled with libpng 1.6.21; using libpng 1.6.21
> Compiled with zlib 1.2.8; using zlib 1.2.8
> Compiled with xpdf version 3.04
>
>> sebald@ ~ $ latex --version
>> pdfTeX 3.1415926-2.5-1.40.14 (TeX Live 2013/Debian)
>> kpathsea version 6.1.1
>
> The TeX Live version does make quite some difference though. 2013 is
> quite a bit old and many packages have changed during the three years.

I figured that.  I'm not about to upgrade though because I think it 
might be a lot of work.  I'll leave that to Mint.


> Anyway, it seems that the preview package was indeed the fault. Other
> suggestions in the ticket work for me in principle, but I didn't yet
> figure out how to fix the bounding box (I haven't used dvi for more
> than a decade at least and I'm no longer a LaTeX user either).
>
>> the patterns are different
>> from qt.  (I think patterns are terminal specific.)
>
> Indeed. Gnuplot doesn't even prescribe what the terminals could use to
> be at least somewhat compatible.
>
>> The one difference is
>> that the result from "latex" on the tikz-tex output does not have the alpha
>> blending, while "pdflatex" processing produces an output with alpha blending
>> of the hexagons.
>
> PostScript doesn't support transparencies, but for some reason (that I
> don't quite understand) I'm getting the alpha blending in the final
> PDF even via the tex->dvi->ps->pdf route. (I'm using GhostScript's
> ps2pdf to convert from PS to PDF.)

Oh, sure enough; that works here as well.  Let's look at the PostScript 
code.  Here looks to be the closed path definition for the green hexagon:

  save
  0.03984 pgfw
  0 1 0 setrgbcolor
  0.50 .pgfsetstrokeopacityalpha
  0.50 .pgfsetfillopacityalpha
  283.4681 197.01022 moveto
  274.59543 212.34586 lineto
  256.87868 212.34586 lineto
  248.03458 197.01022 lineto
  256.87868 181.64647 lineto
  274.59543 181.64647 lineto
  closepath

The definition for .pgfsetfillopacityalpha looks to be some creative 
conditional, probably interpreted differently in the PostScript screen 
driver than when sent to ps2pdf.


> It would probably be best if there was a way to fix the problem in
> preview directly.

Do you see nicely cropped plots (i.e., no missing border in test.pdf, or 
extra white pixels outside the border) from the preview package?

Dan

Re: Lua / Tikz / pgf works for pdflatex only ?

[Mojca M. <moj...@gm...>]

On 29 June 2016 at 04:19, Daniel J Sebald wrote:
> On 06/28/2016 08:06 PM, Mojca Miklavec wrote:
>>
>> On 28 June 2016 at 23:38, Daniel J Sebald wrote:
>>>
>>> Does anyone familiar with Tikz/pgf know why the output from pdflatex
>>> terminal or "lua tikz" terminal should not work with the normal "latex"
>>> command line processing as opposed to requiring "pdflatex"?
>>>
>>> https://sourceforge.net/p/gnuplot/bugs/1822/
>>
>>
>> The "plot sin(x)" as opposed to "test" works for me. I would suspect
>> fill patterns, but there might of course be any other reason for the
>> failure.
>
> That sounds slightly different from what I'm experiencing (and should be
> fixed, as I think the TikZ shouldn't have too much trouble with patterns,
> judging from how extensive the manual is).

Officially not. But bugs keep popping up and some combinations of
features and engines are simply too rarely used and tested. In ConTeXt
not even things like "color=red!20!yellow" work properly and there are
other problems.

But again, I didn't claim that this was in fact the problem, it was
just a quick thought because simple graphs work just fine for me.

> It sounds like you are at least
> able to get the TikZ code to work with "latex".  I'm not getting that far.
> The PostScript files that "latex" is generating from TikZ/PGF targeted code
> is missing the definition "pgfo".  That definition is in the file
>
> /usr/share/texmf/tex/generic/pgf/systemlayer/pgfsys-dvips.def
>
> Right, I suppose that is what Chapter 10 of the TikZ/PGF manual addresses.
> The manual does suggest that dvips should work, by somehow including the
> pgfsys-dvips.def file.  (It contains PostScript definitions required for
> dvips.)  But I can't seem to include those definitions no matter what I
> include or redefine in the TikZ-targeted LaTeX code.

The official way to do that is supposed to be
    \def\pgfsysdriver{pgfsys-dvips.def}
before including TikZ, but dvips *is* already the default driver when
you compile with "latex", redefining the sysdriver only makes sense if
you plan to use dvipdfmx or anything like that. So you won't achieve
anything by including pgfsys-dvips.def.

> On what system are you running "latex", and what version?  I have

I don't believe the system makes any difference, but it's an old OS X
version with the latest TeX Live.

> latex --version
pdfTeX 3.14159265-2.6-1.40.17 (TeX Live 2016)
kpathsea version 6.2.2
...
Compiled with libpng 1.6.21; using libpng 1.6.21
Compiled with zlib 1.2.8; using zlib 1.2.8
Compiled with xpdf version 3.04

> sebald@ ~ $ latex --version
> pdfTeX 3.1415926-2.5-1.40.14 (TeX Live 2013/Debian)
> kpathsea version 6.1.1

The TeX Live version does make quite some difference though. 2013 is
quite a bit old and many packages have changed during the three years.

Anyway, it seems that the preview package was indeed the fault. Other
suggestions in the ticket work for me in principle, but I didn't yet
figure out how to fix the bounding box (I haven't used dvi for more
than a decade at least and I'm no longer a LaTeX user either).

> the patterns are different
> from qt.  (I think patterns are terminal specific.)

Indeed. Gnuplot doesn't even prescribe what the terminals could use to
be at least somewhat compatible.

> The one difference is
> that the result from "latex" on the tikz-tex output does not have the alpha
> blending, while "pdflatex" processing produces an output with alpha blending
> of the hexagons.

PostScript doesn't support transparencies, but for some reason (that I
don't quite understand) I'm getting the alpha blending in the final
PDF even via the tex->dvi->ps->pdf route. (I'm using GhostScript's
ps2pdf to convert from PS to PDF.)

It would probably be best if there was a way to fix the problem in
preview directly.

Mojca

Re: Lua / Tikz / pgf works for pdflatex only ?

[Ethan M. <eam...@gm...>]

On Tuesday, June 28, 2016 10:45:18 PM Daniel J Sebald wrote:
> On 06/28/2016 09:19 PM, Daniel J Sebald wrote:
> > On 06/28/2016 08:06 PM, Mojca Miklavec wrote:
> >> On 28 June 2016 at 23:38, Daniel J Sebald wrote:
> > Right, I suppose that is what Chapter 10 of the TikZ/PGF manual
> > addresses.  The manual does suggest that dvips should work, by somehow
> > including the pgfsys-dvips.def file.  (It contains PostScript
> > definitions required for dvips.)  But I can't seem to include those
> > definitions no matter what I include or redefine in the TikZ-targeted
> > LaTeX code.
> 
> I've isolated the problem on my system to the inclusion of the preview
> package.  

See:
  http://tex.stackexchange.com/questions/37982/problem-with-preview-package-and-standalone-class-with-tikz

Re: Lua / Tikz / pgf works for pdflatex only ?

[Daniel J S. <dan...@ie...>]

On 06/28/2016 09:19 PM, Daniel J Sebald wrote:
> On 06/28/2016 08:06 PM, Mojca Miklavec wrote:
>> On 28 June 2016 at 23:38, Daniel J Sebald wrote:

> Right, I suppose that is what Chapter 10 of the TikZ/PGF manual
> addresses.  The manual does suggest that dvips should work, by somehow
> including the pgfsys-dvips.def file.  (It contains PostScript
> definitions required for dvips.)  But I can't seem to include those
> definitions no matter what I include or redefine in the TikZ-targeted
> LaTeX code.

I've isolated the problem on my system to the inclusion of the preview 
package.  So I've gotten a bit further, and will keep investigating on 
the bug/ticket tracker:

https://sourceforge.net/p/gnuplot/bugs/1822/

Dan

PS: test works for both latex and pdflatex, but the patterns are 
different from qt.  (I think patterns are terminal specific.)  The one 
difference is that the result from "latex" on the tikz-tex output does 
not have the alpha blending, while "pdflatex" processing produces an 
output with alpha blending of the hexagons.

Re: Lua / Tikz / pgf works for pdflatex only ?

[Daniel J S. <dan...@ie...>]

On 06/28/2016 08:06 PM, Mojca Miklavec wrote:
> On 28 June 2016 at 23:38, Daniel J Sebald wrote:
>> Does anyone familiar with Tikz/pgf know why the output from pdflatex
>> terminal or "lua tikz" terminal should not work with the normal "latex"
>> command line processing as opposed to requiring "pdflatex"?
>>
>> https://sourceforge.net/p/gnuplot/bugs/1822/
>
> The "plot sin(x)" as opposed to "test" works for me. I would suspect
> fill patterns, but there might of course be any other reason for the
> failure.

That sounds slightly different from what I'm experiencing (and should be 
fixed, as I think the TikZ shouldn't have too much trouble with 
patterns, judging from how extensive the manual is).  It sounds like you 
are at least able to get the TikZ code to work with "latex".  I'm not 
getting that far.  The PostScript files that "latex" is generating from 
TikZ/PGF targeted code is missing the definition "pgfo".  That 
definition is in the file

/usr/share/texmf/tex/generic/pgf/systemlayer/pgfsys-dvips.def


>> If there is a good reason, should we put in the output LaTeX/TeX code
>> some type of error message that complains when compiled without pdflatex?
>
> That would be a very bad idea in my opinion.
>
> First of all please note that many graphics (like "plot sin(x)") work
> even when compiled via dvi->ps route. There's probably just something
> wrong with pattern fill or one of those special features which might
> just as well be a bug in TikZ (using pdflatex makes a lot more sense
> nowadays than going through dvi, so I doubt that special features in
> TikZ really get any extensive testing in the old workflow).
>
> The other problem is that it is very tricky if not impossible to
> determine the engine being used. People could use dvipdfmx instead of
> dvips to process the files, or XeTeX, LuaTeX, upTeX or any other
> exotic workflow. How exactly would you decide when to break the
> compilation and when not?

Right, I suppose that is what Chapter 10 of the TikZ/PGF manual 
addresses.  The manual does suggest that dvips should work, by somehow 
including the pgfsys-dvips.def file.  (It contains PostScript 
definitions required for dvips.)  But I can't seem to include those 
definitions no matter what I include or redefine in the TikZ-targeted 
LaTeX code.

On what system are you running "latex", and what version?  I have

sebald@ ~ $ latex --version
pdfTeX 3.1415926-2.5-1.40.14 (TeX Live 2013/Debian)
kpathsea version 6.1.1

Thanks,

Dan

Re: Lua / Tikz / pgf works for pdflatex only ?

[Mojca M. <moj...@gm...>]

On 28 June 2016 at 23:38, Daniel J Sebald wrote:
> Does anyone familiar with Tikz/pgf know why the output from pdflatex
> terminal or "lua tikz" terminal should not work with the normal "latex"
> command line processing as opposed to requiring "pdflatex"?
>
> https://sourceforge.net/p/gnuplot/bugs/1822/

The "plot sin(x)" as opposed to "test" works for me. I would suspect
fill patterns, but there might of course be any other reason for the
failure.

> If there is a good reason, should we put in the output LaTeX/TeX code
> some type of error message that complains when compiled without pdflatex?

That would be a very bad idea in my opinion.

First of all please note that many graphics (like "plot sin(x)") work
even when compiled via dvi->ps route. There's probably just something
wrong with pattern fill or one of those special features which might
just as well be a bug in TikZ (using pdflatex makes a lot more sense
nowadays than going through dvi, so I doubt that special features in
TikZ really get any extensive testing in the old workflow).

The other problem is that it is very tricky if not impossible to
determine the engine being used. People could use dvipdfmx instead of
dvips to process the files, or XeTeX, LuaTeX, upTeX or any other
exotic workflow. How exactly would you decide when to break the
compilation and when not?

Mojca

Lua / Tikz / pgf works for pdflatex only ?

[Daniel J S. <dan...@ie...>]

Does anyone familiar with Tikz/pgf know why the output from pdflatex 
terminal or "lua tikz" terminal should not work with the normal "latex" 
command line processing as opposed to requiring "pdflatex"?

https://sourceforge.net/p/gnuplot/bugs/1822/

If there is a good reason, should we put in the output LaTeX/TeX code 
some type of error message that complains when compiled without pdflatex?

Dan

--with-readline=DIR doesn't work with CVS HEAD

[Jun T. <tak...@kb...>]

"--with-readline=/usr/local" does not work on Mac OS X after the
following patch (configure.ac, revision 1.21):
----------------------------
revision 1.21
date: 2016/05/23 19:02:18;  author: markisch;  state: Exp;  lines: +34 -36
Detect if --with-readline=DIR refers to GNU readline or NetBSD editline.
----------------------------

On Mac OS X, Apple installs editline (libedit) in /usr/{include,lib}
but I want to use GNU readline instead. So I've installed GNU readline
in /usr/local/{include,lib}, and was configuring gnuplot as:

./configure --with-readline=/usr/local ...

This was working fine, but after the above patch configure started to use
/usr/lib/libedit.dylib even if "--with-readline=/usr/local" is specified.
I want to use /usr/local/lib/libreadline.dylib instead.

The problem is at line 447 of configure.ac:

      AC_CHECK_HEADERS(editline/readline.h,[with_readline=bsd],,)               

This finds /usr/include/editline/readline.h and set with_readline=bsd
even if -I/usr/local/include is added to CPPFLAGS (configure.ac:441).

A possible patch is attached.

Jun

Re: conflict of separator and missing

[Ethan A M. <sf...@us...>]

On Saturday, 18 June, 2016 00:18:31 pl...@pi... wrote:
> On 17/06/16 20:19, Ethan A Merritt wrote:
> > On Friday, 17 June, 2016 08:42:46 pl...@pi... wrote:
> >
> >> Can you comment on whether there is a reason to strip leading WS but not
> >> strip trailing WS ( as is currently done )?  This seems a little odd and
> >> is unlikely to be a combination that one would expect.
> >>
> >> Unless there is a positive reason for this choice or downside that I'm
> >> missing, It would seem more consistent to strip both ends.
> >
> > You may be over-thinking this.
> > Gnuplot does no "stripping" or other pre-processing of the input line.
> > Successive fields are read by standard calls to the C library.
> >
> > For a numeric field this is either atod() or sscanf().
> > In either case the C language formatted input routine
> > 1) skips over any leading whitespace,
> > 2) parses the number, and
> > 3) stops at the first character that is not part of the number.
> > That next character could be anything.
> > In other words, skipping any leading whitespace and ignoring any
> > trailing garbage is all normal behaviour for the libc input routines.
> >
> > For a string field (e.g. 'plot with labels') it's a bit more complicated.
> > In this case yes, unquoted leading and trailing whitespace is eventually
> > stripped but this happens at a later stage, not while parsing the input.
> > Also some escape-character sequences are translated.
> >
> > For detection of a "missing" flag? Well, that's what we're discussing.
> > This thread started with the example of using a numeric value as a
> > "missing" flag, which adds an additional layer of ambiguity.
> > If you think of it as substituting for a number, then trailing characters
> > should be ignored.  If you think of it as a string, then trailing
> > characters including whitespace are potentially significant.  If it
> > really were a string then at a later stage any trailing whitespace would
> > be deleted but nothing in the current input layer (datafile.c) does this,
> > and this is the layer that has to decide whether the current record
> > is missing or not.
> >
> >> It seems that this choice may have been motived by the comment issue and
> >> maybe making the comment behaviour more consistent, as outlined above,
> >> would neatly resolve both.
> >
> > The only comment issue I see is that the documentation could be more
> > clear that it is talking about command lines rather than data files.
> > That much is easy to fix.
> >
> > 	Ethan
> >
> 
> Thanks Ethan.
> 
> last things first:
> 
> gnuplot> help comment
>   Comments are supported as follows: a `#` may appear in most places in 
> a line
>   and `gnuplot` will ignore the rest of the line.  It will not have this 
> effect
>   inside quotes, inside numbers (including complex numbers), inside command
>   substitutions, etc.  In short, it works anywhere it makes sense to work.
> 
>   See also `set datafile commentschars` for specifying comment characters in
>   data files.  Note that if a comment line ends in '\' then the subsequent
>   line is also treated as a comment.
> 
> 
> I don't understand your comment that this is about command lines. Here 
> it is explicitly referring to 'datafile'.

The section you quote is talking only about comments in a command line.
I amended the text earlier today to make this more clear.

It then refers you to a separate section about comments in data files,
which says
%%%%%
 `set datafile commentschars` tells `gnuplot` what characters are used in a
 data file to begin comment lines. If the first non-blank character on a line
 is one of the specified characters then the rest of the input line is ignored.
%%%%%

> What you have previously referred to as  end of line 'garbage' is 
> documented as a feature. This contradicts what you previously said about 
> comment char only being legit at the beginning of a line. The help says 
> : `#` may appear in most places in a line.  I was not mistaken in doing 
> this, I was following the doc.

I hope the amended documentation is clearer.
The key points are

1) A comment can start anywhere on a command line, and may affect
multiple physical lines if the initial lines end in a backslash \.

2) The only comment option in a data file is to comment out an entire 
single line by putting a special character at the start of the line.
A backslash at the end of a data file line has no special meaning.

3) The comment character used for datafiles can be something other
than # but still it is only looked for as the first character on the line.

	Ethan


> 
> 
> 
> Secondly, 'missing' flag:
> 
> To clarify , I never thought the missing string was a number even 
> though, in this case it represents a number. I have always been clear 
> that it is treated as a string.
> set datafile missing "-99.99"    # it's a string.
> 
> 
> Thanks for the detailed explanation, I see the problem: evolutionary 
> growth of the code.
> 
>  > If it really were a string then at a later stage any trailing 
> whitespace would
>  > be deleted but nothing in the current input layer (datafile.c) does this,
>  > and this is the layer that has to decide whether the current record
>  > is missing or not.
> 
> Then this is crux of the problem.
> 
> If all separators cases were made to follow what is described in help: 
> ie commentchars can appear 'almost anywhere', then thing would get 
> simpler and be more consistent. The input line needs to be truncated at 
> the first occurrence of  'commentschars' and this new position becomes 
> an implicit field terminator when the line is parsed for field positions.
> 
> This sounds like one line of code.
> 
> Explicitly stripping off comments at an early stage would seem to be 
> desirable, since hoping they will fall of the end as 'garbage' is non 
> consistent between WS and non-WS separated files.
> 
> Then, since 'missing' is a string, it needs to be processed as a string 
> in a similar way that you say is done later for real strings.
> 
> Once a particular field has been isolated between two field terminators, 
> it needs to be checked against 'missing' string.  Std functions will 
> test for presence of missing in the field. Then a check needs to be made 
> to see whether there is anything other than WS before or after.
> 
> set datafile missing "ignore"
> 1,2,3,4, ignore
> 1,2,3,4, ignore\t
> 1,2,3,4, ignore   # comment
> 1,2,3,4, ignore_not
> 1,2,3,4, ignore not
> 
> To my mind, in  the last two examples the last field does not match 
> missing.
> 
> The last line does not match for the same reason the following line does 
> not. Both contain invalid data.
> 1,2,3,4, ignore not,6
> 
> 
> It seems that ensuring comments work as documented even for csv files 
> actually helps in producing overall consistent behaviour for nearly zero 
> effort and processing time.
> 
> Help should indicate that leading or trailing space is not  allowed in 
> 'set datafile missing'.
> 
> /my2c/
> 
> Peter.

Re: conflict of separator and missing

[<pl...@pi...>]

On 17/06/16 20:19, Ethan A Merritt wrote:
> On Friday, 17 June, 2016 08:42:46 pl...@pi... wrote:
>
>> Can you comment on whether there is a reason to strip leading WS but not
>> strip trailing WS ( as is currently done )?  This seems a little odd and
>> is unlikely to be a combination that one would expect.
>>
>> Unless there is a positive reason for this choice or downside that I'm
>> missing, It would seem more consistent to strip both ends.
>
> You may be over-thinking this.
> Gnuplot does no "stripping" or other pre-processing of the input line.
> Successive fields are read by standard calls to the C library.
>
> For a numeric field this is either atod() or sscanf().
> In either case the C language formatted input routine
> 1) skips over any leading whitespace,
> 2) parses the number, and
> 3) stops at the first character that is not part of the number.
> That next character could be anything.
> In other words, skipping any leading whitespace and ignoring any
> trailing garbage is all normal behaviour for the libc input routines.
>
> For a string field (e.g. 'plot with labels') it's a bit more complicated.
> In this case yes, unquoted leading and trailing whitespace is eventually
> stripped but this happens at a later stage, not while parsing the input.
> Also some escape-character sequences are translated.
>
> For detection of a "missing" flag? Well, that's what we're discussing.
> This thread started with the example of using a numeric value as a
> "missing" flag, which adds an additional layer of ambiguity.
> If you think of it as substituting for a number, then trailing characters
> should be ignored.  If you think of it as a string, then trailing
> characters including whitespace are potentially significant.  If it
> really were a string then at a later stage any trailing whitespace would
> be deleted but nothing in the current input layer (datafile.c) does this,
> and this is the layer that has to decide whether the current record
> is missing or not.
>
>> It seems that this choice may have been motived by the comment issue and
>> maybe making the comment behaviour more consistent, as outlined above,
>> would neatly resolve both.
>
> The only comment issue I see is that the documentation could be more
> clear that it is talking about command lines rather than data files.
> That much is easy to fix.
>
> 	Ethan
>

Thanks Ethan.

last things first:

gnuplot> help comment
  Comments are supported as follows: a `#` may appear in most places in 
a line
  and `gnuplot` will ignore the rest of the line.  It will not have this 
effect
  inside quotes, inside numbers (including complex numbers), inside command
  substitutions, etc.  In short, it works anywhere it makes sense to work.

  See also `set datafile commentschars` for specifying comment characters in
  data files.  Note that if a comment line ends in '\' then the subsequent
  line is also treated as a comment.


I don't understand your comment that this is about command lines. Here 
it is explicitly referring to 'datafile'.

What you have previously referred to as  end of line 'garbage' is 
documented as a feature. This contradicts what you previously said about 
comment char only being legit at the beginning of a line. The help says 
: `#` may appear in most places in a line.  I was not mistaken in doing 
this, I was following the doc.



Secondly, 'missing' flag:

To clarify , I never thought the missing string was a number even 
though, in this case it represents a number. I have always been clear 
that it is treated as a string.
set datafile missing "-99.99"    # it's a string.


Thanks for the detailed explanation, I see the problem: evolutionary 
growth of the code.

 > If it really were a string then at a later stage any trailing 
whitespace would
 > be deleted but nothing in the current input layer (datafile.c) does this,
 > and this is the layer that has to decide whether the current record
 > is missing or not.

Then this is crux of the problem.

If all separators cases were made to follow what is described in help: 
ie commentchars can appear 'almost anywhere', then thing would get 
simpler and be more consistent. The input line needs to be truncated at 
the first occurrence of  'commentschars' and this new position becomes 
an implicit field terminator when the line is parsed for field positions.

This sounds like one line of code.

Explicitly stripping off comments at an early stage would seem to be 
desirable, since hoping they will fall of the end as 'garbage' is non 
consistent between WS and non-WS separated files.

Then, since 'missing' is a string, it needs to be processed as a string 
in a similar way that you say is done later for real strings.

Once a particular field has been isolated between two field terminators, 
it needs to be checked against 'missing' string.  Std functions will 
test for presence of missing in the field. Then a check needs to be made 
to see whether there is anything other than WS before or after.

set datafile missing "ignore"
1,2,3,4, ignore
1,2,3,4, ignore\t
1,2,3,4, ignore   # comment
1,2,3,4, ignore_not
1,2,3,4, ignore not

To my mind, in  the last two examples the last field does not match 
missing.

The last line does not match for the same reason the following line does 
not. Both contain invalid data.
1,2,3,4, ignore not,6


It seems that ensuring comments work as documented even for csv files 
actually helps in producing overall consistent behaviour for nearly zero 
effort and processing time.

Help should indicate that leading or trailing space is not  allowed in 
'set datafile missing'.

/my2c/

Peter.

Re: conflict of separator and missing

[Ethan A M. <sf...@us...>]

On Friday, 17 June, 2016 08:42:46 pl...@pi... wrote:

> Can you comment on whether there is a reason to strip leading WS but not 
> strip trailing WS ( as is currently done )?  This seems a little odd and 
> is unlikely to be a combination that one would expect.
> 
> Unless there is a positive reason for this choice or downside that I'm 
> missing, It would seem more consistent to strip both ends.

You may be over-thinking this.
Gnuplot does no "stripping" or other pre-processing of the input line.
Successive fields are read by standard calls to the C library.

For a numeric field this is either atod() or sscanf().
In either case the C language formatted input routine 
1) skips over any leading whitespace, 
2) parses the number, and
3) stops at the first character that is not part of the number.
That next character could be anything.
In other words, skipping any leading whitespace and ignoring any 
trailing garbage is all normal behaviour for the libc input routines.

For a string field (e.g. 'plot with labels') it's a bit more complicated.
In this case yes, unquoted leading and trailing whitespace is eventually
stripped but this happens at a later stage, not while parsing the input.
Also some escape-character sequences are translated.

For detection of a "missing" flag? Well, that's what we're discussing.
This thread started with the example of using a numeric value as a
"missing" flag, which adds an additional layer of ambiguity.  
If you think of it as substituting for a number, then trailing characters
should be ignored.  If you think of it as a string, then trailing 
characters including whitespace are potentially significant.  If it
really were a string then at a later stage any trailing whitespace would
be deleted but nothing in the current input layer (datafile.c) does this,
and this is the layer that has to decide whether the current record
is missing or not.

> It seems that this choice may have been motived by the comment issue and 
> maybe making the comment behaviour more consistent, as outlined above, 
> would neatly resolve both.

The only comment issue I see is that the documentation could be more
clear that it is talking about command lines rather than data files.
That much is easy to fix.

	Ethan

Re: conflict of separator and missing

[<pl...@pi...>]

On 17/06/16 00:11, Ethan A Merritt wrote:
> On Thursday, 16 June, 2016 20:46:00 pl...@pi... wrote:
>> On 16/06/16 19:44, Ethan A Merritt wrote:
>>> On Thursday, 16 June, 2016 18:00:00 pl...@pi... wrote:
>>>> On 16/06/16 17:23, sfeam wrote:
>>>>
>>>>
>>>>> -   The comparison is to a string, not a numerical value, so -99.00 ne -99.0 ne -99
>>>> Sounds reasonable. From the gnuplot POV it is a missing *string* ; if
>>>> the cvs is output by a spreadsheet or other software it seems reasonable
>>>> to expect consistent string formatting ( although Excel could have
>>>> different cell formats, that is probably too much to try and anticipate.
>>>>
>>>>
>>>>  >-   Leading whitespace is ignored but trailing whitespace is not.
>>>>
>>>> Seems inconsistent. Was this a programming convenience for minimal
>>>> coding changes or is there a functional logic behind this?
>>>
>>> I though the consensus from a couple of days ago was that any difference
>>> in the remainder of the field was significant, hence extra trailing
>>> characters would mean that the match was imperfect.
>>> Previously "missing A" and "missing B" were both matched as "missing".
>>> Now they are not, even if A is a <tab> or '\n' or '\r'.
>>
>> I don't know what the consensus was but my comment on that was that
>> trailing WS should be stripped, as it is with leading WS.  I was
>> suggesting that any non-WS following the missing string meant the match
>> failed. Specifically relating to your  " ignore A" case.  I did not
>> suggest  WS"ignore"WS should fail.
>>
>> If leading space is stripped, I'm not sure I see why trailing is not
>> also stripped.
>>
>>
>>
>>>
>>>>  > ... and requires that the next character is a field-terminator.
>>>>
>>>> I presume field-terminator.means FS or EOL.
>>>
>>> Separator or null.
>>> EOL is legal with in a csv field, although if you have such a file good
>>> luck to you. When a line of data is read in to gnuplot it is transferred
>>> to a null-terminated string, so the check for null should catch the true
>>> end-of-line.
>>>
>>>> Does this cater for WS at end of line without an explicit FS,
>>>> or does this fall foul of previous point?
>>>
>>> You mean like a DOS-style file with <cr><nl> at the end of the line?
>>> So far as I know this is properly handled by stripping away both line
>>> termination characters on input.  But more testing wouldn't hurt.
>>>
>>> 	Ethan
>>>
>>
>> No , I was not talking about CRLF end of line.
>>
>> It is quite common to have 'invisible' WS   after the last field and
>> being the last field probably no FS.  This is especially the case if
>> there was a comment :  WS to provide visual separation or align  comments:
>>
>> 1,2,3,-999             # last column data got lost in paper records !
>
> A comment character is only valid at the start of a data line.
> A trailing comment like the one you show will be treated as
> extraneous garbage in the last field.
>
> Prior to yesterday's change, if "missing" were set to "-999" then the
> rest of the field it would be ignored because of the intervening whitespace.
> Since today the "missing" test will fail because "-999" is not followed
> immediately by a field separator.
>
> Do you think it should revert to terminating the "missing" check
> on whitespace?  That was the example I tried to give by showing
> that
>   set datafile missing "missing"
> would catch both fields 2 and 3 in a line containing
>    1, missing A, missing B, 4
>
>> Once the # is replaced by #0 to truncate out the comment ,
>
> Such replacement does not happen.

Oops. It seems like I've been relying " extraneous garbage in the last 
field" for my comments fro some time !  This may be why:

gnuplot> help comment
  Comments are supported as follows: a `#` may appear in most places in 
a line
  and `gnuplot` will ignore the rest of the line.  It will not have this 
effect
  inside quotes, inside numbers (including complex numbers), inside command
  substitutions, etc.  In short, it works anywhere it makes sense to work.

That behaviour should probably be consistent w.r.t changes in separator. 
If  a `#` may appear in most places in a line will get cropped for WS 
files , it should work for CSV files. The example data file line that I 
gave should truncate in both formats.

Gnuplot is remarkably good at sorting out almost any file in what seems 
like an intuitive way and that is something that I find quite impressive.

Is there any reason not to support comments at end of line by simply 
changing # ( or whatever the commentchars are set to ) to #0 as I 
incorrectly thought was being done?

Moving towards a consistent behaviour would seem preferable to reverting 
because of this difference.


Can you comment on whether there is a reason to strip leading WS but not 
strip trailing WS ( as is currently done )?  This seems a little odd and 
is unlikely to be a combination that one would expect.

Unless there is a positive reason for this choice or downside that I'm 
missing, It would seem more consistent to strip both ends.

It seems that this choice may have been motived by the comment issue and 
maybe making the comment behaviour more consistent, as outlined above, 
would neatly resolve both.

Peter.



>
>> this line  would fall foul of your new scheme I think.
>
> Yes, it will fail.
> So should I partially revert the change to restore checking for
> "missing" only up to the first whitespace?
>
> 	Ethan
>
>
>
>
>> I see no real reason not to remove the tailing WS , it seems a little
>> odd to strip one end an not the other.
>>
>> CSV is pretty illegible at the best of times. If need to dump a
>> spreadsheet to CSV I often separate with " , " to make the result a
>> little easier to read afterwards.
>>
>> Unless I'm missing something , I don't see any reason or advantage to
>> not stripping trailing WS.
>>
>>
>> Not wishing to be finicky, but you seemed interesting is considering any
>> corner cases.
>>
>> Peter.
>>
>>
>>
>>
>>>
>>>
>>>
>>> 	Ethan
>>>
>>
>>

Re: conflict of separator and missing

[Ethan A M. <sf...@us...>]

On Thursday, 16 June, 2016 20:46:00 pl...@pi... wrote:
> On 16/06/16 19:44, Ethan A Merritt wrote:
> > On Thursday, 16 June, 2016 18:00:00 pl...@pi... wrote:
> >> On 16/06/16 17:23, sfeam wrote:
> >>
> >>
> >>> -   The comparison is to a string, not a numerical value, so -99.00 ne -99.0 ne -99
> >> Sounds reasonable. From the gnuplot POV it is a missing *string* ; if
> >> the cvs is output by a spreadsheet or other software it seems reasonable
> >> to expect consistent string formatting ( although Excel could have
> >> different cell formats, that is probably too much to try and anticipate.
> >>
> >>
> >>  >-   Leading whitespace is ignored but trailing whitespace is not.
> >>
> >> Seems inconsistent. Was this a programming convenience for minimal
> >> coding changes or is there a functional logic behind this?
> >
> > I though the consensus from a couple of days ago was that any difference
> > in the remainder of the field was significant, hence extra trailing
> > characters would mean that the match was imperfect.
> > Previously "missing A" and "missing B" were both matched as "missing".
> > Now they are not, even if A is a <tab> or '\n' or '\r'.
> 
> I don't know what the consensus was but my comment on that was that 
> trailing WS should be stripped, as it is with leading WS.  I was 
> suggesting that any non-WS following the missing string meant the match 
> failed. Specifically relating to your  " ignore A" case.  I did not 
> suggest  WS"ignore"WS should fail.
> 
> If leading space is stripped, I'm not sure I see why trailing is not 
> also stripped.
> 
> 
> 
> >
> >>  > ... and requires that the next character is a field-terminator.
> >>
> >> I presume field-terminator.means FS or EOL.
> >
> > Separator or null.
> > EOL is legal with in a csv field, although if you have such a file good
> > luck to you. When a line of data is read in to gnuplot it is transferred
> > to a null-terminated string, so the check for null should catch the true
> > end-of-line.
> >
> >> Does this cater for WS at end of line without an explicit FS,
> >> or does this fall foul of previous point?
> >
> > You mean like a DOS-style file with <cr><nl> at the end of the line?
> > So far as I know this is properly handled by stripping away both line
> > termination characters on input.  But more testing wouldn't hurt.
> >
> > 	Ethan
> >
> 
> No , I was not talking about CRLF end of line.
> 
> It is quite common to have 'invisible' WS   after the last field and 
> being the last field probably no FS.  This is especially the case if 
> there was a comment :  WS to provide visual separation or align  comments:
> 
> 1,2,3,-999             # last column data got lost in paper records !

A comment character is only valid at the start of a data line.
A trailing comment like the one you show will be treated as
extraneous garbage in the last field. 

Prior to yesterday's change, if "missing" were set to "-999" then the 
rest of the field it would be ignored because of the intervening whitespace.
Since today the "missing" test will fail because "-999" is not followed
immediately by a field separator.

Do you think it should revert to terminating the "missing" check
on whitespace?  That was the example I tried to give by showing
that 
  set datafile missing "missing"
would catch both fields 2 and 3 in a line containing
   1, missing A, missing B, 4

> Once the # is replaced by #0 to truncate out the comment , 

Such replacement does not happen.

> this line  would fall foul of your new scheme I think.

Yes, it will fail.
So should I partially revert the change to restore checking for
"missing" only up to the first whitespace?

	Ethan




> I see no real reason not to remove the tailing WS , it seems a little 
> odd to strip one end an not the other.
>
> CSV is pretty illegible at the best of times. If need to dump a 
> spreadsheet to CSV I often separate with " , " to make the result a 
> little easier to read afterwards.
> 
> Unless I'm missing something , I don't see any reason or advantage to 
> not stripping trailing WS.
> 
> 
> Not wishing to be finicky, but you seemed interesting is considering any 
> corner cases.
> 
> Peter.
> 
> 
> 
> 
> >
> >
> >
> > 	Ethan
> >
> 
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity planning
> reports. http://sdm.link/zohomanageengine
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta