gnuplot-beta — Developers and beta testers of new gnuplot code

Re: [PATCH] cairolatex: respect transparency setting

[Henri M. <hen...@gm...>]

Now there seems to a problem with ordering of the elements in the
resulting TeX file.  The ticklabels are behind the background image.
Even if transparency is enabled, this is probably a bad idea.  I'll
investigate.

Cheers, Henri

On 21/08/20, 13:34, Henri Menke wrote:
> Currently the cairolatex terminal does not respect transparency options
> given in the terminal setting.  In this minimal example the
> notransparent option is ignored:
> 
>     set terminal cairolatex png notransparent
>     set output "test.tex"
>     plot sin(x)
> 
> Running this through gnuplot and then using `file' reveals that the
> alpha channel is still present:
> 
>     $ gnuplot test.gnuplot && file test.png
>     test.png: PNG image data, 1500 x 900, 8-bit/color RGBA, non-interlace
> 
> With this patch the transparency setting will be honoured.  The current
> default setting of implicitly assuming transparency is unaltered.
> ---
>  term/cairo.trm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/term/cairo.trm b/term/cairo.trm
> index ee748dfe2..3ef76ecc6 100644
> --- a/term/cairo.trm
> +++ b/term/cairo.trm
> @@ -849,7 +849,7 @@ void cairotrm_graphics()
>  	plot.background.g = cairo_params->background.g;
>  	plot.background.b = cairo_params->background.b;
>  	gp_cairo_set_background(cairo_params->background);
> -	if (ISCAIROLATEX || cairo_params->transparent)
> +	if (cairo_params->transparent)
>  		gp_cairo_clear_background(&plot);
>  	else
>  		gp_cairo_solid_background(&plot);
> -- 
> 2.28.0
> 

[PATCH] cairolatex: respect transparency setting

[Henri M. <hen...@gm...>]

Currently the cairolatex terminal does not respect transparency options
given in the terminal setting.  In this minimal example the
notransparent option is ignored:

    set terminal cairolatex png notransparent
    set output "test.tex"
    plot sin(x)

Running this through gnuplot and then using `file' reveals that the
alpha channel is still present:

    $ gnuplot test.gnuplot && file test.png
    test.png: PNG image data, 1500 x 900, 8-bit/color RGBA, non-interlace

With this patch the transparency setting will be honoured.  The current
default setting of implicitly assuming transparency is unaltered.
---
 term/cairo.trm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/term/cairo.trm b/term/cairo.trm
index ee748dfe2..3ef76ecc6 100644
--- a/term/cairo.trm
+++ b/term/cairo.trm
@@ -849,7 +849,7 @@ void cairotrm_graphics()
 	plot.background.g = cairo_params->background.g;
 	plot.background.b = cairo_params->background.b;
 	gp_cairo_set_background(cairo_params->background);
-	if (ISCAIROLATEX || cairo_params->transparent)
+	if (cairo_params->transparent)
 		gp_cairo_clear_background(&plot);
 	else
 		gp_cairo_solid_background(&plot);
-- 
2.28.0

Re: Sideways violin plots in one plot cmd?

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> I guess I still don't get it.
> Since the wrapper can pass in any necessary set of commands,
> why not just pass in the commands that do the double plot?

Because my tools would then need special-case logic for this one case. I
would need some sort of "feedgnuplot --violin" and logic internally to
handle it. At the very start I decided that explicitly supporting
everything gnuplot supports with such special-case logic would be
extremely effortful and extremely error-prone and maintenance-full. And
I avoided the issue entirely by passing user-specified strings to
gnuplot transparently. Which works great for most styles.

If (as a random example) I have 3-column input that I want to plot as
errorbars I can do this:

  < 3columns_datafile feedgnuplot --domain --tuplesizeall 3 --with yerrorbars

feegnuplot then knows to be dealing with 3-values-per-point data, and
tells gnuplot to plot with "yerrorbars". There's no special-case logic
for "yerrorbars". And I don't want to create a separate path for violin
plots.

Full disclosure: there IS one special-case path to deal with histograms.
But those are extremely useful, and worth violating the general design.
violin plots aren't nearly as ubiquitous, and I dont want to add a big
special-case path for them.

Re: Sideways violin plots in one plot cmd?

[Juhász P. <pet...@gm...>]

On Sat, 2020-08-15 at 13:17 -0700, Ethan A Merritt wrote:
> On Friday, 14 August 2020 10:34:52 PDT Juhász Péter wrote:
> > On Wed, 2020-08-05 at 23:46 -0700, Dima Kogan wrote:
> > > A question. I'm looking at the violin plot demo page:
> > > 
> > >   http://gnuplot.sourceforge.net/demo/violinplot.html
> > > 
> > > It shows how to make a horizontal density plot using 'smooth
> > > kdensity'.
> > > 
> > > Then it shows to to make a vertical density plot by plotting
> > > horizontally into a table, and then plotting the table, with
> > > "using"
> > > directives used to swap the axes. This works clearly, but is
> > > there a
> > > way
> > > to do this (vertical density plots) with a single plot command?
> > > That
> > > would make my life much easier.
> > > 
> > > Thanks!
> > > 
> > > 
> > > 
> > 
> > Well, recently at $work I used gnuplot to produce a sideways
> > histogram.
> > Because it had to be sideways, I had to resort to manually
> > assembling
> > the plot with the boxxyerror style, with an awkward preprocessing
> > step.
> > 
> > This, and your question, gives me the idea that it might be worth
> > adding a general 'transpose' option to some of the "clever" plot
> > styles
> > that produce non-trivial vertical plots (histograms and boxplots,
> > mostly).
> > 
> > best regards,
> > Peter Juhasz
> 
> "How to make a sideways plot" does seem to be a very
> frequently asked question.  There may be some clever way to select
> an x/y axis transposition in the general case, but if so I have not
> been
> able to find it.  
> 

I might have mislead the discussion with my suggestion, because as it
turns out, the only common thing it has with the original violin plot
question is "would there be a way to do X in a single command". Sorry
for the confusion.

That said, I don't think bringing in 3D (s)plot styles is a good
direction towards a general solution for sideways plots. To me, it
sounds like looking for our lost car keys in the corner only because
that's where there is light.

> The program was designed from the start to have a single 2D plot mode
> with x always horizontal, [...]

Yes, originally it was designed like that, but then more complex plot
styles like histograms and boxplots were added, which do non-trivial
preprocessing and aggregating steps before plotting the data. For
boxplots (and less acutely, even for histograms) the x axis of the plot
is no longer directly maps to any column of the original data.

> Peter - would the plot you constructed
> by hand be a good example to adapt?  If the combination
>     set view projection xz
>     splot ... with boxerror
> doesn't handle it adequately, that would highlight where it would
> be worth some additional work on the code. 

No, not really, unless I misunderstand you - my point was that I wanted
to produce (IIRC) a columnstacked histogram, and if it could have been
a vertical plot, I could have used `set style histogram columnstacked;
set style data histogram`, with all the nice intuitive data layout and
all the automatic box placement that style provides. But the
requirements called for a horizontal plot, so I had to do all that by
hand. Doing the same in 3D wouldn't have helped.

Peter

> 
> 	Ethan
> 
> 

Re: Sideways violin plots in one plot cmd?

[Ethan A M. <me...@uw...>]

On Saturday, 15 August 2020 22:06:34 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> >
> > I don't get it. What is it about the shell interface that would prevent you
> > from doing exactly what is in violinplot.dem?
> >   set table $datablock 
> >   plot ...
> >   unset table
> >   plot $datablock ...
> 
> Both gnuplotlib and feedgnuplot work as very thin wrappers around
> gnuplot, where they pass strings directly to gnuplot without knowing
> what the strings mean. This allows those interfaces to support lots and
> lots of styles, while containing no code that makes those styles work:
> all the logic is inside gnuplot itself. The double-plotting in
> violinplot.dem is qualitatively different than pretty much every other
> gnuplot style, so I would need to add special-case support for it. I
> COULD do that, but I really don't want to: the really simple API of
> feedgnuplot and gnuplotlib is a feature, and this special-case path
> would dilute that feature.
> 
> In a perfect world we'd be able to make violin plots like all others:
> with some "set" commands (that don't have the data in it) and a single
> plot/splot command that is given data. That is sounding like a big
> project inside gnuplot, though.


I guess I still don't get it.
Since the wrapper can pass in any necessary set of commands,
why not just pass in the commands that do the double plot?

Would it help to have a library of helper scripts for gnuplot?
Here's one that creates a vertical violin plot from a single gnuplot
command.  You could obviously add a lot of customization either
by passing in additional parameters or defining them globally.

   gnuplot> set term pngcairo
   gnuplot> set output 'violin.png'
   gnuplot> # unit width violin plot centered at x=10., linetype 3
   gnuplot> call 'violin.gp' 'filename.dat' 10. 3 

violin.gp script follows
%%%%%%%%%%%%%%%%%%%%%%%%
#
# invoke this script from inside gnuplot using
#       call violin.gp filename xcenter linetype
#
filename = ARGV[1]
xcenter = ARGV[2]
LT = ARGV[3]
#
set xrange [*:*]
set table $kdensity
plot filename using 2:(1) smooth kdensity with filledcurves above x1
unset table
stats $kdensity using 2 noout
plot [0:20][0:*]\
     $kdensity using (xcenter + column(2)/STATS_max):1 with filledcurves x=xcenter lt LT, \
     $kdensity using (xcenter - column(2)/STATS_max):1 with filledcurves x=xcenter lt LT
#
%%%%%%%%%%%%%%%%%%%%%%%%%

Re: Sideways violin plots in one plot cmd?

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:
>
> I don't get it. What is it about the shell interface that would prevent you
> from doing exactly what is in violinplot.dem?
>   set table $datablock 
>   plot ...
>   unset table
>   plot $datablock ...

Both gnuplotlib and feedgnuplot work as very thin wrappers around
gnuplot, where they pass strings directly to gnuplot without knowing
what the strings mean. This allows those interfaces to support lots and
lots of styles, while containing no code that makes those styles work:
all the logic is inside gnuplot itself. The double-plotting in
violinplot.dem is qualitatively different than pretty much every other
gnuplot style, so I would need to add special-case support for it. I
COULD do that, but I really don't want to: the really simple API of
feedgnuplot and gnuplotlib is a feature, and this special-case path
would dilute that feature.

In a perfect world we'd be able to make violin plots like all others:
with some "set" commands (that don't have the data in it) and a single
plot/splot command that is given data. That is sounding like a big
project inside gnuplot, though.

Re: Sideways violin plots in one plot cmd?

[Ethan A M. <me...@uw...>]

On Saturday, 15 August 2020 16:37:35 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> > Are you trying to avoid the extra step of saving to an array or
> > datablock and then replotting? That seems like a totally different
> > question from vertical/horizontal layout or coordinate
> > transformations.
> 
> Yes. Avoiding that extra step is the question in this thread; see the
> subject. I've never tried to plot sideways histograms, like Peter did; I
> was assuming that this is simple if we plot into a datablock and then
> replot, but maybe not.
> 
> In any case, here's my problem. I use gnuplot A LOT. But 99% of that
> usage is via one of two wrappers (I wrote both of these):
> 
>   feedgnuplot (a nice shell interface)
>   gnuplotlib (a nice python interface)
> 
> Both of these work as thin wrappers around gnuplot, the idea being that
> with some "set" commands and a fancy "plot" command you can do anything.
> As it currently stands, violin plots are apparently not a part of this
> "anything". One could say that my wrappers are silly and not
> flexible-enough. Which maybe is true, but realistically speaking, I
> don't see ever expanding these tools to use datablocks, since that would
> break all sorts of interface assumptions both of these tools are making.

I don't get it. What is it about the shell interface that would prevent you
from doing exactly what is in violinplot.dem?
  set table $datablock 
  plot ...
  unset table
  plot $datablock ...

	Ethan

Re: Sideways violin plots in one plot cmd?

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> Are you trying to avoid the extra step of saving to an array or
> datablock and then replotting? That seems like a totally different
> question from vertical/horizontal layout or coordinate
> transformations.

Yes. Avoiding that extra step is the question in this thread; see the
subject. I've never tried to plot sideways histograms, like Peter did; I
was assuming that this is simple if we plot into a datablock and then
replot, but maybe not.

In any case, here's my problem. I use gnuplot A LOT. But 99% of that
usage is via one of two wrappers (I wrote both of these):

  feedgnuplot (a nice shell interface)
  gnuplotlib (a nice python interface)

Both of these work as thin wrappers around gnuplot, the idea being that
with some "set" commands and a fancy "plot" command you can do anything.
As it currently stands, violin plots are apparently not a part of this
"anything". One could say that my wrappers are silly and not
flexible-enough. Which maybe is true, but realistically speaking, I
don't see ever expanding these tools to use datablocks, since that would
break all sorts of interface assumptions both of these tools are making.
So I can live without violin plots, or maybe go use matplotlib. Which
I'd rather not do.

Any suggestions appreciated.

Re: Sideways violin plots in one plot cmd?

[Ethan A M. <me...@uw...>]

On Saturday, 15 August 2020 13:51:32 PDT Dima Kogan wrote:
> If we add better support for these, I think supporting more general
> transformations than just a simple x/y transpose would be useful. While
> thinking of violin plots I had a workaround idea: instead of a set of
> horizontally-spaced vertical density plots, I'd be fine with a set of
> vertically-spaced horizontal density plots. But I think gnuplot
> currently can't do that either: the density plots must be horizontal AND
> they must start at y=0. Right? So being able to support translations
> would be nice.

I may not understand what you want.
kdensity smoothing is a sum of Gaussians, so by definition the baseline
is zero.   That is not a limitation of how gnuplot plots things, that is
the definition of the function being plotted.    You can of course
save the resulting curve and plot it with an offset on y, just as the 
violin plot demo replots it vertically with an offset on x.

Are you trying to avoid the extra step of saving to an array or
datablock and then replotting?   That seems like a totally different
question from vertical/horizontal layout or coordinate transformations.

	Ethan

Re: Sideways violin plots in one plot cmd?

[Dima K. <gn...@di...>]

If we add better support for these, I think supporting more general
transformations than just a simple x/y transpose would be useful. While
thinking of violin plots I had a workaround idea: instead of a set of
horizontally-spaced vertical density plots, I'd be fine with a set of
vertically-spaced horizontal density plots. But I think gnuplot
currently can't do that either: the density plots must be horizontal AND
they must start at y=0. Right? So being able to support translations
would be nice.

Re: Sideways violin plots in one plot cmd?

[Ethan A M. <me...@uw...>]

On Friday, 14 August 2020 10:34:52 PDT Juhász Péter wrote:
> On Wed, 2020-08-05 at 23:46 -0700, Dima Kogan wrote:
> > A question. I'm looking at the violin plot demo page:
> > 
> >   http://gnuplot.sourceforge.net/demo/violinplot.html
> > 
> > It shows how to make a horizontal density plot using 'smooth
> > kdensity'.
> > 
> > Then it shows to to make a vertical density plot by plotting
> > horizontally into a table, and then plotting the table, with "using"
> > directives used to swap the axes. This works clearly, but is there a
> > way
> > to do this (vertical density plots) with a single plot command? That
> > would make my life much easier.
> > 
> > Thanks!
> > 
> > 
> > 
> 
> Well, recently at $work I used gnuplot to produce a sideways histogram.
> Because it had to be sideways, I had to resort to manually assembling
> the plot with the boxxyerror style, with an awkward preprocessing step.
> 
> This, and your question, gives me the idea that it might be worth
> adding a general 'transpose' option to some of the "clever" plot styles
> that produce non-trivial vertical plots (histograms and boxplots,
> mostly).
> 
> best regards,
> Peter Juhasz

"How to make a sideways plot" does seem to be a very
frequently asked question.  There may be some clever way to select
an x/y axis transposition in the general case, but if so I have not been
able to find it.  

The program was designed from the start to have a single 2D plot mode
with x always horizontal, and a single 3D plot mode with z always
perpendicular to to horizontal axis. It would be possible to introduce
a third plot mode with x vertical and y horizontal, but that would have
to essentially duplicate all the existing code in plot2d.c and graphics.c,
which I think is far too much effort and code for only a little gain.

It is easier to relax the restrictions on the 3D plot mode. 
Over time the program has gained the options

- "set view map"   reproduces the x-horizontal y-vertical 2D layout.
  This allow use of 3D plot styles but does not address the current request.

- "set view azimuth" relaxes the requirement that the z axis of a 3D
  plot must be perpendicular to the horizontal.  This allows 3D view
  conventions such as altitude-azimuth that were not previously possible.

- "set view projection [xz|yz]".  These are new in version 5.4
 They do allow a "sideways" plot layout, with two drawbacks:
 1) it only supports the subset of styles in "splot" rather than "plot"
 2) the horizontal axis is z (not y), which is a bit confusing

I think there is room to build on this last idea in a couple of ways.

The 1st drawback could be addressed by extending more of the
2D plot styles to work with "splot".
Currently these styles work in projection:
	points lines vectors impulses 
	filledcurves/polygons (2D and 3D not quite the same)
	boxes 
	boxerrorbars (just added, conditional on #ifdef BOXERROR3D)

Smoothing is currently limited to csplines.
Adding kdensity smoothing to the 3D code would be easy, which 
might address Peter's current request.  But we will undoubtedly
continue to see requests for sideways histograms, boxplots, etc.
Those would not be so easy to extend to 3D.

The 2nd drawback may have a clever solution, although I do not
see immediately how it would work.  Would it be possible to hide
the x-is-now-y, z-is-now-x coordinate replacement at the input stage
so that the user doesn't have to worry about it?  How?

I have been thinking about adding a new demo to show that
"set view projection xz" can produce many "sideways" plot styles,
but haven't done much about it yet.   It needs to use some class
of data that legitimately benefits from the horizontal rather than
vertical layout.   Just turning an existing plot sideways is not
IMHO particularly useful.  Peter - would the plot you constructed
by hand be a good example to adapt?  If the combination
    set view projection xz
    splot ... with boxerror
doesn't handle it adequately, that would highlight where it would
be worth some additional work on the code. 

	Ethan

Re: Sideways violin plots in one plot cmd?

[Juhász P. <pet...@gm...>]

On Wed, 2020-08-05 at 23:46 -0700, Dima Kogan wrote:
> A question. I'm looking at the violin plot demo page:
> 
>   http://gnuplot.sourceforge.net/demo/violinplot.html
> 
> It shows how to make a horizontal density plot using 'smooth
> kdensity'.
> 
> Then it shows to to make a vertical density plot by plotting
> horizontally into a table, and then plotting the table, with "using"
> directives used to swap the axes. This works clearly, but is there a
> way
> to do this (vertical density plots) with a single plot command? That
> would make my life much easier.
> 
> Thanks!
> 
> 
> 

Well, recently at $work I used gnuplot to produce a sideways histogram.
Because it had to be sideways, I had to resort to manually assembling
the plot with the boxxyerror style, with an awkward preprocessing step.

This, and your question, gives me the idea that it might be worth
adding a general 'transpose' option to some of the "clever" plot styles
that produce non-trivial vertical plots (histograms and boxplots,
mostly).

best regards,
Peter Juhasz

Sideways violin plots in one plot cmd?

[Dima K. <gn...@di...>]

A question. I'm looking at the violin plot demo page:

  http://gnuplot.sourceforge.net/demo/violinplot.html

It shows how to make a horizontal density plot using 'smooth kdensity'.

Then it shows to to make a vertical density plot by plotting
horizontally into a table, and then plotting the table, with "using"
directives used to swap the axes. This works clearly, but is there a way
to do this (vertical density plots) with a single plot command? That
would make my life much easier.

Thanks!

Re: Minor correction to the demos

[Ethan A M. <me...@uw...>]

On Wednesday, 5 August 2020 15:02:22 PDT Dima Kogan wrote:
> Hi. I'm experimenting with violin plots, and just found a minor bug in
> the demos.
> 
> I'm looking at
> 
>   http://gnuplot.sourceforge.net/demo/violinplot.html
> 
> The "Same data - kernel density" section plot command has "with
> filledcurves above y". Shouldn't it be "above x"? I.e. we're using the x
> axis as one of the boundaries of the polygon we're filling in. The next
> section turns the plots 90%, so there "above y" is correct.

I agree.  Fixed now.

	Ethan

Minor correction to the demos

[Dima K. <gn...@di...>]

Hi. I'm experimenting with violin plots, and just found a minor bug in
the demos.

I'm looking at

  http://gnuplot.sourceforge.net/demo/violinplot.html

The "Same data - kernel density" section plot command has "with
filledcurves above y". Shouldn't it be "above x"? I.e. we're using the x
axis as one of the boundaries of the polygon we're filling in. The next
section turns the plots 90%, so there "above y" is correct.

dima

Re: weirdness and probable bug with set xdata/ydata time

[Ethan M. <eam...@gm...>]

On Saturday, 18 July 2020 15:51:04 PDT Juhász Péter wrote:
> Hi,
> 
> I've ran into an issue with time data that looks like a bug.
> 
> The script below works as expected:
> 
> ################
> $data << END
> 2020-07-01      1.1
> 2020-07-08      2.2
> 2020-07-15      3.3
> END
> 
> set xdata time
> set timefmt "%Y-%m-%d"
> #set ydata time
> plot $data u 1:2 w lp
> ###############
> 
> It produces a plot with correctly parsed and displayed dates on the x
> axis, and the x range is also sensible.
> 
> But uncomment the `set ydata time` line and it fails:
> 
> ################
>          "x.plt" line 10: warning: Skipping data file with no valid
> points
> 
> gnuplot> plot $data u 1:2 w lp
>                               ^
>          "x.plt" line 10: x range is invalid
> ##############
> 
> I can sort of understand that it fails because it can't parse the
> second column as date - but then why does it report the *x* range as
> invalid?

Since all of the points are invalid (because of y), it has no data left to
use for autoscaling x.  Therefore the range remains [VERYLARGE:-VERYLARGE],
which is an invalid range.

Perhaps easier to understand what's going on if you leave time data and
time formats out of it:

############
$NAN << EOD
NaN NaN
NaN NaN
EOD

plot $NAN using 1:2
         x range is invalid
show xrange
         set xrange [ * : * ] noreverse writeback  # (currently [8.98847e+307:-8.98847e+307] )
set xdata time
show xrange
         set xrange [ * : * ] noreverse writeback  # (currently ["         warning: time value out of range
32650/35/00,-468425088:00":"         warning: time value out of range
32650/35/00,-468425088:00"] )
############

> 
> Even more spooky is what it thinks about the x range after the failed
> plot:
> ##############
> gnuplot> sh xr
> 
> 	set xdata time
> 	set xrange [ * : * ] noreverse writeback  # (currently
> ["         warning: time value out of range
> 0000-35-32586":"         warning: time value out of range
> 0000-35-32586"] )
> #############
> 
> If I run gnuplot with valgrind, it barfs some "Conditional jump or move
> depends on uninitialised value(s)" errors, so some internal state is
> definitely corrupted.

Dunno. Valgrind doesn't complain here.

	cheers,

		Ethan


> 
> Best regards,
> Peter Juhasz
> 
> 
> 
> 
> 
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> 

weirdness and probable bug with set xdata/ydata time

[Juhász P. <pet...@gm...>]

Hi,

I've ran into an issue with time data that looks like a bug.

The script below works as expected:

################
$data << END
2020-07-01      1.1
2020-07-08      2.2
2020-07-15      3.3
END

set xdata time
set timefmt "%Y-%m-%d"
#set ydata time
plot $data u 1:2 w lp
###############

It produces a plot with correctly parsed and displayed dates on the x
axis, and the x range is also sensible.

But uncomment the `set ydata time` line and it fails:

################
         "x.plt" line 10: warning: Skipping data file with no valid
points

gnuplot> plot $data u 1:2 w lp
                              ^
         "x.plt" line 10: x range is invalid
##############

I can sort of understand that it fails because it can't parse the
second column as date - but then why does it report the *x* range as
invalid?

Even more spooky is what it thinks about the x range after the failed
plot:
##############
gnuplot> sh xr

	set xdata time
	set xrange [ * : * ] noreverse writeback  # (currently
["         warning: time value out of range
0000-35-32586":"         warning: time value out of range
0000-35-32586"] )
#############

If I run gnuplot with valgrind, it barfs some "Conditional jump or move
depends on uninitialised value(s)" errors, so some internal state is
definitely corrupted.

Best regards,
Peter Juhasz

New Release package - gnuplot version 5.4

[Ethan A M. <me...@uw...>]

Gnuplot version 5.4 - first release

I have placed a source tarball for 5.4 on SourceForge

   https://sf.net/projects/gnuplot/files/gnuplot/5.4.0/gnuplot-5.4.0.tar.gz

Release Notes with new feature list

   http://gnuplot.info/ReleaseNotes_5_4.html

Version 5.4.0 is the start of a new "major release" series.
It supersedes version 5.2, first released in September 2017.

I realize that there may be unresolved issues with building Windows binaries
for Windows 10, but I decided that it is not reasonable to hold up release any
longer for issues that appear to be more related to system configuration than to
the gnuplot source.  If necessary we can put out a Windows-specific patchlevel 1.
As always, I am totally reliant on and extremely grateful to the volunteers
who put together Windows binaries from a source release package.

        cheers, and happy plotting

                Ethan

Re: [PATCH v3] Support building development version without git

[Ethan A M. <me...@uw...>]

On Wednesday, 1 July 2020 17:31:03 PDT Henri Menke wrote:
> ---
> I've changed the commit format from %cs to %cd which is supported in
> older git versions as well.  I'd like to avoid using %ci because this
> requires an extra pipe to strip off the unnecessary time of day and
> timezone information which makes the code even more ugly and unreadable.

Looks good to me.

	Ethan

 
>  src/Makefile.am | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/src/Makefile.am b/src/Makefile.am
> index 1b0d8136e..0b261ea47 100644
> --- a/src/Makefile.am
> +++ b/src/Makefile.am
> @@ -107,12 +107,12 @@ endif
>  
>  DISTCLEANFILES = timestamp.h
>  BUILT_SOURCES = timestamp.h
> -git_current := $(shell cut -c6- $(top_srcdir)/.git/HEAD)
> -timestamp.h: $(top_srcdir)/.git/${git_current} Makefile
> +.PHONY: timestamp.h
> +timestamp.h: Makefile
>  	@echo Making $@
>  	@echo "#ifndef GNUPLOT_TIMEBASE_H_INCLUDED" >$@t
>  	@echo "#define GNUPLOT_TIMEBASE_H_INCLUDED" >>$@t
> -	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%ci | cut -b-11`\";" >>$@t
> +	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --date=short --format=%cd || stat -c '%.10y' Makefile.am`\";" >>$@t
>  	@echo "#endif /* GNUPLOT_TIMEBASE_H_INCLUDED */" >> $@t
>  	@if cmp -s $@ $@t; then rm -f $@t; else mv $@t $@; fi

Re: [PATCH] Support building development version without git

[Henri M. <hen...@gm...>]

On 01/07/20, 17:15, Dima Kogan wrote:
> Henri Menke <hen...@gm...> writes:
> 
> > The original motivation for this change was that I wanted to build the
> > development version on NixOS.  In the Nix packaging system you can build
> > stuff directly from a git checkout, but because the system emphasizes
> > reproducible builds, the build cannot depend on the contents of the .git
> > directory because these are actually non-deterministic (due to git gc),
> > so two checkouts of the same commit may actually have a different .git
> > directory.  This will lead to spurious build failures because the output
> > hashes differ.
> >
> >     https://github.com/NixOS/nixpkgs/issues/8567
> >     https://github.com/NixOS/nixpkgs/issues/20521
> >
> > I thought it would be a good idea to upstream the support for this but
> > if this is too much of a pain here, I can also patch it locally.
> 
> Thanks for explaining. What would you do if this project was hosted
> somewhere that doesn't make these snapshots available? Like github for
> instance?
> 

I'm not actually using the snapshot that is generated by SourceForge (on
GitHub you can also download snapshots, btw).  In my NixOS overlay I'm
cloning from the git repo but normally the .git directory is removed for
the aforementioned reasons.  Below you can see my Nix derivation to
build gnuplot from git which currently randomly fails due to git
repacking its .git directory which changes the output hash of the
derivation (that's how Nix verifies the integrity of packages).  Sorry,
this might all sound very obscure if you've never used Nix.

Kind regards,
Henri

    self: super:
    
    {
      gnuplot_qt = super.gnuplot_qt.overrideAttrs (oldAttrs: {
        version = "5.5_8685df8";
    
        src = self.fetchgit {
          url = "https://git.code.sf.net/p/gnuplot/gnuplot-main";
          rev = "8685df8e070211f99337bdf69162ab0ca6b4b1e6";
          sha256 = "17yv88xxfwgy9r07irbpwf30np8vvgkwy3nd3ngrrqw58ibjlrm6"; # <--- This changes if .git changes
          leaveDotGit = true; # <--- Fix timestamp.h generation
        };
    
        nativeBuildInputs = oldAttrs.nativeBuildInputs ++ [
          self.autoconf
          self.automake
          self.git
        ];
    
        postPatch = ''
          git checkout -b master # <--- Fix timestamp.h generation
          ./prepare
          ${oldAttrs.postPatch}
        '';
      });
    }

> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

[PATCH v3] Support building development version without git

[Henri M. <hen...@gm...>]

Currently building the development version requires git to be installed
to fetch the `last modified' date from the commit information.  This is
okay when building from a git clone, but fails when downloading a
snapshot that does not contain the .git directory.

To this end, I implemented a fallback that uses the last modification
time of the Makefile.am, which corresponds to the last commit date when
using a fresh checkout.  The old version had the advantage that
`timestamp.h' would only be rebuilt when changing the git branch.  If
git is not available, this logic fails of course.  Instead I made
`timestamp.h' a phony target, i.e. it will be rebuilt unconditionally
every time.  The disadvantage here is that this will also trigger a
rebuild of every file that depends on `timestamp.h' but as far as I can
see this is only `version.c'.
---
I've changed the commit format from %cs to %cd which is supported in
older git versions as well.  I'd like to avoid using %ci because this
requires an extra pipe to strip off the unnecessary time of day and
timezone information which makes the code even more ugly and unreadable.

 src/Makefile.am | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/Makefile.am b/src/Makefile.am
index 1b0d8136e..0b261ea47 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -107,12 +107,12 @@ endif
 
 DISTCLEANFILES = timestamp.h
 BUILT_SOURCES = timestamp.h
-git_current := $(shell cut -c6- $(top_srcdir)/.git/HEAD)
-timestamp.h: $(top_srcdir)/.git/${git_current} Makefile
+.PHONY: timestamp.h
+timestamp.h: Makefile
 	@echo Making $@
 	@echo "#ifndef GNUPLOT_TIMEBASE_H_INCLUDED" >$@t
 	@echo "#define GNUPLOT_TIMEBASE_H_INCLUDED" >>$@t
-	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%ci | cut -b-11`\";" >>$@t
+	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --date=short --format=%cd || stat -c '%.10y' Makefile.am`\";" >>$@t
 	@echo "#endif /* GNUPLOT_TIMEBASE_H_INCLUDED */" >> $@t
 	@if cmp -s $@ $@t; then rm -f $@t; else mv $@t $@; fi
 
-- 
2.27.0

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> In the end the fix was trivial, but I wouldn't call it "obvious".
>
> The clipping in proces_image() at step 5 correctly accounts for the pixel
> extent on x and y regardless of what flag is in pixel->type.  So nothing is
> gained by  setting it to OUTRANGE in step 2.  If instead we force all
> pixels to be flagged INRANGE at that stage, the incorrect zrange
> determination at step 3 doesn't happen and all pixels are kept.
> It wasn't obvious to me that marking all pixels INRANGE would preserve
> both autoscaling and clipping, but it does.
>
> fix commited to tip, queued for 5.4

I tested it out, and it appears to work. Thanks a lot! I would've had a
hard time not breaking stuff if I tried to fix this myself.

Re: [PATCH] Support building development version without git

[Dima K. <gn...@di...>]

Henri Menke <hen...@gm...> writes:

> The original motivation for this change was that I wanted to build the
> development version on NixOS.  In the Nix packaging system you can build
> stuff directly from a git checkout, but because the system emphasizes
> reproducible builds, the build cannot depend on the contents of the .git
> directory because these are actually non-deterministic (due to git gc),
> so two checkouts of the same commit may actually have a different .git
> directory.  This will lead to spurious build failures because the output
> hashes differ.
>
>     https://github.com/NixOS/nixpkgs/issues/8567
>     https://github.com/NixOS/nixpkgs/issues/20521
>
> I thought it would be a good idea to upstream the support for this but
> if this is too much of a pain here, I can also patch it locally.

Thanks for explaining. What would you do if this project was hosted
somewhere that doesn't make these snapshots available? Like github for
instance?

Re: [PATCH] Support building development version without git

[Ethan A M. <me...@uw...>]

On Wednesday, 1 July 2020 16:25:59 PDT Ethan A Merritt wrote:
 
> Henri:
> 
> Unfortunately your patch doesn't work for me. 
> 
> When I apply the patch and run
>   ./prepare
>   ./configure
>   make
> 
> I end up with an executable that reports it's version as 
> 
>         G N U P L O T
>         Version 5.5 patchlevel 0    last modified %cs
> 
> Was your change from "--format=%ci" to "--format=%cs" intentional,
> or was it just a typo? 
>   git --version
>   git version 2.21.3
>   git log -1 --format=%cs
>   %cs
>   git log -1 --format="%cs"
>   %cs
> 

Poking around online git docs, I find that %cs was introduced in 
version 2.25 (Jan 2020).  Too new for most distros!
I think we should stick to %ci for now.

	Ethan

Re: [PATCH] Support building development version without git

[Ethan A M. <me...@uw...>]

On Wednesday, 1 July 2020 15:35:03 PDT Henri Menke wrote:
> On 01/07/20, 03:30, Dima Kogan wrote:
> > Henri Menke <hen...@gm...> writes:
> > 
> > > If I go to the SourceForge project page [1] and just hit the `Download
> > > Snapshot' button in the upper right corner, I get a zip file with all
> > > the source but no .git folder and also no timestamp.h. I suspect there
> > > is no or at least no easy way to make SourceForge preprocess the
> > > source when generating a snapshot.
> > >
> > > Within the Makefile determining the latest change would be tricky. The
> > > only way I can think of right now would be to run stat on all the
> > > files in the source tree and pick the newest one, but that is slow and
> > > ugly. Maybe instead of using the current date if git is unavailable
> > > just do echo '0000-00-00'. It should be pretty obvious that this is
> > > not the true `Last modified' then.
> > 
> > Oh. Wow. OK. My feeling is that people should be either
> > 
> > - using release tarballs
> > - using git
> > 
> > It's unfortunate that sourceforge makes another option available without
> > asking, but I don't think it's worth the effort to support it, and a
> > dummy string is just fine. Is a date string of "DATE UNKNOWN; PLEASE USE
> > A RELEASE TARBALL OR A GIT CHECKOUT" valid for the purposes of
> > timestamp.h? Ethan: do you want to support this use case in a better
> > way?
> 
> The original motivation for this change was that I wanted to build the
> development version on NixOS.  In the Nix packaging system you can build
> stuff directly from a git checkout, but because the system emphasizes
> reproducible builds, the build cannot depend on the contents of the .git
> directory because these are actually non-deterministic (due to git gc),
> so two checkouts of the same commit may actually have a different .git
> directory.  This will lead to spurious build failures because the output
> hashes differ.
> 
>     https://github.com/NixOS/nixpkgs/issues/8567
>     https://github.com/NixOS/nixpkgs/issues/20521
> 
> I thought it would be a good idea to upstream the support for this but
> if this is too much of a pain here, I can also patch it locally.
> 
> Kind regards,
> Henri

Let me add a 3rd reason why it would be a good idea to re-think the
current timestamp dependency:   
If you use "git bisect"  for debugging, it is made more cumbersome by the
fact the builds fail because the timestamp dependency in the Makefile is
not satisfied by the content of the git repository as seen during the
bisection.   

Henri:

Unfortunately your patch doesn't work for me. 

When I apply the patch and run
  ./prepare
  ./configure
  make

I end up with an executable that reports it's version as 

        G N U P L O T
        Version 5.5 patchlevel 0    last modified %cs

Was your change from "--format=%ci" to "--format=%cs" intentional,
or was it just a typo? 
  git --version
  git version 2.21.3
  git log -1 --format=%cs
  %cs
  git log -1 --format="%cs"
  %cs


If I change it back to %ci I get 

        G N U P L O T
        Version 5.5 patchlevel 0    last modified 2020-06-30 20:23:26 -0700

which is correct but obviously longer than just the date.

	Ethan