gnuplot-beta — Developers and beta testers of new gnuplot code

Re: [PATCH] cairolatex: respect transparency setting

[Henri M. <hen...@gm...>]

Dear Ehtan,

I just wanted to ask what is the status of this patch?  It doesn't
change the default behaviour but makes the notransparent option actually
do something.  If you want I can write sentence in the documentation
about the layer ordering.

Cheers, Henri

On 21/08/20, 13:34, Henri Menke wrote:
> Currently the cairolatex terminal does not respect transparency options
> given in the terminal setting.  In this minimal example the
> notransparent option is ignored:
> 
>     set terminal cairolatex png notransparent
>     set output "test.tex"
>     plot sin(x)
> 
> Running this through gnuplot and then using `file' reveals that the
> alpha channel is still present:
> 
>     $ gnuplot test.gnuplot && file test.png
>     test.png: PNG image data, 1500 x 900, 8-bit/color RGBA, non-interlace
> 
> With this patch the transparency setting will be honoured.  The current
> default setting of implicitly assuming transparency is unaltered.
> ---
>  term/cairo.trm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/term/cairo.trm b/term/cairo.trm
> index ee748dfe2..3ef76ecc6 100644
> --- a/term/cairo.trm
> +++ b/term/cairo.trm
> @@ -849,7 +849,7 @@ void cairotrm_graphics()
>  	plot.background.g = cairo_params->background.g;
>  	plot.background.b = cairo_params->background.b;
>  	gp_cairo_set_background(cairo_params->background);
> -	if (ISCAIROLATEX || cairo_params->transparent)
> +	if (cairo_params->transparent)
>  		gp_cairo_clear_background(&plot);
>  	else
>  		gp_cairo_solid_background(&plot);
> -- 
> 2.28.0
> 

5.4.0: Mac OS X version available

[Erik L. <eri...@gm...>]

Dear Ethan et al.,

A Mac OS X package for version 5.4.0 is now available at
https://csml-wiki.northwestern.edu/index.php/Binary_versions_of_Gnuplot_for_OS_X

If you encounter any problem, please let me know.

There is one small compilation issue that *seems* to be due to the way that
src/Makefile is constructed. On the more recent versions of OS X, the X11
libraries and header files are in /opt/X11, which ./configure does not
detect. This is fine, since I can use the options '--x-include=/opt/X11/include
--x-libraries=/opt/X11/lib' to specify this. However, the X11 library
location does not get used in the compilation of gnuplot (only for
gnuplot_x11, via the XLIBS variable), even though the main executable needs
the Xpm library. Manually adding -L/opt/X11/lib to LDFLAGS in src/Makefile
fixes this, but should not be necessary.

Regards,

Erik


On Thu, Jul 16, 2020 at 2:28 PM Ethan A Merritt <me...@uw...> wrote:

> Gnuplot version 5.4 - first release
>
> I have placed a source tarball for 5.4 on SourceForge
>
>
> https://sf.net/projects/gnuplot/files/gnuplot/5.4.0/gnuplot-5.4.0.tar.gz
>
> Release Notes with new feature list
>
>    http://gnuplot.info/ReleaseNotes_5_4.html
>
> Version 5.4.0 is the start of a new "major release" series.
> It supersedes version 5.2, first released in September 2017.
>
> I realize that there may be unresolved issues with building Windows
> binaries
> for Windows 10, but I decided that it is not reasonable to hold up release
> any
> longer for issues that appear to be more related to system configuration
> than to
> the gnuplot source.  If necessary we can put out a Windows-specific
> patchlevel 1.
> As always, I am totally reliant on and extremely grateful to the volunteers
> who put together Windows binaries from a source release package.
>
>         cheers, and happy plotting
>
>                 Ethan
>
>
>
>
>
>
>
>
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via:
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>

Re: New Release package - gnuplot version 5.4

[Karl-Friedrich R. <mai...@gm...>]

Many thanks!

I noticed the new feature "Help/Edit (w)gnuplot.ini" in the menu.
Next to the editor (works fine), it additionally opens a black text
window titled "gnuplot 5.4 patchlevel rc1" (?!?) with a blinking
cursor in it, but anything typed there lands in the regular gnuplot
shell window. Closing that window kills wgnuplot.exe.

   Karl


Am 19.09.2020 um 18:00 schrieb Bastian Märkisch:
> Dear all,
>
> You can now find Windows binaries of the 5.4.0 release for testing on SourceForge:
>
>   https://sf.net/projects/gnuplot/files/gnuplot/testing/
>
> Please give them a try.
>
>   Bastian

Aw: New Release package - gnuplot version 5.4

[Bastian M. <bma...@we...>]

Dear all,

You can now find Windows binaries of the 5.4.0 release for testing on SourceForge:

  https://sf.net/projects/gnuplot/files/gnuplot/testing/

Please give them a try.

  Bastian


> Gesendet: Donnerstag, 16. Juli 2020 um 21:25 Uhr
> Von: "Ethan A Merritt" <me...@uw...>
> An: gnu...@li...
> Betreff: New Release package - gnuplot version 5.4
>
> Gnuplot version 5.4 - first release
>
> I have placed a source tarball for 5.4 on SourceForge
>
>    https://sf.net/projects/gnuplot/files/gnuplot/5.4.0/gnuplot-5.4.0.tar.gz
>
> Release Notes with new feature list
>
>    http://gnuplot.info/ReleaseNotes_5_4.html
>
> Version 5.4.0 is the start of a new "major release" series.
> It supersedes version 5.2, first released in September 2017.
>
> I realize that there may be unresolved issues with building Windows binaries
> for Windows 10, but I decided that it is not reasonable to hold up release any
> longer for issues that appear to be more related to system configuration than to
> the gnuplot source.  If necessary we can put out a Windows-specific patchlevel 1.
> As always, I am totally reliant on and extremely grateful to the volunteers
> who put together Windows binaries from a source release package.
>
>         cheers, and happy plotting
>
>                 Ethan
>
>
>
>
>
>
>
>
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>

Re: Strange autoscaling with rgbimage and inverted yranges

[Ethan A M. <me...@uw...>]

On Sunday, 13 September 2020 13:17:59 PDT Dima Kogan wrote:
> Hi. I do observe your sequence work at you state:
> 
> > 1) start gnuplot
> > 2) load script
> > 3) zoom
> > 4) 'u'
> >    plot is now incorrectly scaled
> > 5) 'n'
> >    shows zoom state from (3)
> > 6) 'p'
> >    correctly unzooms to original plot
> 
> Step 6 may be 'u' or 'p'. The reason things don't look the same after
> steps 4 and 6 is the value of
> 
>   { axis_array[FIRST_Y_AXIS].min, axis_array[FIRST_Y_AXIS].max }
> 
> at the start of the refresh_bounds() call. This function assumes
> min<max. And if we're reversing some axis, this is done at the very end
> of refresh_bounds() with the axis_check_range() calls.
> 
> In step 3, refresh_bounds() sees {min,max} = {4000,-500}
> In step 6, refresh_bounds() sees {min,max} = {1850,1400}
> 
> This is dependent on the actual zoom bounds. Both of these have the
> unexpected min>max, obviously, but in step 6 we get lucky, and it ends
> up doing the right thing anyway, in this case.

OK.
But no autoscaling at all should be necessary during zoom or unzoom.
That is true even for nonvolatile data, but is doubly true for volatile data.
The whole point of zoom is that the bounds are being chosen
interactively by the mouse, not by the content of the plot.

My thought is to add a flag somewhere that zoom/unzoom is in progress,
and skip all autoscaling operations if it is set.  In fact there already is a flag
"inside_zoom" but it isn't used for much.  I have not yet inventoried all the
places in the code that would need to be changed.  

	Ethan

Re: Strange autoscaling with rgbimage and inverted yranges

[Dima K. <gn...@di...>]

Hi. I do observe your sequence work at you state:

> 1) start gnuplot
> 2) load script
> 3) zoom
> 4) 'u'
>    plot is now incorrectly scaled
> 5) 'n'
>    shows zoom state from (3)
> 6) 'p'
>    correctly unzooms to original plot

Step 6 may be 'u' or 'p'. The reason things don't look the same after
steps 4 and 6 is the value of

  { axis_array[FIRST_Y_AXIS].min, axis_array[FIRST_Y_AXIS].max }

at the start of the refresh_bounds() call. This function assumes
min<max. And if we're reversing some axis, this is done at the very end
of refresh_bounds() with the axis_check_range() calls.

In step 3, refresh_bounds() sees {min,max} = {4000,-500}
In step 6, refresh_bounds() sees {min,max} = {1850,1400}

This is dependent on the actual zoom bounds. Both of these have the
unexpected min>max, obviously, but in step 6 we get lucky, and it ends
up doing the right thing anyway, in this case.

Specifically the diverging behavior comes from

  refresh_bounds()          calls
  process_image()           calls
  STORE_AND_UPDATE_RANGE()  calls
  store_and_update_range()

The logic in store_and_update_range() is effectively:

    if ( curval > axis->max && curval >= axis->min )
      axis->max = curval;

Which is correct only if axis->min < axis->max

I'm not entirely sure what the values of axis->min, axis->max should be
at the start of refresh_bounds(), but reordering them makes this thing
work correctly. Patch (usable for this specific test case only):

  diff --git a/src/plot2d.c b/src/plot2d.c
  index b2877eb89..7e4e57259 100644
  --- a/src/plot2d.c
  +++ b/src/plot2d.c
  @@ -299,6 +299,17 @@ refresh_bounds(struct curve_points *first_plot, int nplots)
       struct curve_points *this_plot = first_plot;
       int iplot;		/* plot index */

  +
  +    if(axis_array[FIRST_Y_AXIS].min > axis_array[FIRST_Y_AXIS].max)
  +    {
  +        double t = axis_array[FIRST_Y_AXIS].min;
  +        axis_array[FIRST_Y_AXIS].min = axis_array[FIRST_Y_AXIS].max;
  +        axis_array[FIRST_Y_AXIS].max = t;
  +    }
  +
  +
  +
  +
       for (iplot = 0;  iplot < nplots; iplot++, this_plot = this_plot->next) {
          int i;		/* point index */
          struct axis *x_axis = &axis_array[this_plot->x_axis];

Applying this patch makes the test case work, even before any of your
patches from yesterday. Clearly, this should be more general, but I'm
hazy on the details, so it'd be better if you finish this, probably.

Thanks!

Re: Strange autoscaling with rgbimage and inverted yranges

[Ethan A M. <me...@uw...>]

On Saturday, 12 September 2020 21:07:58 PDT Dima Kogan wrote:
> Dima Kogan <gn...@di...> writes:
> 
> > Ethan A Merritt <me...@uw...> writes:
> >
> >> And even after tracking it that far I still don't understand why
> >> 'p' works and 'u' doesn't.     If you can figure that out I'd be 
> >> happier with declaring it fixed.
> >
> > In my tests just now 'p' and 'u' both show this issue. Are you 100% sure
> > that 'p' does NOT show the issue for you?
> 
> I just tried this with the x11, qt and wxt terminals, and in all 3 cases
> both 'u' and 'p' show this issue. I'm at the git HEAD as off a few days
> ago:
> 
>   commit 160d7511bcbdfd0d1943a624af5fc42504a9f84c
>   Author: Ethan A Merritt <merritt@u.washington.edu>
>   Date:   Thu Sep 3 20:20:23 2020 -0700
> 
>       new function (long)argb = rgbcolor("name")
> 
> The test is:
> 
> 1. start gnuplot
> 2. "load" the script in the first email in this thread
> 3. interactively zoom
> 4. 'u' or 'p'
> 
> I'm on GNU/Linux. If I can reproduce the behavior you're seeing ('p' and
> 'u' work differently), then I'll be happy to debug it.

Try this:

1) start gnuplot
2) load script
3) zoom
4) 'u'
   plot is now incorrectly scaled
5) 'n'
   shows zoom state from (3)
6) 'p'
   correctly unzooms to original plot

Now that I try more sequences I notice that replacing 'p' with 'u'
in step (6) also works correctly.  So maybe the issue is whether
the previous operation was 'n'?  Or there's some parity issue
with the count of operations? 

Regardless, I think autoscaling in the current HEAD now works
correctly for your test case.
I also think that it is incorrect for the unzoom operation to invoke
autoscaling at all, but that would require additional code changes.

	Ethan

Re: Strange autoscaling with rgbimage and inverted yranges

[Dima K. <gn...@di...>]

Dima Kogan <gn...@di...> writes:

> Ethan A Merritt <me...@uw...> writes:
>
>> And even after tracking it that far I still don't understand why
>> 'p' works and 'u' doesn't.     If you can figure that out I'd be 
>> happier with declaring it fixed.
>
> In my tests just now 'p' and 'u' both show this issue. Are you 100% sure
> that 'p' does NOT show the issue for you?

I just tried this with the x11, qt and wxt terminals, and in all 3 cases
both 'u' and 'p' show this issue. I'm at the git HEAD as off a few days
ago:

  commit 160d7511bcbdfd0d1943a624af5fc42504a9f84c
  Author: Ethan A Merritt <merritt@u.washington.edu>
  Date:   Thu Sep 3 20:20:23 2020 -0700

      new function (long)argb = rgbcolor("name")

The test is:

1. start gnuplot
2. "load" the script in the first email in this thread
3. interactively zoom
4. 'u' or 'p'

I'm on GNU/Linux. If I can reproduce the behavior you're seeing ('p' and
'u' work differently), then I'll be happy to debug it.

Re: Strange autoscaling with rgbimage and inverted yranges

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> And even after tracking it that far I still don't understand why
> 'p' works and 'u' doesn't.     If you can figure that out I'd be 
> happier with declaring it fixed.

In my tests just now 'p' and 'u' both show this issue. Are you 100% sure
that 'p' does NOT show the issue for you?

Re: Strange autoscaling with rgbimage and inverted yranges

[Ethan A M. <me...@uw...>]

On Friday, 11 September 2020 22:51:03 PDT Ethan A Merritt wrote:

> I _think_ that the problem involves this macro in axis.h
> 
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> /* Simplest form of autoscaling (no check on autoscale constraints).
>  * Used by refresh_bounds() and refresh_3dbounds().
>  * Used also by autoscale_boxplot.
>  */
> #define autoscale_one_point(axis, x) do {\
>     if (axis->set_autoscale & AUTOSCALE_MIN && x < axis->min) \
>         axis->min = x; \
>     if (axis->set_autoscale & AUTOSCALE_MAX && x > axis->max) \
>         axis->max = x; \
>     } while (0);
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> 
> as called from plot2d.c:refresh_bounds() when zooming volatile data.
> 
> Note that the macro does not consider the case of reversed axes.
> Adding a test for (axis->min < axis->max) seems to make your
> test case work.   See attached patch.

Better patch (but still not fully correct)

	Ethan

Re: Strange autoscaling with rgbimage and inverted yranges

[Ethan A M. <me...@uw...>]

On Friday, 11 September 2020 20:13:33 PDT Dima Kogan wrote:

> In any case, this worked in 5.2. I just did a bisection, and I can now
> see why this seemed familiar. The commit that broke it:
> 
>   https://sourceforge.net/p/gnuplot/gnuplot-main/ci/36a2407e50/
> 
> The person who broke it was me! But despite what git says, I have very
> little recollection of any of this. Ethan: do you have a good sense of
> what's happening here, and can you easily fix it now? If not, I can poke
> at it, and figure it out.
> 
> dima

I am puzzled because I do not understand the difference between
returning to the top level zoom state by pressing 'p' to step backwards
through the stack of zoom operations and pressing 'u' to jump back to the
top of the stack.  Either way it should be applying the same state
(confirmed in the debugger) but for some reason that I don't understand
the former works correcly and the latter doesn't.

I _think_ that the problem involves this macro in axis.h

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
/* Simplest form of autoscaling (no check on autoscale constraints).
 * Used by refresh_bounds() and refresh_3dbounds().
 * Used also by autoscale_boxplot.
 */
#define autoscale_one_point(axis, x) do {\
    if (axis->set_autoscale & AUTOSCALE_MIN && x < axis->min) \
        axis->min = x; \
    if (axis->set_autoscale & AUTOSCALE_MAX && x > axis->max) \
        axis->max = x; \
    } while (0);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

as called from plot2d.c:refresh_bounds() when zooming volatile data.

Note that the macro does not consider the case of reversed axes.
Adding a test for (axis->min < axis->max) seems to make your
test case work.   See attached patch.

The unpatched code does the wrong thing for reversed axes.
The patched code does nothing at all for reversed axes,
is it probably wrong also but for some case other than yours.
A more complete fix for the macro is wanted.

And even after tracking it that far I still don't understand why
'p' works and 'u' doesn't.     If you can figure that out I'd be 
happier with declaring it fixed.

                 Ethan
-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
MS 357742,   University of Washington, Seattle 98195-7742

Re: Strange autoscaling with rgbimage and inverted yranges

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> I think the issue is that because your example uses in-line data it
> cannot be re-read. The program tries to work around this by reusing
> the contents of the various internal data structures but in this case
> the workaround fails. Possibly fixable, but I think the primary lesson
> is that data blocks are much preferred to in-line data if you are
> going to be manipulating/re-drawing the plot.

Hi.

100% of my fairly heavy usage of gnuplot uses inline data. The
particular case in this report is fairly common too: when doing any sort
of computer vision that annotates an image, you want to display the
source image with a flipped y axis, since the origin pixel is at the
top-left.

In any case, this worked in 5.2. I just did a bisection, and I can now
see why this seemed familiar. The commit that broke it:

  https://sourceforge.net/p/gnuplot/gnuplot-main/ci/36a2407e50/

The person who broke it was me! But despite what git says, I have very
little recollection of any of this. Ethan: do you have a good sense of
what's happening here, and can you easily fix it now? If not, I can poke
at it, and figure it out.

dima

Re: [bug] plot bins does not clip output to plot area

[Achim G. <Str...@ne...>]

Ethan A Merritt writes:
>> I'm using it as a way to average over a large number of samples that are
>> sampled equidistantly (kernel density is way slower obviously). The
>> documentation doesn't really tell how it is implemented or supposed to
>> work, so I've experimented a bit to find out.
>
> I take the blame for any failures in either the implementation or
> the documentation of the "bins" option, since I wrote both.
> I would be happy to amend the code or the description to cover
> uses that I didn't anticipate.

No blame to go around… any feature is prone to be (mis)used in ways
never intended, once it's there.  :-)

> Since I envisioned it as a histogramming tool, it did not occur to
> me that users would want to plot "with lines" rather than using
> boxes or impulses.   Therefore I did not think about or test the
> clipping behaviour.

No good deed goes unpunished, as they say.

> As it happens, the routines that draw boxes or impulses do the clipping as
> they draw each one.  The routine that draws "with lines" assumes that the
> inrange/outrange/undefined status of each point has already been flagged,
> and only considers clipping if a particular line segment changes state.
> This is so that successive in range points are drawn as a single poly-line
> with smooth joins rather than being split into individual segments.
> Normally the inrange/outrange flag is set on data entry, but those flags
> apply to the original data points rather than to the binned totals.
> I have now added a separate pass to re-check the binned data against
> yrange so that clipping works as you expect it to.
> (New code added for both 5.4 and 5.5).

Great, I'll check it out later.

> I remain a little uneasy about the use of negative values in the weighting
> column, although I don't have a specific example in mind that will fail.

So far I have not yet encountered any problems with that and I have
likely given this part quite a bit of exercise as most of the data I'm
plotting is roughly centered about zero.

> Thanks for the explanation of what you are using it for.
> I agree that in that mode "bins" is similar to a kernel density model
> that uses a delta function rather than a Gaussian kernel.

I would suggest that it would be welcome if one could specify the kernel
function and/or provide a way to implement general FIR filtering on the
original data.  While the moving average trick is cute, it is also quite
slow and cumbersome once you need to average over a large number of
samples.

> I may not have phrased that well.  It weights the _contribution_ of each
> sample by the value in the seconds column.  If no second column is
> provided, the weight is 1.

That seems a more accessible description to me, thanks.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [bug] plot bins does not clip output to plot area

[Ethan A M. <me...@uw...>]

> I'm using it as a way to average over a large number of samples that are
> sampled equidistantly (kernel density is way slower obviously). The
> documentation doesn't really tell how it is implemented or supposed to
> work, so I've experimented a bit to find out.

I take the blame for any failures in either the implementation or
the documentation of the "bins" option, since I wrote both.
I would be happy to amend the code or the description to cover
uses that I didn't anticipate.

Since I envisioned it as a histogramming tool, it did not occur to
me that users would want to plot "with lines" rather than using
boxes or impulses.   Therefore I did not think about or test the
clipping behaviour.

As it happens, the routines that draw boxes or impulses do the clipping as
they draw each one.  The routine that draws "with lines" assumes that the
inrange/outrange/undefined status of each point has already been flagged,
and only considers clipping if a particular line segment changes state.
This is so that successive in range points are drawn as a single poly-line
with smooth joins rather than being split into individual segments.
Normally the inrange/outrange flag is set on data entry, but those flags
apply to the original data points rather than to the binned totals.
I have now added a separate pass to re-check the binned data against
yrange so that clipping works as you expect it to.
(New code added for both 5.4 and 5.5).

I remain a little uneasy about the use of negative values in the weighting
column, although I don't have a specific example in mind that will fail.

Thanks for the explanation of what you are using it for.
I agree that in that mode "bins" is similar to a kernel density model
that uses a delta function rather than a Gaussian kernel.

	cheers,

		Ethan

On Friday, 11 September 2020 11:59:28 PDT ASSI wrote:
> Ethan A Merritt writes:
> 
> > If there is a second column of data this is interpreted as a weight.
> 
> Let's assume the sample spacing is 1 and no samples are missing, then
> you'll get the sum over the bin width.  Different sampling density just
> scales the result.  Dividing by the number of samples gives you the
> average.

Correct.

[snip]

> 
> > Your test script provides no "using" specifier, however, so the plot
> > command draws values from column 1 (essentially the numbers 0 to 100)
> > and weights each one by the value in the second column (sin(x)).
> 
> It doesn't do that either or the result would be a linearly rising
> function with a sin riding on top of it.  It reproduces the function if
> I arrange the binwidth to contain just a single sample, so clearly the x
> column isn't used directly.

I may not have phrased that well.  It weights the _contribution_ of each
sample by the value in the seconds column.  If no second column is
provided, the weight is 1.

Re: [bug] plot bins does not clip output to plot area

[ASSI <Str...@ne...>]

Ethan A Merritt writes:
> I am confused as to what exactly you are trying to plot.
> The intent of the "bins" option is similar to the "smooth frequency" option.
> It derives an approximation to the distribution of values read from
> a single column of data.

I'm using it as a way to average over a large number of samples that are
sampled equidistantly (kernel density is way slower obviously).  The
documentation doesn't really tell how it is implemented or supposed to
work, so I've experimented a bit to find out.

> If there is a second column of data this is interpreted as a weight.

Let's assume the sample spacing is 1 and no samples are missing, then
you'll get the sum over the bin width.  Different sampling density just
scales the result.  Dividing by the number of samples gives you the
average.

> To show the distribution of values for f(x) over a given range of x, 
> the plotting command could be
>
>       plot sample [xmin:xmax:increment] '+' using (f(x)) bins {binwidth=foo}
>
> If I rewrite your test case in this form, shown as a histogram, it becomes
>
>       plot sample [0:100:1] '+' using (sin(x)) bins binwidth=0.1 with boxes
>
> The autoscaling works correctly on both x and y, and remains correct if 
> I plot instead "with lines". Note that this is a single column of data, so
> the samples have uniform weights.

We've already established that "with boxes" and "with impulses" would
clip the plotted data correctly, but both "with points" and "with lines"
doesn't.  That's the only thing I think is a bug and needs fixing.

> Your test case creates this same set of samples but stores it in a 
> data block with two columns   x and f(x).   To reproduce the plot from
> the stored samples the command would be
>
>      plot $data using 2 bins binwidth=0.1

That's different from what I'm trying to get, though; and it doesn't
reproduce the data either.  It does produce a distribution of the
samples, resolved into 0.1 wide buckets.

> Your test script provides no "using" specifier, however, so the plot
> command draws values from column 1 (essentially the numbers 0 to 100)
> and weights each one by the value in the second column (sin(x)).

It doesn't do that either or the result would be a linearly rising
function with a sin riding on top of it.  It reproduces the function if
I arrange the binwidth to contain just a single sample, so clearly the x
column isn't used directly.  It integrates over the bin if I arrange the
bin to contain a larger number of samples (10 in my example).  So if I
divide the result by the number of samples I get an average over the
bin.

> The presence of negative weights in particular confuses the program.
> I don't rule out the possibility that there is a legitimate use for such
> but the code assumes positive weights, typically either 1 or a set of
> fractional values summing to 1 so that the distribution is normalized.
> Also since the range of values is [0:100], using a bin width of 0.1 means
> that the vast majority of bins are empty.
> This produces the strange-looking plot you saw.

The plot is the result of most of the bins having an integral either
far above 0.1 or below -0.1, so it should have been clipped at the axis
borders, but it wasn't.

> If your test case is intended as a simple stand-in for some real world
> data that does in fact use negative weights, please clarify the
> expected properties of the resulting distribution and I will try to
> modify the program's expectations to allow it.  As it stands, the 
> programs expectations about y values lead it to totally mis-assign
> all the points as "inrange" and therefore not clipped.

Again, I'm not using it this way and the actual code doesn't seem to
work the way you described it either.  Now that you've described how to
get a binned distribution/histogram I do have uses for that too, but that's
not what I was having problems with.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

Re: [bug] plot bins does not clip output to plot area

[Ethan A M. <me...@uw...>]

I am confused as to what exactly you are trying to plot.
The intent of the "bins" option is similar to the "smooth frequency" option.
It derives an approximation to the distribution of values read from
a single column of data.
If there is a second column of data this is interpreted as a weight.

To show the distribution of values for f(x) over a given range of x,
the plotting command could be

      plot sample [xmin:xmax:increment] '+' using (f(x)) bins {binwidth=foo}

If I rewrite your test case in this form, shown as a histogram, it becomes

      plot sample [0:100:1] '+' using (sin(x)) bins binwidth=0.1 with boxes

The autoscaling works correctly on both x and y, and remains correct if
I plot instead "with lines". Note that this is a single column of data, so
the samples have uniform weights.

Your test case creates this same set of samples but stores it in a
data block with two columns   x and f(x).   To reproduce the plot from
the stored samples the command would be

     plot $data using 2 bins binwidth=0.1

Your test script provides no "using" specifier, however, so the plot
command draws values from column 1 (essentially the numbers 0 to 100)
and weights each one by the value in the second column (sin(x)).
The presence of negative weights in particular confuses the program.
I don't rule out the possibility that there is a legitimate use for such
but the code assumes positive weights, typically either 1 or a set of
fractional values summing to 1 so that the distribution is normalized.
Also since the range of values is [0:100], using a bin width of 0.1 means
that the vast majority of bins are empty.
This produces the strange-looking plot you saw.

If your test case is intended as a simple stand-in for some real world
data that does in fact use negative weights, please clarify the
expected properties of the resulting distribution and I will try to
modify the program's expectations to allow it.  As it stands, the
programs expectations about y values lead it to totally mis-assign
all the points as "inrange" and therefore not clipped.

       Ethan

Re: [bug] plot bins does not clip output to plot area

[Achim G. <Str...@ne...>]

Ethan A Merritt writes:
> The y clipping seems to fail specifically when plotting "with lines".
> It clips properly for "with impulses" or "with boxes", right?

I haven't tried these two specifically, but "with points" also plots
outside the boundary.  OK, impulses and boxes are correctly clipped.
Things aer more clearly visible if you additionally

 set tmargin at screen 0.8
 set bmargin at screen 0.2

in my reproducer before the plot command.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [bug] plot bins does not clip output to plot area

[Ethan A M. <me...@uw...>]

On Wednesday, 9 September 2020 22:36:56 PDT ASSI wrote:
> Ethan A Merritt writes:
> > The program extends auto-scaled ranges to the next axis tic unless
> > you tell it not to.  If you do not want the range to be extended, tell it
> 
> The y axis is not autoscaled in my example and there is no clipping at
> all, which gets obvious if you do multiplots.

Ah, sorry.  I misunderstood.

The y clipping seems to fail specifically when plotting "with lines".
It clips properly for "with impulses" or "with boxes", right?

	Ethan

> 
> 
> Regards,
> Achim.
> 

Re: [bug] plot bins does not clip output to plot area

[ASSI <Str...@ne...>]

Ethan A Merritt writes:
> The program extends auto-scaled ranges to the next axis tic unless
> you tell it not to.  If you do not want the range to be extended, tell it

The y axis is not autoscaled in my example and there is no clipping at
all, which gets obvious if you do multiplots.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [bug] plot bins does not clip output to plot area

[Ethan A M. <me...@uw...>]

On Wednesday, 9 September 2020 11:05:16 PDT Achim Gratz wrote:
> 
> To reproduce:
> 
> gnuplot> set table $data
> gnuplot> plot [0:100] sin(x)
> gnuplot> unset table
> gnuplot> plot [*:*][-0.1:0.1] $data w lines bins binwidth=0.1
> 
> 
> Regards,
> Achim.

The program extends auto-scaled ranges to the next axis tic unless
you tell it not to.  If you do not want the range to be extended, tell it

    set xrange noextend       #version 5.4
or
    set xrange [*:*] noextend #version 5.2

	cheers,

		Ethan
 

[bug] plot bins does not clip output to plot area

[Achim G. <Str...@ne...>]

To reproduce:

gnuplot> set table $data
gnuplot> plot [0:100] sin(x)
gnuplot> unset table
gnuplot> plot [*:*][-0.1:0.1] $data w lines bins binwidth=0.1


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Strange autoscaling with rgbimage and inverted yranges

[Ethan A M. <me...@uw...>]

On Wednesday, 9 September 2020 02:07:50 PDT Dima Kogan wrote:
> Hi. I vaguely recall mentioning this already, but I don't see it in my
> email. Apologies if this has already been reported. I have this script:
> 
>     set yrange [:] reverse
>     set size ratio -1
>     plot "/tmp/gray.jpg" binary filetype=auto flipy with rgbimage,'-'   using 1:2 notitle with points pt 3 ps 3 
[snip]
>     e
> 
> I.e. we plot some points on top of the attached image. The contents of
> the image don't matter. Here we just have a gray field. I'm using the
> latest master branch, but I belive 5.4 does this too.
> 
> 1. Running that script in an interactive terminal (I tried x11 and qt),
>    the plot pops up just fine: the autoscale considered the image AND the
>    points
> 
> 2. We interactively zoom on any area with the mouse. This works too
> 
> 3. We press 'u' to zoom out. The expectation is that we get back to the
>    full view after step 1. Instead we get to a new view, zoomed-in on
>    the points only, NOT on the image.

Hmm.  
But if you press 'p' for "previous" the unzoom works correctly.
That is probably a clue.

Also, if the inline data is replaced by a datablock the problem goes away.

I think the issue is that because your example uses in-line data it
cannot be re-read.  The program tries to work around this by reusing
the contents of the various internal data structures but in this case the
workaround fails.  Possibly fixable, but I think the primary lesson is
that data blocks are much preferred to in-line data if you are going to be
manipulating/re-drawing the plot.

	Ethan

> Something about the reverse yrange is confusing I think. Have I reported
> this already?
> 
> Thanks.

Strange autoscaling with rgbimage and inverted yranges

[Dima K. <gn...@di...>]

Hi. I vaguely recall mentioning this already, but I don't see it in my
email. Apologies if this has already been reported. I have this script:

    set yrange [:] reverse
    set size ratio -1
    plot "/tmp/gray.jpg" binary filetype=auto flipy with rgbimage,'-'   using 1:2 notitle with points pt 3 ps 3 
    1854.0 1599.0
    1596.0 1568.0
    2170.0 1626.0
    3649.0 1762.0
    1937.0 1604.0
    1243.0 1544.0
    2412.0 1648.0
    886.0 1508.0
    141.0 1436.0
    3088.0 1709.0
    292.0 1455.0
    2311.0 1639.0
    647.0 1484.0
    2662.0 1672.0
    1105.0 1529.0
    803.0 1499.0
    4218.0 1811.0
    731.0 1498.0
    3743.0 1769.0
    1371.0 1555.0
    4100.0 1800.0
    1446.0 1544.0
    2092.0 1587.0
    2863.0 1669.0
    2930.0 1681.0
    3197.0 1702.0
    3341.0 1709.0
    3483.0 1733.0
    e

I.e. we plot some points on top of the attached image. The contents of
the image don't matter. Here we just have a gray field. I'm using the
latest master branch, but I belive 5.4 does this too.

1. Running that script in an interactive terminal (I tried x11 and qt),
   the plot pops up just fine: the autoscale considered the image AND the
   points

2. We interactively zoom on any area with the mouse. This works too

3. We press 'u' to zoom out. The expectation is that we get back to the
   full view after step 1. Instead we get to a new view, zoomed-in on
   the points only, NOT on the image.

Something about the reverse yrange is confusing I think. Have I reported
this already?

Thanks.

Re: 5.4.rc2 second release candidate for gnuplot version 5.4

[Shigeharu T. <sh...@ie...>]

shige 08/28 2020
----------------

Tait wrote at 2020-06-09: 

| I ran gp54rc1-win64-mingw.exe downloaded from SF.
...
| I wanted to try the Windows terminal, too, but it hangs for a
| few seconds and then crashes, seemingly no matter what I try to
| plot.

Currently, I can't download the binary from SF. 

Well, I build windows binary from gnuplot source version 5.4.0 by
msys2-mingw64, and I ran it with wxt terminal as 
  gnuplot.exe -e "set term wxt; cd 'demo' ; load 'all.dem'"
then it works fine. 
For windows terminal, certainly
  gnuplot.exe -e "set term win; cd 'demo' ; load 'all.dem'"
crashes as reported. 
But, when I specify the window size explicitly as
  gnuplot.exe -e "set term win size 640,400; cd 'demo' ; load 'all.dem'"
it works fine. Please try.

+========================================================+
 Shigeharu TAKENO     NIigata Institute of Technology
                       kashiwazaki,Niigata 945-1195 JAPAN
 sh...@ie...   TEL(&FAX): +81-257-22-8161
+========================================================+

Re: [PATCH] cairolatex: respect transparency setting

[Henri M. <hen...@gm...>]

On 20/08/20, 21:31, Ethan A Merritt wrote:
> On Thursday, 20 August 2020 20:17:44 PDT Henri Menke wrote:
> > On 20/08/20, 20:01, Ethan A Merritt wrote:
> > > On Thursday, 20 August 2020 18:34:54 PDT Henri Menke wrote:
> > > > Currently the cairolatex terminal does not respect transparency options
> > > > given in the terminal setting.  In this minimal example the
> > > > notransparent option is ignored:
> > > > 
> > > >     set terminal cairolatex png notransparent
> > > >     set output "test.tex"
> > > >     plot sin(x)
> > > > 
> > > > Running this through gnuplot and then using `file' reveals that the
> > > > alpha channel is still present:
> > > > 
> > > >     $ gnuplot test.gnuplot && file test.png
> > > >     test.png: PNG image data, 1500 x 900, 8-bit/color RGBA, non-interlace
> > > 
> > > I think the raionale for the current code is that the png image created for
> > > inclusion in a LaTeX document must have a transparent background in order
> > > to support the "back" and "behind" attributes for text elements, since they
> > > will be drawn by the latex engine and then covered by inclusion of the
> > > graphics image.  The structure of the latex processing is:
> > > 
> > >     render text elements that are "behind" or "back"
> > >     \includegraphics{gnuplot_figure}
> > >     render text elements that are "front"
> > > 
> > > If you use a solid color background in the graphics image, it will hide
> > > any previously rendered text elements.
> > 
> > The reason why I'm looking into this is that currently it is impossible
> > to generate PDF/A when using plots generated by gnuplot.  PDF/A doesn't
> > allow transparency, so I'm trying to remove all of it (or at least make
> > it optional).  As noted in the patch, the default behaviour of using
> > transparency remains, so this is entirely optional.
> 
> I am researching this as I type, so please bear with me.
> 
> From Wikipedia I see
> 	Transparent objects and layers (Optional Content Groups) are 
> 	forbidden in PDF/A-1, but are allowed in PDF/A-2.
> PDF/A-1 was published in 2005; PDF/A-2 in 2011; PDF/A-3 in 2012;
> So maybe this is no longer a limitation?
> 
> From my own knowledge, there are commonly available tools that 
> render the transparency in a PDF document once, then save the result
> as an embedded image that no longer contains transparency.
> For example if you use gnuplot to create a PostScript file directly it cannot
> use transparency because PostScript itself does not support transparency.
> But if you use gnuplot to create a PDF file with transparency and then
> convert it
> 	pdf2ps input.pdf output.ps
> you get a PostScript file in which the transparency is "baked in" rather
> than interpreted.   Logically then you could take that PostScript file
> and convert it back to PDF, keeping the pre-rendered image that no
> longer uses an alpha channel. 
> 	ps2pdf output.ps new.pdf
> Whether that particular pair of tools produces a PDF/A compliant
> file I do not know.

This is actually a pretty good proposal.  In my actual plot I now have
added

    set terminal cairolatex pdf standalone header '\usepackage[a-1b]{pdfx}'
    set output "test.tex"
    # ...
    unset output
    !pdf2ps test-inc.pdf test-inc.ps
    !ps2pdf -dPDFA=1 test-inc.ps test-inc.pdf
    !pdflatex --interaction=nonstopmode test.tex

and the resulting document validates correctly with VeraPDF.  It's a bit
ugly and I don't know whether the ghostscript roundtrip would correctly
preserve fonts, but since the background image doesn't contain fonts, I
don't care.  Note that adding the -dPDFA=1 flag will make the PDF
structurally PDF/A compliant, but will not add metadata or color
profiles, so test-inc.pdf does not validate.  These extras are handled
by the pdfx LaTeX package.

To get my plots with PNG background compliant I use the patch from this
thread together with your suggestion `set tics front'.  Another option
would be to run mogrify to remove the transparency.

    set terminal cairolatex png standalone header '\usepackage[a-1b]{pdfx}'
    set output "test.tex"
    # ...
    unset output
    !mogrify -alpha remove test-inc.png
    !pdflatex --interaction=nonstopmode test.tex

So bottomlined my thesis now validates as PDF/A-1b, which is really
nice.

Cheers, Henri

> 
>    best,
> 
> 	Ethan
> 
> > 
> > Kind regards,
> > Henri
> > 
> > > 
> > > 	Ethan
> > > 
> > > > 
> > > > With this patch the transparency setting will be honoured.  The current
> > > > default setting of implicitly assuming transparency is unaltered.
> > > > ---
> > > >  term/cairo.trm | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > 
> > > > diff --git a/term/cairo.trm b/term/cairo.trm
> > > > index ee748dfe2..3ef76ecc6 100644
> > > > --- a/term/cairo.trm
> > > > +++ b/term/cairo.trm
> > > > @@ -849,7 +849,7 @@ void cairotrm_graphics()
> > > >  	plot.background.g = cairo_params->background.g;
> > > >  	plot.background.b = cairo_params->background.b;
> > > >  	gp_cairo_set_background(cairo_params->background);
> > > > -	if (ISCAIROLATEX || cairo_params->transparent)
> > > > +	if (cairo_params->transparent)
> > > >  		gp_cairo_clear_background(&plot);
> > > >  	else
> > > >  		gp_cairo_solid_background(&plot);
> > > > 
> > > 
> > > 
> > > 
> > > 
> 
> 
>