gnuplot-beta — Developers and beta testers of new gnuplot code

[PATCH v2] Support building development version without git

[Henri M. <hen...@gm...>]

Currently building the development version requires git to be installed
to fetch the `last modified' date from the commit information.  This is
okay when building from a git clone, but fails when downloading a
snapshot that does not contain the .git directory.

To this end, I implemented a fallback that uses the last modification
time of the Makefile.am, which corresponds to the last commit date when
using a fresh checkout.  The old version had the advantage that
`timestamp.h' would only be rebuilt when changing the git branch.  If
git is not available, this logic fails of course.  Instead I made
`timestamp.h' a phony target, i.e. it will be rebuilt unconditionally
every time.  The disadvantage here is that this will also trigger a
rebuild of every file that depends on `timestamp.h' but as far as I can
see this is only `version.c'.
---
This implements the suggestion by Miquel Garriga to use the mtime of
Makefile.am which is set to the last commit time on a fresh checkout.

 src/Makefile.am | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/Makefile.am b/src/Makefile.am
index 1b0d8136e..bb9461c52 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -107,12 +107,12 @@ endif
 
 DISTCLEANFILES = timestamp.h
 BUILT_SOURCES = timestamp.h
-git_current := $(shell cut -c6- $(top_srcdir)/.git/HEAD)
-timestamp.h: $(top_srcdir)/.git/${git_current} Makefile
+.PHONY: timestamp.h
+timestamp.h: Makefile
 	@echo Making $@
 	@echo "#ifndef GNUPLOT_TIMEBASE_H_INCLUDED" >$@t
 	@echo "#define GNUPLOT_TIMEBASE_H_INCLUDED" >>$@t
-	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%ci | cut -b-11`\";" >>$@t
+	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%cs || stat -c '%.10y' Makefile.am`\";" >>$@t
 	@echo "#endif /* GNUPLOT_TIMEBASE_H_INCLUDED */" >> $@t
 	@if cmp -s $@ $@t; then rm -f $@t; else mv $@t $@; fi
 
-- 
2.27.0

Re: [PATCH] Support building development version without git

[Henri M. <hen...@gm...>]

On 01/07/20, 03:30, Dima Kogan wrote:
> Henri Menke <hen...@gm...> writes:
> 
> > If I go to the SourceForge project page [1] and just hit the `Download
> > Snapshot' button in the upper right corner, I get a zip file with all
> > the source but no .git folder and also no timestamp.h. I suspect there
> > is no or at least no easy way to make SourceForge preprocess the
> > source when generating a snapshot.
> >
> > Within the Makefile determining the latest change would be tricky. The
> > only way I can think of right now would be to run stat on all the
> > files in the source tree and pick the newest one, but that is slow and
> > ugly. Maybe instead of using the current date if git is unavailable
> > just do echo '0000-00-00'. It should be pretty obvious that this is
> > not the true `Last modified' then.
> 
> Oh. Wow. OK. My feeling is that people should be either
> 
> - using release tarballs
> - using git
> 
> It's unfortunate that sourceforge makes another option available without
> asking, but I don't think it's worth the effort to support it, and a
> dummy string is just fine. Is a date string of "DATE UNKNOWN; PLEASE USE
> A RELEASE TARBALL OR A GIT CHECKOUT" valid for the purposes of
> timestamp.h? Ethan: do you want to support this use case in a better
> way?

The original motivation for this change was that I wanted to build the
development version on NixOS.  In the Nix packaging system you can build
stuff directly from a git checkout, but because the system emphasizes
reproducible builds, the build cannot depend on the contents of the .git
directory because these are actually non-deterministic (due to git gc),
so two checkouts of the same commit may actually have a different .git
directory.  This will lead to spurious build failures because the output
hashes differ.

    https://github.com/NixOS/nixpkgs/issues/8567
    https://github.com/NixOS/nixpkgs/issues/20521

I thought it would be a good idea to upstream the support for this but
if this is too much of a pain here, I can also patch it locally.

Kind regards,
Henri

> 
> 
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

Re: [PATCH] Support building development version without git

[Miquel G. <mi...@ic...>]

On Wed, 1 Jul 2020 12:18:16 +1200
Henri Menke <hen...@gm...> wrote:

> If I go to the SourceForge project page [1] and just hit the `Download
> Snapshot' button in the upper right corner, I get a zip file with all
> the source but no .git folder and also no timestamp.h.  I suspect there
> is no or at least no easy way to make SourceForge preprocess the source
> when generating a snapshot.
> 

You can use the mtime of any file inside the zip file. 
All them are set to the commit time of the git-hash you hit 'Download Snapshot' for.
Thus "stat -c '%y ' src/Makefile.am" will give the commit date.

For tarballs obtained via "git archive -o snapshot.tar githash" 
all the files in the tarball have the commit-time as mtime.

Miquel

Re: [PATCH] Support building development version without git

[Dima K. <gn...@di...>]

Henri Menke <hen...@gm...> writes:

> If I go to the SourceForge project page [1] and just hit the `Download
> Snapshot' button in the upper right corner, I get a zip file with all
> the source but no .git folder and also no timestamp.h. I suspect there
> is no or at least no easy way to make SourceForge preprocess the
> source when generating a snapshot.
>
> Within the Makefile determining the latest change would be tricky. The
> only way I can think of right now would be to run stat on all the
> files in the source tree and pick the newest one, but that is slow and
> ugly. Maybe instead of using the current date if git is unavailable
> just do echo '0000-00-00'. It should be pretty obvious that this is
> not the true `Last modified' then.

Oh. Wow. OK. My feeling is that people should be either

- using release tarballs
- using git

It's unfortunate that sourceforge makes another option available without
asking, but I don't think it's worth the effort to support it, and a
dummy string is just fine. Is a date string of "DATE UNKNOWN; PLEASE USE
A RELEASE TARBALL OR A GIT CHECKOUT" valid for the purposes of
timestamp.h? Ethan: do you want to support this use case in a better
way?

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Ethan A M. <me...@uw...>]

On Monday, 29 June 2020 19:51:04 PDT Dima Kogan wrote:
> Alright, I debugged this (rr was invaluable), and I know mostly what's
> going on. The flow is roughly this:
> 
> 1. The data is read here:
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/plot3d.c#l1037
> 
>    At the end of the first line, x is 3999.0000000000005, which is just outside the
>    xrange [0:3999]. This isn't unexpected due to the "using" expression
> 
> 2. Later we evaluate the x axis for that point (first row, last col):
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/axis.c#l2782
> 
>    3999.0000000000005>3999, so we set this to OUTRANGE
> 
>    In a perfect world we should be looking at the "with image" pixel
>    edges instead of just the center; like we do in step 5 below.
> 
> 3. Later we evaluate the z axis for that same point, but this is already
>    OUTRANGE, so we don't bother to use that point to set axis->min or
>    axis->max
> 
> 4. As a result the view_port_z[] range we set later is narrower than it
>    should be:
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/graphics.c#l4822
> 
> 5. And then, further on in in that function, that corner pixel is deemed
>    invisible. So the first row then has only 15 pixels, but the next row
>    has all 16, so we complain
> 
> The logic in step 5 (what we do for !visible pixels) looks suspicious,
> but I want to say the fundamental problem is that step 2 and step 5 have
> different logic for inrange/outrange. The top-right corner of the
> failing plot is OUT in step 2, but IN in step 5.
> 
> Any obvious fixes?

In the end the fix was trivial, but I wouldn't call it "obvious".

The clipping in proces_image() at step 5 correctly accounts for the pixel
extent on x and y regardless of what flag is in pixel->type.  So nothing is
gained by  setting it to OUTRANGE in step 2.  If instead we force all
pixels to be flagged INRANGE at that stage, the incorrect zrange
determination at step 3 doesn't happen and all pixels are kept. 
It wasn't obvious to me that marking all pixels INRANGE would preserve
both autoscaling and clipping, but it does.

fix commited to tip, queued for 5.4

	Ethan

Re: [PATCH] Support building development version without git

[Henri M. <hen...@gm...>]

On 30/06/20, 16:19, Ethan Merritt wrote:
> On Tuesday, 30 June 2020 16:03:22 PDT Henri Menke wrote:
> > Currently building the development version requires git to be installed
> > to fetch the `last modified' date from the commit information.  This is
> > okay when building from a git clone, but fails when downloading a
> > snapshot that does not contain the .git directory.
> 
> Yeah, that has been a problem.
> Thanks for looking at this.
> 
> > To this end, I implemented a fallback which just uses the current date
> > if git is not available.  
> 
> If you are building from a snapshot, the correct date should be the
> date of the snapshot.  I don't know how to do that.  Do you?

If I go to the SourceForge project page [1] and just hit the `Download
Snapshot' button in the upper right corner, I get a zip file with all
the source but no .git folder and also no timestamp.h.  I suspect there
is no or at least no easy way to make SourceForge preprocess the source
when generating a snapshot.

Within the Makefile determining the latest change would be tricky.  The
only way I can think of right now would be to run stat on all the files
in the source tree and pick the newest one, but that is slow and ugly.
Maybe instead of using the current date if git is unavailable just do
echo '0000-00-00'.  It should be pretty obvious that this is not the
true `Last modified' then.

Kind regards,
Henri

> 
> 	Ethan
> 
> > The old version had the advantage that
> > `timestamp.h' would only be rebuilt when changing the git branch.  If
> > git is not available, this logic fails of course.  Instead I made
> > `timestamp.h' a phony target, i.e. it will be rebuilt unconditionally
> > every time.  The disadvantage here is that this will also trigger a
> > rebuild of every file that depends on `timestamp.h' but as far as I can
> > see this is only `version.c'.
> > ---
> >  src/Makefile.am | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/src/Makefile.am b/src/Makefile.am
> > index 1b0d8136e..7eca23cfa 100644
> > --- a/src/Makefile.am
> > +++ b/src/Makefile.am
> > @@ -107,12 +107,12 @@ endif
> >  
> >  DISTCLEANFILES = timestamp.h
> >  BUILT_SOURCES = timestamp.h
> > -git_current := $(shell cut -c6- $(top_srcdir)/.git/HEAD)
> > -timestamp.h: $(top_srcdir)/.git/${git_current} Makefile
> > +.PHONY: timestamp.h
> > +timestamp.h: Makefile
> >  	@echo Making $@
> >  	@echo "#ifndef GNUPLOT_TIMEBASE_H_INCLUDED" >$@t
> >  	@echo "#define GNUPLOT_TIMEBASE_H_INCLUDED" >>$@t
> > -	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%ci | cut -b-11`\";" >>$@t
> > +	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%cs || date +'%Y-%m-%d'`\";" >>$@t
> >  	@echo "#endif /* GNUPLOT_TIMEBASE_H_INCLUDED */" >> $@t
> >  	@if cmp -s $@ $@t; then rm -f $@t; else mv $@t $@; fi
> >  
> > 
> 
> 
> 
> 

Re: [PATCH] Support building development version without git

[Ethan A M. <me...@uw...>]

On Tuesday, 30 June 2020 17:08:17 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> > It sounds reasonable to me.
> >
> > This seems to be surprisingly tricky. Previous attempts did not work out.
> > See commits:
> > 	9d0f148d  	d543154a
> 
> OK, I'll look in a bit. In the meantime, can you tell me how we build
> snapshots? Do we just 'tar cz --exclude .git ...' or is there a script
> of some sort?

That's a large part of the problem.

We clearly can build a stand-alone tarball if we have control over it.
That's what I do to make a release package.

Unfortunately so far as I know we don't have control over the snapshot
building process on the SourceForge page.
It looks like they just dump the current content of the repository into a tarball.

I could try sending a support request to the SourceForge ops, but I'm
not all that hopeful about getting a useful response.

	Ethan  

Re: [PATCH] Support building development version without git

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> It sounds reasonable to me.
>
> This seems to be surprisingly tricky. Previous attempts did not work out.
> See commits:
> 	9d0f148d  	d543154a

OK, I'll look in a bit. In the meantime, can you tell me how we build
snapshots? Do we just 'tar cz --exclude .git ...' or is there a script
of some sort?

Re: [PATCH] Support building development version without git

[Ethan A M. <me...@uw...>]

On Tuesday, 30 June 2020 16:31:06 PDT Dima Kogan wrote:
> Dima Kogan <gn...@di...> writes:
> 
> > Ethan Merritt <eam...@gm...> writes:
> >
> >> If you are building from a snapshot, the correct date should be the
> >> date of the snapshot. I don't know how to do that. Do you?
> >
> > This should be encoded in the snapshot in some way.

> Looking at the patch, I think that we should generate the timestamp.h at
> snapshot-building time, and the Makefile in the snapshot shouldn't ever
> generate it or delete it. Reasonable?

It sounds reasonable to me.

This seems to be surprisingly tricky. Previous attempts did not work out.
See commits:
	9d0f148d  	d543154a

	Ethan
	

Re: [PATCH] Support building development version without git

[Dima K. <gn...@di...>]

Dima Kogan <gn...@di...> writes:

> Ethan Merritt <eam...@gm...> writes:
>
>> If you are building from a snapshot, the correct date should be the
>> date of the snapshot. I don't know how to do that. Do you?
>
> This should be encoded in the snapshot in some way.

Looking at the patch, I think that we should generate the timestamp.h at
snapshot-building time, and the Makefile in the snapshot shouldn't ever
generate it or delete it. Reasonable?

Re: [PATCH] Support building development version without git

[Dima K. <gn...@di...>]

Ethan Merritt <eam...@gm...> writes:

> If you are building from a snapshot, the correct date should be the
> date of the snapshot. I don't know how to do that. Do you?

This should be encoded in the snapshot in some way. How are we building
the snapshot? The Makefile in the snapshot shouldn't have any git
commands, or at least should know to not run them.

Re: [PATCH] Support building development version without git

[Ethan M. <eam...@gm...>]

On Tuesday, 30 June 2020 16:03:22 PDT Henri Menke wrote:
> Currently building the development version requires git to be installed
> to fetch the `last modified' date from the commit information.  This is
> okay when building from a git clone, but fails when downloading a
> snapshot that does not contain the .git directory.

Yeah, that has been a problem.
Thanks for looking at this.

> To this end, I implemented a fallback which just uses the current date
> if git is not available.  

If you are building from a snapshot, the correct date should be the
date of the snapshot.  I don't know how to do that.  Do you?

	Ethan

> The old version had the advantage that
> `timestamp.h' would only be rebuilt when changing the git branch.  If
> git is not available, this logic fails of course.  Instead I made
> `timestamp.h' a phony target, i.e. it will be rebuilt unconditionally
> every time.  The disadvantage here is that this will also trigger a
> rebuild of every file that depends on `timestamp.h' but as far as I can
> see this is only `version.c'.
> ---
>  src/Makefile.am | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/src/Makefile.am b/src/Makefile.am
> index 1b0d8136e..7eca23cfa 100644
> --- a/src/Makefile.am
> +++ b/src/Makefile.am
> @@ -107,12 +107,12 @@ endif
>  
>  DISTCLEANFILES = timestamp.h
>  BUILT_SOURCES = timestamp.h
> -git_current := $(shell cut -c6- $(top_srcdir)/.git/HEAD)
> -timestamp.h: $(top_srcdir)/.git/${git_current} Makefile
> +.PHONY: timestamp.h
> +timestamp.h: Makefile
>  	@echo Making $@
>  	@echo "#ifndef GNUPLOT_TIMEBASE_H_INCLUDED" >$@t
>  	@echo "#define GNUPLOT_TIMEBASE_H_INCLUDED" >>$@t
> -	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%ci | cut -b-11`\";" >>$@t
> +	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%cs || date +'%Y-%m-%d'`\";" >>$@t
>  	@echo "#endif /* GNUPLOT_TIMEBASE_H_INCLUDED */" >> $@t
>  	@if cmp -s $@ $@t; then rm -f $@t; else mv $@t $@; fi
>  
> 

[PATCH] Support building development version without git

[Henri M. <hen...@gm...>]

Currently building the development version requires git to be installed
to fetch the `last modified' date from the commit information.  This is
okay when building from a git clone, but fails when downloading a
snapshot that does not contain the .git directory.

To this end, I implemented a fallback which just uses the current date
if git is not available.  The old version had the advantage that
`timestamp.h' would only be rebuilt when changing the git branch.  If
git is not available, this logic fails of course.  Instead I made
`timestamp.h' a phony target, i.e. it will be rebuilt unconditionally
every time.  The disadvantage here is that this will also trigger a
rebuild of every file that depends on `timestamp.h' but as far as I can
see this is only `version.c'.
---
 src/Makefile.am | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/Makefile.am b/src/Makefile.am
index 1b0d8136e..7eca23cfa 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -107,12 +107,12 @@ endif
 
 DISTCLEANFILES = timestamp.h
 BUILT_SOURCES = timestamp.h
-git_current := $(shell cut -c6- $(top_srcdir)/.git/HEAD)
-timestamp.h: $(top_srcdir)/.git/${git_current} Makefile
+.PHONY: timestamp.h
+timestamp.h: Makefile
 	@echo Making $@
 	@echo "#ifndef GNUPLOT_TIMEBASE_H_INCLUDED" >$@t
 	@echo "#define GNUPLOT_TIMEBASE_H_INCLUDED" >>$@t
-	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%ci | cut -b-11`\";" >>$@t
+	@echo "const char gnuplot_date[] = \"`git --git-dir '$(top_srcdir)/.git' log -1 --format=%cs || date +'%Y-%m-%d'`\";" >>$@t
 	@echo "#endif /* GNUPLOT_TIMEBASE_H_INCLUDED */" >> $@t
 	@if cmp -s $@ $@t; then rm -f $@t; else mv $@t $@; fi
 
-- 
2.27.0

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Hans-Bernhard B. <HBB...@t-...>]

Am 30.06.2020 um 20:49 schrieb Ethan A Merritt:
> 
> I have some reservations about whether it ever makes sense to clip
> the contents of a plot `with image`, since in the general case of
> rotation+translation, clipping will not leave a rectangle and thus the
> optimized image-data protocols that pass the image as a single chunk
> of data won't work.

OTOH, the alternative would be that the boundaries set up for clipping 
willl be disrespected completely.  That cannot really be considered correct.

> The current gnuplot code seems to assume that clipping will cleanly
> remove a constant width slice from each edge.  Perhaps one fix is to first
> detect that clipping exists, and if so fall back to 'with image pixels' so
> that each pixel is considered individually.

That does make more sense, IMHO.

> B) For any given axis (z in this case) the program should never try to
> apply both autoscaling and clipping!   

Not so fast.  It's still possible for one end of an axis to be clipped, 
while the other is autoscaled.

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Ethan A M. <me...@uw...>]

On Monday, 29 June 2020 19:51:04 PDT Dima Kogan wrote:
> Alright, I debugged this (rr was invaluable), and I know mostly what's
> going on. The flow is roughly this:
> 
> 1. The data is read here:
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/plot3d.c#l1037
> 
>    At the end of the first line, x is 3999.0000000000005, which is just outside the
>    xrange [0:3999]. This isn't unexpected due to the "using" expression
> 
> 2. Later we evaluate the x axis for that point (first row, last col):
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/axis.c#l2782
> 
>    3999.0000000000005>3999, so we set this to OUTRANGE
> 
>    In a perfect world we should be looking at the "with image" pixel
>    edges instead of just the center; like we do in step 5 below.
> 
> 3. Later we evaluate the z axis for that same point, but this is already
>    OUTRANGE, so we don't bother to use that point to set axis->min or
>    axis->max
> 
> 4. As a result the view_port_z[] range we set later is narrower than it
>    should be:
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/graphics.c#l4822
> 
> 5. And then, further on in in that function, that corner pixel is deemed
>    invisible. So the first row then has only 15 pixels, but the next row
>    has all 16, so we complain
> 
> The logic in step 5 (what we do for !visible pixels) looks suspicious,
> but I want to say the fundamental problem is that step 2 and step 5 have
> different logic for inrange/outrange. The top-right corner of the
> failing plot is OUT in step 2, but IN in step 5.
> 
> Any obvious fixes?

I agree with your analysis.

I have some reservations about whether it ever makes sense to clip
the contents of a plot `with image`, since in the general case of
rotation+translation, clipping will not leave a rectangle and thus the
optimized image-data protocols that pass the image as a single chunk
of data won't work.
The current gnuplot code seems to assume that clipping will cleanly
remove a constant width slice from each edge.  Perhaps one fix is to first
detect that clipping exists, and if so fall back to 'with image pixels' so
that each pixel is considered individually.

Your case, however, triggers uneven clipping not because of rotation
but simply due to round-off error.  Two fixes occur to me.

A) The one implied by your point 2 above: auto-scaling and clipping
should take into account the full width of the pixel.
That can probably be done but I will put that on hold for now.

B) For any given axis (z in this case) the program should never try to
apply both autoscaling and clipping!   I don't know whether this glitch
can happen for other plot styles, but I will look for the most generic
possible place to prevent it.

	Ethan

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Ethan A M. <me...@uw...>]

On Monday, 29 June 2020 20:55:26 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> > On Monday, 29 June 2020 19:51:04 PDT Dima Kogan wrote:
> >> Alright, I debugged this (rr was invaluable), and I know mostly what's
> >> going on. The flow is roughly this:
> >>
> >> 1. The data is read here:
> >>
> >>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/plot3d.c#l1037
> >>
> >>    At the end of the first line, x is 3999.0000000000005, which is just outside the
> >>    xrange [0:3999]. This isn't unexpected due to the "using" expression
> >
> > But the x coordinates should come out identically the same in the "good" and "bad"
> > cases, right?  So how does this explain a different between them?
> 
> In the "good" case we're getting lucky in step 5. The corners (which
> were out-of-bounds in step 2, and were thus ignored for computing
> view_port_z[]) still end up in-bounds in step 5. There's an extra step
> that I didn't mention in the previous email:

Very strange.  The z range should be totally irrelevant to "set view map".
z as a coordinate doesn't matter because it's in projection.
z as an image pixel value doesn't matter because the image grid doesn't
care about the pixel value, only the number of pixels.

But you're right, it does seem to make a difference in this case
For instance:
    gnuplot> splot $MATRIX matrix  using ($1*266.6):($2*199.9090909090909):3 title "" with image
         warning: Visible pixel grid has a scan line longer than previous scan lines.
    gnuplot> splot $MATRIX matrix  using ($1*266.6):($2*199.9090909090909):(0):3 title "" with image
         works with no problem

But opening up the z range explicitly does not fix the problem.
So I still don't understand exactly what's going on here. 

	I'll look harder tomorrow,

		Ethan


> 3.5. We expand the autoranged axis extents to nice, round numbers:
> 
>      https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/axis.c#l960
> 
> This provides some margin so that getting lucky in this way is possible.

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> On Monday, 29 June 2020 19:51:04 PDT Dima Kogan wrote:
>> Alright, I debugged this (rr was invaluable), and I know mostly what's
>> going on. The flow is roughly this:
>>
>> 1. The data is read here:
>>
>>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/plot3d.c#l1037
>>
>>    At the end of the first line, x is 3999.0000000000005, which is just outside the
>>    xrange [0:3999]. This isn't unexpected due to the "using" expression
>
> But the x coordinates should come out identically the same in the "good" and "bad"
> cases, right?  So how does this explain a different between them?

In the "good" case we're getting lucky in step 5. The corners (which
were out-of-bounds in step 2, and were thus ignored for computing
view_port_z[]) still end up in-bounds in step 5. There's an extra step
that I didn't mention in the previous email:

3.5. We expand the autoranged axis extents to nice, round numbers:

     https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/axis.c#l960

This provides some margin so that getting lucky in this way is possible.

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Ethan A M. <me...@uw...>]

On Monday, 29 June 2020 19:51:04 PDT Dima Kogan wrote:
> Alright, I debugged this (rr was invaluable), and I know mostly what's
> going on. The flow is roughly this:
> 
> 1. The data is read here:
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/plot3d.c#l1037
> 
>    At the end of the first line, x is 3999.0000000000005, which is just outside the
>    xrange [0:3999]. This isn't unexpected due to the "using" expression

But the x coordinates should come out identically the same in the "good" and "bad"
cases, right?  So how does this explain a different between them?

	Ethan

> 
> 2. Later we evaluate the x axis for that point (first row, last col):
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/axis.c#l2782
> 
>    3999.0000000000005>3999, so we set this to OUTRANGE
> 
>    In a perfect world we should be looking at the "with image" pixel
>    edges instead of just the center; like we do in step 5 below.
> 
> 3. Later we evaluate the z axis for that same point, but this is already
>    OUTRANGE, so we don't bother to use that point to set axis->min or
>    axis->max
> 
> 4. As a result the view_port_z[] range we set later is narrower than it
>    should be:
> 
>    https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/graphics.c#l4822
> 
> 5. And then, further on in in that function, that corner pixel is deemed
>    invisible. So the first row then has only 15 pixels, but the next row
>    has all 16, so we complain
> 
> The logic in step 5 (what we do for !visible pixels) looks suspicious,
> but I want to say the fundamental problem is that step 2 and step 5 have
> different logic for inrange/outrange. The top-right corner of the
> failing plot is OUT in step 2, but IN in step 5.
> 
> Any obvious fixes?
> 
> Thanks
> 
> 
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> 


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
MS 357742,   University of Washington, Seattle 98195-7742

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Dima K. <gn...@di...>]

Alright, I debugged this (rr was invaluable), and I know mostly what's
going on. The flow is roughly this:

1. The data is read here:

   https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/plot3d.c#l1037

   At the end of the first line, x is 3999.0000000000005, which is just outside the
   xrange [0:3999]. This isn't unexpected due to the "using" expression

2. Later we evaluate the x axis for that point (first row, last col):

   https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/axis.c#l2782

   3999.0000000000005>3999, so we set this to OUTRANGE

   In a perfect world we should be looking at the "with image" pixel
   edges instead of just the center; like we do in step 5 below.

3. Later we evaluate the z axis for that same point, but this is already
   OUTRANGE, so we don't bother to use that point to set axis->min or
   axis->max

4. As a result the view_port_z[] range we set later is narrower than it
   should be:

   https://sourceforge.net/p/gnuplot/gnuplot-main/ci/branch-5-4-stable/tree/src/graphics.c#l4822

5. And then, further on in in that function, that corner pixel is deemed
   invisible. So the first row then has only 15 pixels, but the next row
   has all 16, so we complain

The logic in step 5 (what we do for !visible pixels) looks suspicious,
but I want to say the fundamental problem is that step 2 and step 5 have
different logic for inrange/outrange. The top-right corner of the
failing plot is OUT in step 2, but IN in step 5.

Any obvious fixes?

Thanks

Re: Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Ethan A M. <me...@uw...>]

On Monday, 29 June 2020 16:37:12 PDT Dima Kogan wrote:
> Can somebody tell me what this means?
> 
> I'm plotting a grid of data as an image. And I have two identical plot
> commands, differing only in the data being plotted (attached). One
> works, and the other doesn't, throwing the error in the subject. Does
> anybody know what this means? It should work...

I have no idea what is happening.

"with image pixels" makes it work.
set xrange [0:4000] also makes it work.
But the two images show a different cbrange, which is worrisome.

	Ethan

Plot failure: "Visible pixel grid has a scan line longer than previous scan lines"

[Dima K. <gn...@di...>]

Can somebody tell me what this means?

I'm plotting a grid of data as an image. And I have two identical plot
commands, differing only in the data being plotted (attached). One
works, and the other doesn't, throwing the error in the subject. Does
anybody know what this means? It should work...

Re: Cloning errors - SOLVED

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> On Saturday, 27 June 2020 11:55:25 PDT Dima Kogan wrote:
>
>> From git's point of view this new master branch would be a fork that has
>> branched off at the time of the date fix: in 2006.
>
> What state would that leave the 5.2 and 5.4 stable branches in,
> that split off after 2006?  Do they see the same fix?
> Does the branch structure even survive?

Each git commit has a hash (a long hex sequence) that is used to
uniquely refer to it. A branch or a tag is just a convenient alias to a
specific hash.

If we go back to fix the date, then that commit gets a new hash AND ALL
DOWNSTREAM COMMITS get a new hash too. I.e. we made a fork. The original
hashes that we had before the fix are still there: the filter-branch
command creates a new sequence of commits, without deleting the old
ones.

What we do with that new sequences of commits is up to us. Since
branches are aliases to existing hashes, by creating a new sequence of
commits with a fixed date we haven't actually changed anything about
existing branches or tags. Each branch or tag we want to fix would need
to be fixed with filter-branch and then force-pushed to the server
(because this wouldn't be the usual append-only push). Probably the best
bet would be to leave all the old tags and branches alone, and to only
fix the under-development master branch. But that's a choice we make.


> Yeah, I don't think this particular glitch warrants a fork. I don't
> really see why a date typo is any worse than any other typo in the
> commit message - the structure and sequence of the commit chain is not
> affected.

I think I agree. It doesn't sound like many people are experiencing this
failure.

Re: Cloning errors - SOLVED

[Ethan A M. <me...@uw...>]

On Saturday, 27 June 2020 11:55:25 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> >> It isn't particularly difficult to fix this for the master branch,
> >> but since this would be a history rewrite, the resulting tree would
> >> be a fork, effectively. There aren't a ton of people committing to
> >> this repo, so fixing it wouldn't be too disruptive. Strong feelings
> >> either way?
> >
> > Really? It's fixable? I thought you guys all told me it was basically
> > impossible to edit a commit log entry deep in the tree.
> 
> /I/ never told you anything about it :)
> 
> Commits are immutable. A fix would be to edit the offending commits to
> create new commits with a more reasonable date. Then everything in the
> future from those rewritten commits would be replayed to create a new
> master branch that doesn't have the tainted history.
> 
> From git's point of view this new master branch would be a fork that has
> branched off at the time of the date fix: in 2006.

What state would that leave the 5.2 and 5.4 stable branches in,
that split off after 2006?  Do they see the same fix?
Does the branch structure even survive?

> > What's the recipe for doing so?
> 
> "git filter-branch"

Thanks. I will read up on it.
 
> > 
> > FWIW there is at least one other date error in the history. But really
> > there are many places I'd go back to correct typos if it were
> > possible. I think git would be much improved by keeping the commit
> > messages in such a way that they could be edited later without
> > disruption.
> 
> You can go back and change whatever you like, but each time you do that,
> you're making a new fork. In my view it could be justified to fix a
> failure (like the one that started this thread), but not for anything
> else. Even in this case, it's not obviously justified: you need an old
> git and/or i686 AND the failure only shows up if you "git fsck"

Yeah, I don't think this particular glitch warrants a fork.
I don't really see why a date typo is any worse than any other typo
in the commit message - the structure and sequence of the commit
chain is not affected. 

> This is relatively straightforward. I can make a fixed master branch, if
> you'd like to look at it.

I'd rather learn about the process and experiment on my own.
I'm not sure that looking at the result of someone else's fix teaches
me everything needed.

	thanks,
		Ethan

Re: Cloning errors - SOLVED

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

>> It isn't particularly difficult to fix this for the master branch,
>> but since this would be a history rewrite, the resulting tree would
>> be a fork, effectively. There aren't a ton of people committing to
>> this repo, so fixing it wouldn't be too disruptive. Strong feelings
>> either way?
>
> Really? It's fixable? I thought you guys all told me it was basically
> impossible to edit a commit log entry deep in the tree.

/I/ never told you anything about it :)

Commits are immutable. A fix would be to edit the offending commits to
create new commits with a more reasonable date. Then everything in the
future from those rewritten commits would be replayed to create a new
master branch that doesn't have the tainted history.

>From git's point of view this new master branch would be a fork that has
branched off at the time of the date fix: in 2006.


> What's the recipe for doing so?

"git filter-branch"

This is relatively straightforward. I can make a fixed master branch, if
you'd like to look at it.


> FWIW there is at least one other date error in the history. But really
> there are many places I'd go back to correct typos if it were
> possible. I think git would be much improved by keeping the commit
> messages in such a way that they could be edited later without
> disruption.

You can go back and change whatever you like, but each time you do that,
you're making a new fork. In my view it could be justified to fix a
failure (like the one that started this thread), but not for anything
else. Even in this case, it's not obviously justified: you need an old
git and/or i686 AND the failure only shows up if you "git fsck"

Re: Cloning errors - SOLVED

[Ethan A M. <me...@uw...>]

On Saturday, 27 June 2020 11:20:40 PDT Dima Kogan wrote:
> Colin Baxter <m4...@ya...> writes:
> 
> > If I comment-out the `fsckObjects = true' setting in my ~/.gitconfig
> > then I can pull the gnuplot repro successfully.
> 
> Ah, I can reproduce it now. Thanks for poking at it, Colin. On an older
> i686 box:
> 
>   $ git fsck
> 
>   Checking object directories: 100% (256/256), done.
>   error in commit f20ad5ffaa212876da3efb6a9a6f1ea3f8082734: badDateOverflow: invalid author/committer line - date causes integer overflow
>   error in commit 3cc239823f5447fdb739014e076463c8da6225dd: badDateOverflow: invalid author/committer line - date causes integer overflow
>   Checking objects: 100% (86573/86573), done.
> 
> Both of these commits are from 2006, but "git log" says "Date: Thu Jan 1
> 00:00:00 1970 +0000". On a more recent amd64 box I don't get "git fsck"
> to complain, but the commits are still there, and their date is more
> interesting: "Date: Sun Jul 27 02:03:19 2206 +0200".
> 
> It isn't particularly difficult to fix this for the master branch, but
> since this would be a history rewrite, the resulting tree would be a
> fork, effectively. There aren't a ton of people committing to this repo,
> so fixing it wouldn't be too disruptive. Strong feelings either way?

Really?   It's fixable?
I thought you guys all told me it was basically impossible to edit a commit log
entry deep in the tree.  What's the recipe for doing so?  

FWIW there is at least one other date error in the history.  But really there are
many places I'd go back to correct typos if it were possible.   I think git would be
much improved by keeping the commit messages in such a way that they could
be edited later without disruption.

	Ethan