gnuplot-beta — Developers and beta testers of new gnuplot code

Re: Short vectors are not plotted

[Dima K. <gn...@di...>]

sfeam <sf...@us...> writes:

>> I.e. we convert the vector start/end to integer terminal coords; short
>> vectors become 0-vectors in this representation, so the
>> terminal-specific arrow() function plots nothing.
>>
>> This is a bug: checking for a 0-vector should happen before we convert
>> to integer pixel coordinates.
>
> I am not following the logic.
>
> If it is a 0-length vector then isn't it correct to not draw it?
> Even if you keep the floating point representation for a bit longer,
> at the time you convert to integer terminal coordinates it will
> again become 0-length and thus not drawn.  What am I missing?
>
> Is the idea that you want to draw an arrowhead even if the
> shaft length is zero?

Right. It's not actually a length-0 vector, but its length is < 1 pixel
long. We should still draw an arrowhead, and the arrowhead should point
in the correct direction. Once we convert to integers, then even if we
wanted to draw something, we can't do it, since the direction has been
lost.

Here's a (hopefully convincing) thought: let's say you're looking at a
plot of a vector field. Then you zoom out repeatedly. I would expect the
plotted vector lengths to get shorted as you zoom out. The arrowheads
maybe would get smaller too, depending on how they're plotted. But the
transitions should be smooth. What currently happens is that as you zoom
out, some subset of the vectors disappears. As you zoom out more, some
OTHER subset disappears. This subset could be entirely different, and
some vectors actually come back. Eventually they all disappear, but as
they do so, sometimes they're plotted in an intermediate state where the
arrowhead IS plotted, but its direction is completely wrong.

Try with the example attached in the last email; use the mouse wheel to
zoom out.

Re: Roundup of GNUPLOT conversion issues

[Hans-Bernhard B. <HBB...@t-...>]

Am 29.10.2017 um 22:41 schrieb Daniel J Sebald:

> Thanks for the -I '\$[A-Z]*[a-z]*', it's a major help.  I've done a 
> comparison of cvs2git and CVS at GNUPLOT_RELEASE_3_7_3 and get agreement 
> aside from the cruft of empty CVS directories an so on.  There are these 
> few variations on the RCS-style lines:
[...]

Those are caused by the $Log$ keyword, which expands differently from 
all the other RCS/SCCS keywords, so it cannot be filtered out by diff.

Re: Short vectors are not plotted

[sfeam <sf...@us...>]

On Sunday, 29 October 2017 13:26:39 Dima Kogan wrote:
> Hi all, a non-version-control question for yall!
> 
> I'm trying to plot a vector field (attached). The vectors start at an
> evenly-spaced 20x20 grid. On my machine when I run the script, plotting
> into an x11 terminal, I don't see all 400 vectors, I see maybe 1/4 of
> them; the rest don't show up at all. It turns out what happens is this:
> 
>   do_plot()
>   {
>     ...
>     plot_vectors();
>     ...
>   }
> 
>   plot_vectors()
>   {
>     ...
>     for(vectors)
>     {
>       double x1_data_coords, x2_data_coords, ...;
>       ...
>       int x1_terminal_coords = map_x(x1_data_coords);
>       int x2_terminal_coords = map_x(x2_data_coords);
>       ...
>       (terminal->arrow) (x1_terminal_coords, y1_terminal_coords, ...);
>     }
>   }
> 
>   arrow(int x1, int y1, ...)
>   {
>      if( x1 == x2 && y1 == y2 )
>        return;
> 
>      ...
>   }
> 
> 
> I.e. we convert the vector start/end to integer terminal coords; short
> vectors become 0-vectors in this representation, so the
> terminal-specific arrow() function plots nothing.
> 
> This is a bug: checking for a 0-vector should happen before we convert
> to integer pixel coordinates.

I am not following the logic.

If it is a 0-length vector then isn't it correct to not draw it?
Even if you keep the floating point representation for a bit longer,
at the time you convert to integer terminal coordinates it will
again become 0-length and thus not drawn.  What am I missing?

Is the idea that you want to draw an arrowhead even if the
shaft length is zero?
 
	Ethan


> I'm attaching a patch series to fix this. The fundamental difference is
> a change to the arrow() terminal callback to take FLOATING-POINT
> terminal coordinates, so the vector direction, length can still be
> accurately computed. This feels incomplete, since I suspect there're
> other terminal functions that can benefit from such a change, but this
> works.
> 
> Note that these patches are made with 'git format-patch', so once the
> conversion is complete, they should be applied with 'git am'. This
> imports the patch series together with the commit metadata (author,
> message, etc).
> 
> 
> 

Re: Roundup of GNUPLOT conversion issues

[Daniel J S. <dan...@ie...>]

On 10/29/2017 03:48 PM, Hans-Bernhard Bröker wrote:
> Am 29.10.2017 um 20:31 schrieb Daniel J Sebald:
> 
>> I don't think the exact tarballs are there.  I sort of have a 
>> recollection of whenever a release was to be made, the tarball 
>> required some extra effort on someone's part.
> 
> Not back then they didn't.  The tarballs can be generated directly from 
> a fresh working copy, just by running
> 
>      ./configure && make dist
> 
> But all that effectively does is make a version-named copy of the source 
> tree without "CVS" directories and .cvsignore files, and tarball that.
> 
> Any actual differences between that and the official release tarball are 
> down to either
> 
> a) auto-tools of different versions being re-run at 'make' time, or
> b) last-minute fixes applied directly to the tarball, manually
> 
> Only when we switched to automake, dropped all generated files from the 
> repository, and started using the 'prepare' script instead, did tarball 
> contents start to be really different from fresh working copies of their 
> corresponding CVS tags.

Thanks for the -I '\$[A-Z]*[a-z]*', it's a major help.  I've done a 
comparison of cvs2git and CVS at GNUPLOT_RELEASE_3_7_3 and get agreement 
aside from the cruft of empty CVS directories an so on.  There are these 
few variations on the RCS-style lines:

diff -ur '--exclude=.git' -I '\$[A-Z]*[a-z]*' 
/home/sebald/gnuplot/gnuplot/gnuplot/docs/old/ChangeLog.old 
/home/sebald/gnuplot/git_translation/cvs2git_test/clonedgit/myproject/gnuplot/docs/old/ChangeLog.old
--- /home/sebald/gnuplot/gnuplot/gnuplot/docs/old/ChangeLog.old 
2017-10-29 15:27:21.998077850 -0500
+++ 
/home/sebald/gnuplot/git_translation/cvs2git_test/clonedgit/myproject/gnuplot/docs/old/ChangeLog.old 
2017-10-28 22:02:43.908845219 -0500
@@ -8,10 +8,7 @@
   * working. This means that the revision numbers in the individual files
   * do not agree with the one in the log. I hope to fix this some time 
soon.
   *
- * $Log: ChangeLog.old,v $
- * Revision 1.1  1998/11/16 13:07:33  lhecking
- * Moved from version.c.
- *
+ * $Log$
   *
   * I have moved these entries from version.c. All future code changes
   * should be logged in ChangeLog. Lars Hecking
Only in /home/sebald/gnuplot/gnuplot/gnuplot/docs/old: CVS
diff -ur '--exclude=.git' -I '\$[A-Z]*[a-z]*' 
/home/sebald/gnuplot/gnuplot/gnuplot/docs/old/makefile.r 
/home/sebald/gnuplot/git_translation/cvs2git_test/clonedgit/myproject/gnuplot/docs/old/makefile.r
--- /home/sebald/gnuplot/gnuplot/gnuplot/docs/old/makefile.r	2002-01-26 
12:55:02.000000000 -0600
+++ 
/home/sebald/gnuplot/git_translation/cvs2git_test/clonedgit/myproject/gnuplot/docs/old/makefile.r 
2017-10-29 15:49:42.034090965 -0500
@@ -1,10 +1,7 @@

-# $Id: makefile.r,v 1.1.2.1 2002/01/26 18:55:02 lhecking Exp $
-#
-# $Log: makefile.r,v $
-# Revision 1.1.2.1  2002/01/26 18:55:02  lhecking
-# Support for pdf and W3C Scalable Vector Graphics output.
+# $Id$
  #
+# $Log$
  # Revision 1.1  1998/12/09 17:24:30  lhecking
  # Moved from ../..
  #
diff -ur '--exclude=.git' -I '\$[A-Z]*[a-z]*' 
/home/sebald/gnuplot/gnuplot/gnuplot/docs/old/README.3p5 
/home/sebald/gnuplot/git_translation/cvs2git_test/clonedgit/myproject/gnuplot/docs/old/README.3p5
--- /home/sebald/gnuplot/gnuplot/gnuplot/docs/old/README.3p5	2017-10-29 
15:27:22.006077850 -0500
+++ 
/home/sebald/gnuplot/git_translation/cvs2git_test/clonedgit/myproject/gnuplot/docs/old/README.3p5 
2017-10-28 22:02:43.912845219 -0500
@@ -1,12 +1,9 @@
  This is a bugfix to version 3.4.


-# $Id: README.3p5,v 1.1 1998/11/16 13:01:23 lhecking Exp $
-#
-# $Log: README.3p5,v $
-# Revision 1.1  1998/11/16 13:01:23  lhecking
-# Moved from top level dir.
+# $Id$
  #
+# $Log$
  # Revision 1.1  1993/09/27 17:07:30  alex
  # gnuplot 3.5 release
  #


The cvs-fast-export result, with reposurgeon rebase of the branch to the 
same point of the cvs2git branch doesn't match so well.  Here are the 
files that differ:

amiga.c
Copyright
ctrl87.c
ctrl87.h
demo/gnuplot.rot
docs/gpcard.tex
docs/old/ChangeLog.old  [This is RCSid diff, does not count]
docs/old/makefile.r     [Ditto]
docs/old/README.3p5     [Ditto]
fnproto.h
intergra.x11
os9.c
win/wgnuplib.c
win/wprinter.c
win/wresourc.h

In qgit applied to cvs2git translation, I can trace the changes in these 
files back to the changeset "Import of beta 343." (amiga.c, Copyright, 
et al.) and "Import of beta 344." (demo/gnuplot.rot).  I believe what we 
are missing is as simple as merging those utility branches back into 
master branch at the start of the repository.

Dan

Re: Let's clean up the recent stuff and hide old messy stuff under a rug

[Eric S. R. <es...@th...>]

Daniel J Sebald <dan...@ie...>:
> I didn't suggest any effort to remove tags etc.  Would have been fine with
> historical tags and whatever synthetic branches are added.  What has
> mattered is recalling something from CVS and from the repository and have
> them match.

esr@snark:/mnt/vault/esr-vault/gnuplot-conversion$ checkbranches
Master branch differences:
Switched to branch 'master'
5-2 stable differences:
Switched to branch 'branch-5-2-stable'
5-0 stable differences:
Switched to branch 'branch-5-0-stable'
4-6 stable differences:
Switched to branch 'branch-4-6-stable'
4-4 stable differences:
Switched to branch 'branch-4-4-stable'
4-2 stable differences:
Switched to branch 'branch-4-2-stable'
4-0 stable differences:
Switched to branch 'branch-4-0-stable'
/mnt/vault/esr-vault/gnuplot-conversion/gnuplot-git
Switched to branch 'master'
esr@snark:/mnt/vault/esr-vault/gnuplot-conversion$ 

For 4-0-stable and everything later we're there already.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Let's clean up the recent stuff and hide old messy stuff under a rug

[Daniel J S. <dan...@ie...>]

On 10/29/2017 03:34 PM, sfeam wrote:
> On Sunday, 29 October 2017 14:01:15 Daniel J Sebald wrote:
>> I've experimented with something called cvs2git, which I believe does a
>> pretty good job of constructing branches and merges, but has some
>> drawbacks.
> 
> I think we are past the stage of looking for alternative conversion utilities.

The point was something for comparison purposes and confirmation of the 
branch points for which cvs2git seems to match what I've been 
concluding, but others don't seem to trust.  The reason for doing this 
is that we seem to be guessing at where things should go and how to 
devise shims to compensate.


> Can't we just flatten everything prior to version 4.0 into a baseline
> snapshot that is the root for everything subsequent to that?

Spend time on that, or spend time to make that first dozen changesets in 
line with cvs2git.  Whatever the case, we have to *start* from a point 
which is in agreement with the CVS repository so that the diffs going 
forward keep things in sync.


> I really don't see why anyone will ever need to modify or branch from
> anything prior to 4.0.  The tarballs are good enough for historical
> reference.  I just don't see why anyone would care about the branch
> structure of 3.4/3.5/3.7/3.8 now that it's 20 years done and over.

I didn't suggest any effort to remove tags etc.  Would have been fine 
with historical tags and whatever synthetic branches are added.  What 
has mattered is recalling something from CVS and from the repository and 
have them match.

Dan

Re: Let's clean up the recent stuff and hide old messy stuff under a rug

[Eric S. R. <es...@th...>]

sfeam <sf...@us...>:
> On Sunday, 29 October 2017 14:01:15 Daniel J Sebald wrote:
> > I've experimented with something called cvs2git, which I believe does a 
> > pretty good job of constructing branches and merges, but has some 
> > drawbacks. 
> 
> I think we are past the stage of looking for alternative conversion utilities.
> 
> Can't we just flatten everything prior to version 4.0 into a baseline
> snapshot that is the root for everything subsequent to that?

We don't have to be quite that drastic.  In particular, we can keep the
archival releases you wanted glued to the tail without causing further
difficulties.

I do think we have readhed the point where it makes sense to just drop
the 3.7.x branch and move forward.

> I really don't see why anyone will ever need to modify or branch from
> anything prior to 4.0.  The tarballs are good enough for historical 
> reference.  I just don't see why anyone would care about the branch
> structure of 3.4/3.5/3.7/3.8 now that it's 20 years done and over.
> 
> And if someone _does_ want the historical information for some reason
> other than development going forward, they would do better to go back to
> the archived cvs repository itself rather than having to deal with any
> post-processing layers you might add now to make the cvs->git conversion
> work.

I concur.

> I suggest that at this point the only issue worth spending more time
> on is authorship attributions transferred from the ChangeLog series.
> Once that is sorted out, let's call it a wrap.

Well, filling in at least the wore recent empty log messages would be
good too.  But other than that, yes.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Roundup of GNUPLOT conversion issues

[Hans-Bernhard B. <HBB...@t-...>]

Am 29.10.2017 um 20:31 schrieb Daniel J Sebald:

> I don't think the exact tarballs are there.  I sort of have a 
> recollection of whenever a release was to be made, the tarball required 
> some extra effort on someone's part.

Not back then they didn't.  The tarballs can be generated directly from 
a fresh working copy, just by running

	./configure && make dist

But all that effectively does is make a version-named copy of the source 
tree without "CVS" directories and .cvsignore files, and tarball that.

Any actual differences between that and the official release tarball are 
down to either

a) auto-tools of different versions being re-run at 'make' time, or
b) last-minute fixes applied directly to the tarball, manually

Only when we switched to automake, dropped all generated files from the 
repository, and started using the 'prepare' script instead, did tarball 
contents start to be really different from fresh working copies of their 
corresponding CVS tags.

Short vectors are not plotted

[Dima K. <gn...@di...>]

Hi all, a non-version-control question for yall!

I'm trying to plot a vector field (attached). The vectors start at an
evenly-spaced 20x20 grid. On my machine when I run the script, plotting
into an x11 terminal, I don't see all 400 vectors, I see maybe 1/4 of
them; the rest don't show up at all. It turns out what happens is this:

  do_plot()
  {
    ...
    plot_vectors();
    ...
  }

  plot_vectors()
  {
    ...
    for(vectors)
    {
      double x1_data_coords, x2_data_coords, ...;
      ...
      int x1_terminal_coords = map_x(x1_data_coords);
      int x2_terminal_coords = map_x(x2_data_coords);
      ...
      (terminal->arrow) (x1_terminal_coords, y1_terminal_coords, ...);
    }
  }

  arrow(int x1, int y1, ...)
  {
     if( x1 == x2 && y1 == y2 )
       return;

     ...
  }


I.e. we convert the vector start/end to integer terminal coords; short
vectors become 0-vectors in this representation, so the
terminal-specific arrow() function plots nothing.

This is a bug: checking for a 0-vector should happen before we convert
to integer pixel coordinates.

I'm attaching a patch series to fix this. The fundamental difference is
a change to the arrow() terminal callback to take FLOATING-POINT
terminal coordinates, so the vector direction, length can still be
accurately computed. This feels incomplete, since I suspect there're
other terminal functions that can benefit from such a change, but this
works.

Note that these patches are made with 'git format-patch', so once the
conversion is complete, they should be applied with 'git am'. This
imports the patch series together with the commit metadata (author,
message, etc).

Let's clean up the recent stuff and hide old messy stuff under a rug

[sfeam <sf...@us...>]

On Sunday, 29 October 2017 14:01:15 Daniel J Sebald wrote:
> I've experimented with something called cvs2git, which I believe does a 
> pretty good job of constructing branches and merges, but has some 
> drawbacks. 

I think we are past the stage of looking for alternative conversion utilities.

Can't we just flatten everything prior to version 4.0 into a baseline
snapshot that is the root for everything subsequent to that?

I really don't see why anyone will ever need to modify or branch from
anything prior to 4.0.  The tarballs are good enough for historical 
reference.  I just don't see why anyone would care about the branch
structure of 3.4/3.5/3.7/3.8 now that it's 20 years done and over.

And if someone _does_ want the historical information for some reason
other than development going forward, they would do better to go back to
the archived cvs repository itself rather than having to deal with any
post-processing layers you might add now to make the cvs->git conversion
work.

I suggest that at this point the only issue worth spending more time
on is authorship attributions transferred from the ChangeLog series.
Once that is sorted out, let's call it a wrap.

	Ethan


> All I did was follow the instructions here: 
> http://cvs2svn.tigris.org/cvs2git.html 
> with the exception I had to create a link cvs2git-tmp to cvs2svn-tmp 
> because the program (based off cvs2svn) was erroring out when it 
> couldn't find the blob and dump files in a non-existent cvs2git-tmp 
> directory.  (Surely a bug in the version I have, maybe not present in 
> more recent releases.)  cvs2svn takes about forty five minutes to run 
> (creates a mondo blob file), so have something else to work on in the 
> meantime.
> 
> It's major drawbacks are:
> 
> 1) No corrected author info.
> 
> 2) All of the RCSid strings are replaced by empty string, e.g.,
> 
> diff -ur '--exclude=.git' /home/sebald/gnuplot/gnuplot/gnuplot/time.c 
> gnuplot/time.c
> --- /home/sebald/gnuplot/gnuplot/gnuplot/time.c	2017-10-28 
> 22:16:04.820853058 -0500
> +++ gnuplot/time.c	2017-10-28 22:18:35.340854531 -0500
> @@ -1,5 +1,5 @@
>   #ifndef lint
> -static char *RCSid = "$Id: time.c,v 1.5 1998/12/09 15:25:54 lhecking 
> Exp $";
> +static char *RCSid = "$Id$";
>   #endif
> 
> which makes it even worse than the differences I'm now seeing.
> 
> 3) It seems to get the tags at the start of the repository wrong, even 
> though the branches look correct.
> 
> 
> At least cvs2git is a good guide for branching, and I'm wondering if we 
> can use reposurgeon on the cvs-fast-export result to make the branching 
> match cvs2git.  (I did this last night.)
> 
> In contrast, I would summarize cvs-fast-export's deficiencies as follows:
> 
> 1) I think cvs-fast-export is getting hung up on the scenario by which 
> the "Initial import of beta340." branch is the exact same time as the 
> "Initial revision".  That is, all the files in the "Initial import of 
> beta340." group match "Initial revision" as follows (same time, and zero 
> lines difference):
> 
> ----------------------------
> revision 1.1
> date: 1998/04/15 19:16:47;  author: lhecking;  state: Exp;
> branches:  1.1.1;
> Initial revision
> ----------------------------
> revision 1.1.1.1
> date: 1998/04/15 19:16:47;  author: lhecking;  state: Exp;  lines: +0 -0
> Initial import of beta340.
> ----------------------------
> 
> I'm not quite sure how Lars managed that.  Nonetheless, the conversion 
> program has to pick a name to use here.  As does cvs2git, 
> cvs-fast-export picks "Initial import of beta340.".  But cvs2git 
> recognizes this is a branch (still no files) and then does "Import of 
> beta 343" (containing files) and merges back into the master branch with 
> a comment such as
> 
> "
> This commit was generated by cvs2svn to compensate for changes in r5, 
> which included commits to RCS files with non-trunk default branches.
> "
> 
> It does the same thing with "Import of beta 344.", "Import of beta 
> 345.", "Import of beta 346." and then later something from "Initial 
> import of beta340."  So, yes, it's complex and it is best to have the 
> cvs2git result handy to understand this, but it makes clear the system 
> Lars was using as described by HBB, i.e., vendor branches.  I'm 
> wondering if we can just use reposurgeon to reconstruct that series of 
> branches and merges.
> 
> What cvs-fast-export does do for those tricky utility-branches is put in 
> the extra *** empty log message *** probably because of the time span of 
> the cvs initial checkins.  (Recall, 1999 computers were much slower than 
> today, so the file time stamps are spread out over 30 seconds.  We need 
> a wide fuzz-factor and if Lars did some action immediately after that 
> first CVS creation command, something could get pulled into the first 
> collection.)  Anyway, somehow the *** empty log message *** appears to 
> adjust things, and for cvs-fast-export we get:
> 
> *** empty log message ***         (no files)
> Initial import of beta340.        (no files)
> *** empty log message ***         (has files)
> Initial import of beta340.        (deletes some of those files)
> Synthetic commit for tag BETA_340 (puts those files back in)
> Import of beta 343.               (mostly modifications)
> Synthetic commit for tag BETA_343 (adds files back in)
> *** empty log message ***         (master branch start, has files)
> Include stdfn.h.                  (adds just one file)
> 
> In contrast cvs2git has the following first changesets:
> 
> Initial import of beta340.        (no files)
> Import of beta 343.               (has files)
> This commit was manufactured by cvs2svn to create branch 'GNUPLOT_BETA'.
>                                    (has two files)
> Import of beta 344.               (all modifications)
> Import of beta 345.               (all modifications)
> Import of beta 346.               (all modifications)
> Import of beta 347.               (all modifications)
> This commit was generated by cvs2svn to compensate for changes in r2, 
> which included commits to RCS files with non-trunk default branches.
>                                    (no files)
> This commit was generated by cvs2svn to compensate for changes in r5, 
> which included commits to RCS files with non-trunk default branches.
>                                    (merge files into master branch)
> Include stdfn.h.                  (adds just one file)
> 
> The above cvs2git arrangement looks a lot like Eric's original 
> repository result, except that result was missing the merge back into 
> master branch.
> 
> So, the ./reconvert file.  I'm going to email Eric a diff file of the 
> changes I've made.
> 
> 1) I've restored all the tags and branches, at least for the time being, 
> so that I could compare cvs-fast-export to cvs2git with greater detail. 
> Honestly, my preference would be to keep all tags, and I'd prefer that 
> HBB and Ethan (et al.) have a chance to see the full repository in a git 
> viewer first (I recommend qgit for this) before deciding to cut things 
> out.  I like having the tags GNUPLOT_RELEASE_3_7_0, etc. and maybe 
> instead we could just add additional tags 3.7.0, etc. alongside the 
> original.
> 
> HBB and Ethan should be able to run Eric's ./reconvert script to 
> generate the repository (it's just one command), so there's no need to 
> post any new repository to SourceForge.  (And creating the cvs2git 
> version isn't too difficult either.  Just use as an input to cvs2git the 
> gnuplot-cvs directory created by Eric's ./reconvert script.)
> 
> 2) I used cvs2git's branch points in directives for reposurgeon.  (Yes, 
> stuff like
> 
> /Initial import of 3.6 beta340./ assign root37
> @min(/New file.$/),<root37> reparent --use-order --rebase
> 
> (My learning curve was understanding various syntax in order to create 
> singletons.)
> 
> 
> After having done the above, I find that tree-wise the two conversions 
> look similar, except for the first dozen changesets.  cvs2git has more 
> logical file changes at the rebased branch point: the diffs look 
> correct, but I suspect the difference at the start of the repo result in 
> other files being swept in.
> 
> Tag-wise, again very similar ("cvs2git created this branch for a tag", 
> etc. coincides well with "cvs-fast-export created this branch for a 
> tag").  However, I think cvs-fast-export is a little better at this at 
> the start of the repo than cvs2git.
> 
> And exact file matches between the two conversions is difficult because 
> cvs2git drops all the RCSid strings.  (We'll make sure all those things 
> are gone from the start of the new repo, which Eric may have already done.)
> 
> 
> 3) I commented out the pre-CVS archival changesets for the time being. 
> I think we should just concentrate on that initial dozen changesets (try 
> to match cvs2git more, I couldn't figure out how to add merges via 
> reposurgeon), and if we get those cleared up my hope is that it works 
> its way through the whole tree and fixes up the branches without the 
> need of shims.
> 
> Dan
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

Re: The first repository branch, "Windows linestyle fix."

[Hans-Bernhard B. <HBB...@t-...>]

Am 29.10.2017 um 20:01 schrieb Daniel J Sebald:
> I've experimented with something called cvs2git, which I believe does a 
> pretty good job of constructing branches and merges, but has some 
> drawbacks.  
[...]
> It's major drawbacks are:
> 
> 1) No corrected author info.
> 
> 2) All of the RCSid strings are replaced by empty string, e.g.,

That's not a drawback; it's a necessary consequence of moving to git. 
One way or another, all these strings (technically: RCS-style keywords) 
have to go.

For comparisons of working copies, I would therefore strongly recommend 
adding an appropriate ignore option to 'diff', such as

	diff -ur '--exclude=.git' -I '\$[A-Z]*[a-z]*'

to avoid getting side-tracked.

> In contrast, I would summarize cvs-fast-export's deficiencies as follows:
> 
> 1) I think cvs-fast-export is getting hung up on the scenario by which 
> the "Initial import of beta340." branch is the exact same time as the 
> "Initial revision".  

> I'm not quite sure how Lars managed that.  

Lars just used the feature of CVS advertised for the job he was doing at 
the time: "cvs import".  This creates a vendor branch, which behaves 
rather differently from ordinary branches.  One side effect of that is 
that the 1.1 revisions ("Initial revision") of all files that existed at 
the time have another revision at the exact same time, numbered 1.1.1.1, 
on the vendor branch.

These 1.1 revisions really only exist as an anchor point for the vendor 
branch --- they never appear in working copies.  You can still see that 
today: files that already existed back then, and never changed since, 
still have revision 1.1.1.1:

$ cvs status demo/world.cor
===================================================================
File: world.cor         Status: Up-to-date

    Working revision:    1.1.1.1
    Repository revision: 1.1.1.1 /cvsroot/gnuplot/gnuplot/demo/world.cor,v
    Sticky Tag:          (none)
    Sticky Date:         (none)
    Sticky Options:      (none)

One weird aspect of vendor branches is that the first modification of 
such a file will not get revision number 1.1.1.2, as might be expected, 
but rather 1.2.  That's because revision 1.1.1.2 is reserved for the 
next "cvs import".

Re: Roundup of GNUPLOT conversion issues

[Daniel J S. <dan...@ie...>]

On 10/29/2017 11:05 AM, Eric S. Raymond wrote:
> Hans-Bernhard Bröker <HBB...@t-...>:
>> Am 29.10.2017 um 13:45 schrieb Eric S. Raymond:
>>> Daniel J Sebald <dan...@ie...>:
>>>
>>> Does cvs2git get the tip content of 3.7.x right - that is, coincident with the
>>> 3.7.3 tarball?
>>
>> Please note that the tip of the 3.7.* branch is not coincident with the
>> 3.7.3 tag or tarball.  Some further changes were made on the branch after
>> 3.7.3.
> 
> I do not see any in gitspace. How would I retrieve or list them in CVS?
> 
>> The same goes for all of our release branches.  They all received a couple
>> more changes after the last release was made from them.  So none of the
>> branch tips coincides to any released tarball.  Comparisons really would
>> have to be made to cvs checkouts of the branches, instead.
> 
> Which just brings up the more general question: how do I see these in CVS?

I don't think the exact tarballs are there.  I sort of have a 
recollection of whenever a release was to be made, the tarball required 
some extra effort on someone's part.


>>>>> 1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
>>>>> the archived 3.7.[0123] trees in sequence (I now have them all).
>>
>> That would be slightly wrong because 3.7.0 is not on the branch.  It's on
>> the trunk.
> 
> OK. No one has told me where yet, though.

cvs-fast-export seems to have gotten it right.  This is why I restored 
tags/branches in ./reconvert, to get greater detail.

Dan

Re: Roundup of GNUPLOT conversion issues

[Daniel J S. <dan...@ie...>]

On 10/29/2017 10:20 AM, Hans-Bernhard Bröker wrote:
> Am 29.10.2017 um 13:45 schrieb Eric S. Raymond:
>> Daniel J Sebald <dan...@ie...>:
>>
>> Does cvs2git get the tip content of 3.7.x right - that is, coincident 
>> with the
>> 3.7.3 tarball?
> 
> Please note that the tip of the 3.7.* branch is not coincident with the 
> 3.7.3 tag or tarball.  Some further changes were made on the branch 
> after 3.7.3.
> 
> The same goes for all of our release branches.  They all received a 
> couple more changes after the last release was made from them.  So none 
> of the branch tips coincides to any released tarball.  Comparisons 
> really would have to be made to cvs checkouts of the branches, instead.

Correct.  I'm not sure, but it may be that the tarball doesn't exactly 
match CVS either.  It could have been that Lars tweaked some files on 
his system just prior to making the tarball.

In any case, its a very easy matter in git to checkout the converted 
release version as a new branch, wipe all the files in the directory 
(except .git directory of course!), unpack the tarball, press the 
"reload" button of qgit, gitg, or git-gui, write a commit message and 
create a commit... and there you have "GNUPLOT_RELESE_X_Y_Z-tarball" as 
a little stub branch.  The diffs are relatively small so won't take much 
size in the repository.


>>>> 1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
>>>> the archived 3.7.[0123] trees in sequence (I now have them all).
> 
> That would be slightly wrong because 3.7.0 is not on the branch.  It's 
> on the trunk.

This is correct.  GNUPLOT_RELEASE_3_7_0 is on the trunk while the other 
GNUPLOT_RELEASE_3_7_x are on the branch.  I'm attaching screenshots of 
the cvs2git tree and the cvs-fast-export tree after I've made the rebase 
with reposurgeon.  The GNUPLOT_RELEASE_3_7_0 synthetic commit was done 
by cvs-fast-export, not reposurgeon.  So, cvs-fast-export got the 
tag/branch correct but didn't realize that's where the pertinent branch 
should go.

Dan

Re: The first repository branch, "Windows linestyle fix."

[Daniel J S. <dan...@ie...>]

I've experimented with something called cvs2git, which I believe does a 
pretty good job of constructing branches and merges, but has some 
drawbacks.  All I did was follow the instructions here:

http://cvs2svn.tigris.org/cvs2git.html

with the exception I had to create a link cvs2git-tmp to cvs2svn-tmp 
because the program (based off cvs2svn) was erroring out when it 
couldn't find the blob and dump files in a non-existent cvs2git-tmp 
directory.  (Surely a bug in the version I have, maybe not present in 
more recent releases.)  cvs2svn takes about forty five minutes to run 
(creates a mondo blob file), so have something else to work on in the 
meantime.

It's major drawbacks are:

1) No corrected author info.

2) All of the RCSid strings are replaced by empty string, e.g.,

diff -ur '--exclude=.git' /home/sebald/gnuplot/gnuplot/gnuplot/time.c 
gnuplot/time.c
--- /home/sebald/gnuplot/gnuplot/gnuplot/time.c	2017-10-28 
22:16:04.820853058 -0500
+++ gnuplot/time.c	2017-10-28 22:18:35.340854531 -0500
@@ -1,5 +1,5 @@
  #ifndef lint
-static char *RCSid = "$Id: time.c,v 1.5 1998/12/09 15:25:54 lhecking 
Exp $";
+static char *RCSid = "$Id$";
  #endif

which makes it even worse than the differences I'm now seeing.

3) It seems to get the tags at the start of the repository wrong, even 
though the branches look correct.


At least cvs2git is a good guide for branching, and I'm wondering if we 
can use reposurgeon on the cvs-fast-export result to make the branching 
match cvs2git.  (I did this last night.)

In contrast, I would summarize cvs-fast-export's deficiencies as follows:

1) I think cvs-fast-export is getting hung up on the scenario by which 
the "Initial import of beta340." branch is the exact same time as the 
"Initial revision".  That is, all the files in the "Initial import of 
beta340." group match "Initial revision" as follows (same time, and zero 
lines difference):

----------------------------
revision 1.1
date: 1998/04/15 19:16:47;  author: lhecking;  state: Exp;
branches:  1.1.1;
Initial revision
----------------------------
revision 1.1.1.1
date: 1998/04/15 19:16:47;  author: lhecking;  state: Exp;  lines: +0 -0
Initial import of beta340.
----------------------------

I'm not quite sure how Lars managed that.  Nonetheless, the conversion 
program has to pick a name to use here.  As does cvs2git, 
cvs-fast-export picks "Initial import of beta340.".  But cvs2git 
recognizes this is a branch (still no files) and then does "Import of 
beta 343" (containing files) and merges back into the master branch with 
a comment such as

"
This commit was generated by cvs2svn to compensate for changes in r5, 
which included commits to RCS files with non-trunk default branches.
"

It does the same thing with "Import of beta 344.", "Import of beta 
345.", "Import of beta 346." and then later something from "Initial 
import of beta340."  So, yes, it's complex and it is best to have the 
cvs2git result handy to understand this, but it makes clear the system 
Lars was using as described by HBB, i.e., vendor branches.  I'm 
wondering if we can just use reposurgeon to reconstruct that series of 
branches and merges.

What cvs-fast-export does do for those tricky utility-branches is put in 
the extra *** empty log message *** probably because of the time span of 
the cvs initial checkins.  (Recall, 1999 computers were much slower than 
today, so the file time stamps are spread out over 30 seconds.  We need 
a wide fuzz-factor and if Lars did some action immediately after that 
first CVS creation command, something could get pulled into the first 
collection.)  Anyway, somehow the *** empty log message *** appears to 
adjust things, and for cvs-fast-export we get:

*** empty log message ***         (no files)
Initial import of beta340.        (no files)
*** empty log message ***         (has files)
Initial import of beta340.        (deletes some of those files)
Synthetic commit for tag BETA_340 (puts those files back in)
Import of beta 343.               (mostly modifications)
Synthetic commit for tag BETA_343 (adds files back in)
*** empty log message ***         (master branch start, has files)
Include stdfn.h.                  (adds just one file)

In contrast cvs2git has the following first changesets:

Initial import of beta340.        (no files)
Import of beta 343.               (has files)
This commit was manufactured by cvs2svn to create branch 'GNUPLOT_BETA'.
                                   (has two files)
Import of beta 344.               (all modifications)
Import of beta 345.               (all modifications)
Import of beta 346.               (all modifications)
Import of beta 347.               (all modifications)
This commit was generated by cvs2svn to compensate for changes in r2, 
which included commits to RCS files with non-trunk default branches.
                                   (no files)
This commit was generated by cvs2svn to compensate for changes in r5, 
which included commits to RCS files with non-trunk default branches.
                                   (merge files into master branch)
Include stdfn.h.                  (adds just one file)

The above cvs2git arrangement looks a lot like Eric's original 
repository result, except that result was missing the merge back into 
master branch.

So, the ./reconvert file.  I'm going to email Eric a diff file of the 
changes I've made.

1) I've restored all the tags and branches, at least for the time being, 
so that I could compare cvs-fast-export to cvs2git with greater detail. 
Honestly, my preference would be to keep all tags, and I'd prefer that 
HBB and Ethan (et al.) have a chance to see the full repository in a git 
viewer first (I recommend qgit for this) before deciding to cut things 
out.  I like having the tags GNUPLOT_RELEASE_3_7_0, etc. and maybe 
instead we could just add additional tags 3.7.0, etc. alongside the 
original.

HBB and Ethan should be able to run Eric's ./reconvert script to 
generate the repository (it's just one command), so there's no need to 
post any new repository to SourceForge.  (And creating the cvs2git 
version isn't too difficult either.  Just use as an input to cvs2git the 
gnuplot-cvs directory created by Eric's ./reconvert script.)

2) I used cvs2git's branch points in directives for reposurgeon.  (Yes, 
stuff like

/Initial import of 3.6 beta340./ assign root37
@min(/New file.$/),<root37> reparent --use-order --rebase

(My learning curve was understanding various syntax in order to create 
singletons.)


After having done the above, I find that tree-wise the two conversions 
look similar, except for the first dozen changesets.  cvs2git has more 
logical file changes at the rebased branch point: the diffs look 
correct, but I suspect the difference at the start of the repo result in 
other files being swept in.

Tag-wise, again very similar ("cvs2git created this branch for a tag", 
etc. coincides well with "cvs-fast-export created this branch for a 
tag").  However, I think cvs-fast-export is a little better at this at 
the start of the repo than cvs2git.

And exact file matches between the two conversions is difficult because 
cvs2git drops all the RCSid strings.  (We'll make sure all those things 
are gone from the start of the new repo, which Eric may have already done.)


3) I commented out the pre-CVS archival changesets for the time being. 
I think we should just concentrate on that initial dozen changesets (try 
to match cvs2git more, I couldn't figure out how to add merges via 
reposurgeon), and if we get those cleared up my hope is that it works 
its way through the whole tree and fixes up the branches without the 
need of shims.

Dan

Re: Roundup of GNUPLOT conversion issues

[Hans-Bernhard B. <HBB...@t-...>]

Am 29.10.2017 um 17:05 schrieb Eric S. Raymond:
> Hans-Bernhard Bröker <HBB...@t-...>:
>> Am 29.10.2017 um 13:45 schrieb Eric S. Raymond:
>>> Daniel J Sebald <dan...@ie...>:
>>>
>>> Does cvs2git get the tip content of 3.7.x right - that is, coincident with the
>>> 3.7.3 tarball?
>>
>> Please note that the tip of the 3.7.* branch is not coincident with the
>> 3.7.3 tag or tarball.  Some further changes were made on the branch after
>> 3.7.3.
> 
> I do not see any in gitspace. How would I retrieve or list them in CVS?

The pedestrian way would be to just check out a working copy on the 
branch, and one of the last release tag from the same branch, and 
compare those.  In a generated reposurgeon Makefile setup:

   make gnuplot-{Release_5_0_7,branch-5-0-stable}-checkout
   diff -u gnuplot-{Release_5_0_7,branch-5-0-stable}-checkout/ChangeLog

Or one can look at a "cvs log ChangeLog" or the ChangeLog,v archive file 
itself, and check if there are revisions 1.{N}.2.{M} with N the same as 
that of the last release tag, and M bigger than that of the same tag.

>>>>> 1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
>>>>> the archived 3.7.[0123] trees in sequence (I now have them all).
>>
>> That would be slightly wrong because 3.7.0 is not on the branch.  It's on
>> the trunk.
> 
> OK. No one has told me where yet, though.

The GNUPLOT_RELEASE_3_7_0 tag is genuine.

Re: Roundup of GNUPLOT conversion issues

[Eric S. R. <es...@th...>]

Hans-Bernhard Bröker <HBB...@t-...>:
> Am 29.10.2017 um 13:45 schrieb Eric S. Raymond:
> >Daniel J Sebald <dan...@ie...>:
> >
> >Does cvs2git get the tip content of 3.7.x right - that is, coincident with the
> >3.7.3 tarball?
> 
> Please note that the tip of the 3.7.* branch is not coincident with the
> 3.7.3 tag or tarball.  Some further changes were made on the branch after
> 3.7.3.

I do not see any in gitspace. How would I retrieve or list them in CVS? 

> The same goes for all of our release branches.  They all received a couple
> more changes after the last release was made from them.  So none of the
> branch tips coincides to any released tarball.  Comparisons really would
> have to be made to cvs checkouts of the branches, instead.

Which just brings up the more general question: how do I see these in CVS?

> >>>1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
> >>>the archived 3.7.[0123] trees in sequence (I now have them all).
> 
> That would be slightly wrong because 3.7.0 is not on the branch.  It's on
> the trunk.

OK. No one has told me where yet, though.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Roundup of GNUPLOT conversion issues

[Hans-Bernhard B. <HBB...@t-...>]

Am 29.10.2017 um 13:45 schrieb Eric S. Raymond:
> Daniel J Sebald <dan...@ie...>:
> 
> Does cvs2git get the tip content of 3.7.x right - that is, coincident with the
> 3.7.3 tarball?

Please note that the tip of the 3.7.* branch is not coincident with the 
3.7.3 tag or tarball.  Some further changes were made on the branch 
after 3.7.3.

The same goes for all of our release branches.  They all received a 
couple more changes after the last release was made from them.  So none 
of the branch tips coincides to any released tarball.  Comparisons 
really would have to be made to cvs checkouts of the branches, instead.

>>> 1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
>>> the archived 3.7.[0123] trees in sequence (I now have them all).

That would be slightly wrong because 3.7.0 is not on the branch.  It's 
on the trunk.

Re: Roundup of GNUPLOT conversion issues

[Eric S. R. <es...@th...>]

Daniel J Sebald <dan...@ie...>:
> I'm interested in retaining history.  Why shape it into a different history
> that no one is familiar with?

Seems to me like the release tarballs are the best-known history there is.

> >We wouldn't even *want* that kind of 'perfection' - the old-style
> >change comments wouldn't play well with git log, for one thing.
> 
> Projects like Octave use FSF format
> 
> http://hg.savannah.gnu.org/hgweb/octave/rev/1680d425bb38
> 
> and it produces a nice log which highlights with color in the pager.

That isn't actually FSF format at all.  It adopts two practices the
FSF never had: (1) having a headline separated by a blank line from the
body of the comment, and (2) restricting the length of that line so it
will fit in an 80-column window with 4 leading spaces to spare.

What I sean by "old-style" is what was commonly done in CVS and
Subversion.  That is, paragraphs of running text *without* a separated
headline (and often with no internal line breaks). Those don't play
well with git.

> >It's corrupt in the sense that because of the vendor-branch confusion we
> >not only can't guarantee that the gitspace sequence of commits really
> >matches the CVS development history, we can't even get the right tip
> >state out of the conversion.
> 
> cvs2git seems to handle the vendor-branch via a branch-and-merge approach.
> (It only goes for a year or so, right?)  We'll come back to this, as I think
> it is the missing piece we need to deal with still.

Does cvs2git get the tip content of 3.7.x right - that is, coincident with the
3.7.3 tarball?

> >1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
> >the archived 3.7.[0123] trees in sequence (I now have them all).
> 
> This is more work than just moving the branch to the correct location.
> cvs2git has chosen the branch point that I deduced by version/dates within
> the files, i.e., "New file."  I'll work this out.

I'm guessing you mean

<1998-04-11T19:43:11Z!lhe...@nm...>

which is the earliest commit with a "New file." comment. The commands you want
would probably be 

/Initial import of 3.6 beta340./ assign root37
@min(/New file.$/),<root37> reparent --use-order --rebase

except I suspect you want to sctually do this:

/Windows linestyle fix/ assign root37

and then lose the preceding reimports.

> >3. Fill in the empty log comments that are left.
> 
> We'll see if Ethan wants to do this.  He and I can split the work, but only
> if it is something easy to do with reposurgeon.

You two don't have to drive reposurgeon st all to do this.  Just fill in
comments in the enclosed file and send it back to me.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Roundup of GNUPLOT conversion issues

[Daniel J S. <dan...@ie...>]

On 10/29/2017 12:30 AM, Eric S. Raymond wrote:
> The latest GNUPLOT conversion-machinery tarball, available as usual at
> 
>      wget http://www.catb.org/~esr/gnuplot-conversion.tar.gz
> 
> cleans up a number of minor issues, including:
> 
> (1) Patching out CVS release numbers in comments.
> 
> (2) Better explanatory text and comments for the very early section of
> the repository.
> 
> (3) Patch that incorrect author date due to a ChangeLog typo.
> 
> I shall round up my responses to all recent traffic and then propose
> what to do next.

I've been making some ./reconvert mods, supplying all the branch points 
as determined with cvs2git.  However, the branches at the start of the 
repository for importing beta340, etc. are going to need some work on 
your part.

I'll describe in a follow-up email.  But wait for me to get a diff file 
to you and we can put my mods in place.


> Bastian Märkisch:
>> Unfortunately, the last problem we discussed is not solved, though:
>> If there's an insertion into the ChangeLog below an existing author line,
>> the current algorithm picks the author line _below_ instead of the one
>> _above_. This happens quite frequently and currently still leads to hundreds
>> of wrong attributions.
>> For examples try:
>>   git log --author=Ethan --committer=Bastian --oneline | wc -l
> 
> I still need an example pair of Changelogs with a difference that
> triggers this bug. Given that, I can (a) fix the bug, and (b) add it
> to my regression tedt so we can be certain it does not recur.
> 
> Daniel Sebald, Fri Oct 27 12:50:51 2017
>>                                                    These files with the
>> wrong RCSid and the demo-file discrepancies and the backward-progressing
>> copyright dates reside in the master branch *until* some changeset resolves
>> them.  In the case of 3.7.x branch, there are a hundred "corrupted" files
>> still, but shortly after 3.7.0 was completed there was a mass restructuring
>> which moved files like alloc.c to a new subdirectory.  That action resulted in
>> the correct files going forward, such that 4.x and 5.x series all match
>> well...except for the 4.0 shim that Eric noted, probably a vestige still of
>> what I described.
> 
>> So, we need to go back to the start of the repository and make sure those
>> first few commits, the "Import of beta 340", etc. are done properly such that
>> something like the following set of files all match in terms of stamped
>> revisions and branches:
> 
>> How should we proceed Eric?
> 
> What I'm hearing is, exact for the demo shim I added, all the remaining
> differences early in branches other than 3.7.x are incorrect RCS IDs.
> 
> What I think is that trying to get the RCS IDs perfect would be wasted
> effort.  One of my last patches removes all of those, anyway.  It's
> time to remember that the purpose of the repository transition is to
> support development going forward, not to have a theoretically perfect
> and literal transcription of the CVS history.

I'm interested in retaining history.  Why shape it into a different 
history that no one is familiar with?  I'm fine with the RCS IDs not 
matching--Ethan (with my help), can branch a stub off of a release with 
bad RCS IDs and then put in the actual release code.  That will then 
produce the actual release code in the repository--for the price of just 
a few diffs.


> We wouldn't even *want* that kind of 'perfection' - the old-style
> change comments wouldn't play well with git log, for one thing.

Projects like Octave use FSF format

http://hg.savannah.gnu.org/hgweb/octave/rev/1680d425bb38

and it produces a nice log which highlights with color in the pager.


> Hans-Bernhard Bröker <HBB...@t-...>:
>>> On the other hand, I'm not getting
>>> a good feeling from Dan's reports on his attempts to disentangle that branch.
>>> We know it's corrupt - doesn't have the right tip state.
>>
>> I'm not quite convinced Dan's analysis is correct.  The branch isn't
>> corrupt, it was just set up somewhat strangely at the time.
> 
> It's corrupt in the sense that because of the vendor-branch confusion we
> not only can't guarantee that the gitspace sequence of commits really
> matches the CVS development history, we can't even get the right tip
> state out of the conversion.

cvs2git seems to handle the vendor-branch via a branch-and-merge 
approach.  (It only goes for a year or so, right?)  We'll come back to 
this, as I think it is the missing piece we need to deal with still.


> The extent to which this is a CVS problem versus a git problem versus
> a cvs-fast-export problem is really only of theoretical interest.
> 
> I've spent three weeks on this conversion and my patience for
> wandering further off into the weeds is evaporating.  It only
> lasted this long because getting ChangeLog analysis right seemed
> like a thing worth doing.

Yes, ChangeLog analysis is good.


> Can we please refocus on *finishing* this with changes that actually add
> value for forward development?

The only major thing still is that first dozen or so commits at the 
start.  I'm not sure, but that may be cvs-fast-export dependent. 
Otherwise, I'm getting the hang of reposurgeon and ./reconvert, so I can 
deal with verifying correctness of branches and rebuilding.


>> Vendor branches with other branches partially rooted in them cannot be
>> collapsed or ignored.  Nor can they be treated completely like ordinary
>> branches (or conversion tools would have been doing that since day one).
> 
> I know.  Vendor branches have been a huge pain in the ass on other
> conversions I've done, too.
> 
> Here is what I suggest we do:
> 
> 1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
> the archived 3.7.[0123] trees in sequence (I now have them all).

This is more work than just moving the branch to the correct location. 
cvs2git has chosen the branch point that I deduced by version/dates 
within the files, i.e., "New file."  I'll work this out.


> 2. Fix any remaining deficiences in ChangeLog scanning if possible.
> 
> 3. Fill in the empty log comments that are left.

We'll see if Ethan wants to do this.  He and I can split the work, but 
only if it is something easy to do with reposurgeon.

Dan

Roundup of GNUPLOT conversion issues

[<es...@th...>]

The latest GNUPLOT conversion-machinery tarball, available as usual at

    wget http://www.catb.org/~esr/gnuplot-conversion.tar.gz

cleans up a number of minor issues, including:

(1) Patching out CVS release numbers in comments.

(2) Better explanatory text and comments for the very early section of
the repository.

(3) Patch that incorrect author date due to a ChangeLog typo.

I shall round up my responses to all recent traffic and then propose
what to do next.

Bastian Märkisch:
>Unfortunately, the last problem we discussed is not solved, though:
>If there's an insertion into the ChangeLog below an existing author line,
>the current algorithm picks the author line _below_ instead of the one
>_above_. This happens quite frequently and currently still leads to hundreds
>of wrong attributions.
>For examples try:
>  git log --author=Ethan --committer=Bastian --oneline | wc -l

I still need an example pair of Changelogs with a difference that
triggers this bug. Given that, I can (a) fix the bug, and (b) add it
to my regression tedt so we can be certain it does not recur.

Daniel Sebald, Fri Oct 27 12:50:51 2017
>                                                   These files with the
>wrong RCSid and the demo-file discrepancies and the backward-progressing
>copyright dates reside in the master branch *until* some changeset resolves
>them.  In the case of 3.7.x branch, there are a hundred "corrupted" files
>still, but shortly after 3.7.0 was completed there was a mass restructuring
>which moved files like alloc.c to a new subdirectory.  That action resulted in
>the correct files going forward, such that 4.x and 5.x series all match
>well...except for the 4.0 shim that Eric noted, probably a vestige still of
>what I described.

>So, we need to go back to the start of the repository and make sure those
>first few commits, the "Import of beta 340", etc. are done properly such that
>something like the following set of files all match in terms of stamped
>revisions and branches:

>How should we proceed Eric?

What I'm hearing is, exact for the demo shim I added, all the remaining
differences early in branches other than 3.7.x are incorrect RCS IDs.

What I think is that trying to get the RCS IDs perfect would be wasted
effort.  One of my last patches removes all of those, anyway.  It's
time to remember that the purpose of the repository transition is to
support development going forward, not to have a theoretically perfect
and literal transcription of the CVS history.

We wouldn't even *want* that kind of 'perfection' - the old-style
change comments wouldn't play well with git log, for one thing.

Daniel J Sebald, Sat Oct 28 14:48:42 2017
> I've dug into the actual ./reconvert process a bit.  I'm bewildered by some
> fundamental things regarding the CVS repository on
> 
> :pserver:ano...@gn...:/cvsroot/gnuplot
> 
> I assume that this repository is the same that is used to get the CVS copy
> used in the ./reconvert process
> 
> cvssync -c gnuplot.cvs.sourceforge.net:/cvsroot/gnuplot gnuplot
> mv gnuplot gnuplot-cvs
> 
> Those are the same repository, correct?

Yes.

> OK, I've been comparing the gnuplot-git repository against "cvs update
> -rGNUPLOT_3_7_0" and so on.  As I pointed out I've often gotten good matches
> in terms of code mods, with the exception being all these RCSid strings.

There is, however, that known problem with the 3.7.x tip revision, which has
non-RCSid differences between the tip and the tarball.

For reference, here is HBB's explanation.

    3.7.1 has one check-in to ChangeLog that's tagged but not tarred, and an extra
    pair of quotes around a line in configure.in that tarred but not tagged:

    diff -uwrp -x CVS cvs-3.7.1/ChangeLog gnuplot-3.7.1/ChangeLog
    --- cvs-3.7.1/ChangeLog 1999-11-01 13:33:32.000000000 +0100
    +++ gnuplot-3.7.1/ChangeLog     1999-10-21 16:15:07.000000000 +0200
    @@ -1,7 +1,3 @@
    -1999-11-01  Berthold Hoellmann  <ho...@ge...>
    -
    -       * docs/Makefile.in: Fix pdf target for compiling outside source dir.
    -
     1999-10-21  Lars Hecking  <lhe...@nm...>

	    * configure.in: Fix compile/link on NeXT without side effects ...
    diff -uwrp -x CVS cvs-3.7.1/configure gnuplot-3.7.1/configure
    --- cvs-3.7.1/configure 1999-10-27 13:00:15.000000000 +0200
    +++ gnuplot-3.7.1/configure     1999-11-07 16:57:22.000000000 +0100
    @@ -2591,7 +2591,7 @@ fi
     fi


    -if test $ac_cv_lib_nsl_gethostbyname = no; then
    +if test "$ac_cv_lib_nsl_gethostbyname" = no; then
       echo $ac_n "checking for gethostbyname in -lbsd""... $ac_c" 1>&6
     echo "configure:2597: checking for gethostbyname in -lbsd" >&5
     ac_lib_var=`echo bsd'_'gethostbyname | sed 'y%./+-%__p_%'`
    diff -uwrp -x CVS cvs-3.7.1/configure.in gnuplot-3.7.1/configure.in
    --- cvs-3.7.1/configure.in      1999-10-22 20:29:24.000000000 +0200
    +++ gnuplot-3.7.1/configure.in  1999-11-04 18:21:00.000000000 +0100
    @@ -115,7 +115,7 @@ AC_SUBST(GNUPLOT_X11)
     AC_PATH_XTRA

     dnl Needed for LynxOS until AC_PATH_XTRA is fixed
    -if test $ac_cv_lib_nsl_gethostbyname = no; then
    +if test "$ac_cv_lib_nsl_gethostbyname" = no; then
       AC_CHECK_LIB(bsd, gethostbyname, X_EXTRA_LIBS="$X_EXTRA_LIBS -lbsd")
     fi

    diff -uwrp -x CVS cvs-3.7.1/term/tgif.trm gnuplot-3.7.1/term/tgif.trm
    --- cvs-3.7.1/term/tgif.trm     1998-12-16 20:48:20.000000000 +0100
    +++ gnuplot-3.7.1/term/tgif.trm 1998-12-16 20:48:20.000000000 +0100
    @@ -371,7 +371,7 @@ TERM_PUBLIC void TGIF_init()
	 fprintf(gpoutfile, "\
     %%TGIF 2.15-p7\n\

    state(%d,30,%u,0,0,%u,16,1,9,1,1,0,0,0,0,1,0,'%s',0,%u,0,0,1,10,0,0,1,1,0,16,0,0,1,
    +1,1).\n\
    -%%\n%% @(#)$Header: /home/hbbro/prg/gp/backup/cvs/gnuplot/term/tgif.trm,v
    1.10 1998/12/16 19:48:20 lhecking Exp $\n%% %%W%%\n%%\n\
    +%%\n%% @(#)$Header: /var/tmp/CVSROOT/gnuplot/term/tgif.trm,v 1.10 1998/12/16
    19:48:20 lhecking Exp $\n%% %%W%%\n%%\n\
     page(1,\"\").\n",
		TgifPortrait ? 0 : 1, uActResolution, uActZoom, sActFont,
    uActFontSize);
	 eTgifState = NEWPOLY;

Dan again:
> Why are these strings not matching what is listed in the gnuplot-cvs files?
> 
> Perhaps this has something to do with this
> 
> # In order not to introduce noise into tree comparisons, you must specify
> the
> # argument 'finish' to do the part of the conversion that involves checking
> # in the FAQ and patching out the RCS cookies
> [snip]
> #
> # Also requires patch files COOKIEPATCH
> #
> 
> Is there something about the file COOKIEPATCH possibly missing some fixes
> needed for files tucked away in the Attic?

No, that's not it at all.  All COOKIEPATCH does is *remove all RCS cookies
from the tree* (well,except for one diff in the doc directory where I
couldn't figure out how to rip them out without damaging the diff).

> I agree these RCSid aren't that important, but it sure would be a nice
> feeling to have a match between, say, "cvs update -rXYZ" and "git checkout
> XYZ" from remote locations.

Too little gain for too much effort.  Once SourceForge shuts down CVS,
nobody will ever care about "cvs update -rXYZ" again.

Hans-Bernhard Bröker <HBB...@t-...>:
> >On the other hand, I'm not getting
> >a good feeling from Dan's reports on his attempts to disentangle that branch.
> >We know it's corrupt - doesn't have the right tip state.
> 
> I'm not quite convinced Dan's analysis is correct.  The branch isn't
> corrupt, it was just set up somewhat strangely at the time.

It's corrupt in the sense that because of the vendor-branch confusion we
not only can't guarantee that the gitspace sequence of commits really
matches the CVS development history, we can't even get the right tip
state out of the conversion.

The extent to which this is a CVS problem versus a git problem versus
a cvs-fast-export problem is really only of theoretical interest.

I've spent three weeks on this conversion and my patience for
wandering further off into the weeds is evaporating.  It only
lasted this long because getting ChangeLog analysis right seemed
like a thing worth doing.

Can we please refocus on *finishing* this with changes that actually add
value for forward development?

> Vendor branches with other branches partially rooted in them cannot be
> collapsed or ignored.  Nor can they be treated completely like ordinary
> branches (or conversion tools would have been doing that since day one).

I know.  Vendor branches have been a huge pain in the ass on other
conversions I've done, too.

Here is what I suggest we do:

1. Drop the 3.7.x branch.  Replace it with a branch consisting of just
the archived 3.7.[0123] trees in sequence (I now have them all).

2. Fix any remaining deficiences in ChangeLog scanning if possible.

3. Fill in the empty log comments that are left.

4. Wrap it up and call it done.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Morality is always the product of terror; its chains and
strait-waistcoats are fashioned by those who dare not trust others,
because they dare not trust themselves, to walk in liberty.
	-- Aldous Huxley 

Release 5.2.1

[sfeam <sf...@us...>]

I have uploaded a tarball gnuplot-5.2.1.tar.gz to the usual SourceForge 
distribution site.

   https://sourceforge.net/projects/gnuplot/files/gnuplot/5.2.1/ 

This is a bit early for an incremental release but I want to put it out now
for two reasons.  

A cluster of related regressions were reported for the 5.2.0 handling of
log-scaled  axis ranges and tic generation.  I consider this regression
to be serious and worth repairing sooner rather than later.

It is not clear when the cvs->git conversion process will settle down to
the point where I am comfortable preparing an incremental release
from the new git repository.  Rather than waiting for that, I would rather
package up 5.2.1 from a final cvs snapshot plus a couple of last-minute
fixes that I will maintain on the side until the git conversion completes.

The list of changes between 5.2.0 and 5.2.1 includes

* NEW set table separator {tab|comma|"char"} allows creation of csv files
* NEW hotkey for changing azimuth in 3D plots with mousing
* NEW splot ... with lines title at {beg|end}
* NEW Rework gstrptime() to handle relative time formats tH tM tS
* NEW command "set rgbmax <value>" controls interpretation of input RGB values
* FIX allow mixed use of in-key plot titles and manually placed titles
* FIX prevent runaway iterations of the form   plot for [i=start:*] ...
* FIX handle in-line range limits for linked or nonlinear axes
* FIX restore pre-5.2 interpretation of logscaled tic increment as a multiplier
* FIX logscale tic placement is closer to that of versions before 5.2.0
* FIX recheck inrange/outrange points after spline or bezier smoothing
* FIX autoscaling of plots with linked axes where data is plotted on x2 or y2
* FIX sampling on x2 if linked to x1;  e.g. plot sample [t=1:5:1] '+' axes x2y1
* FIX empty range on logscale y axis is handled by auto-extending the range

More information in the ChangeLog.

	happy gnuplotting,

		Ethan

Re: Update on the branch surgery

[Daniel J S. <dan...@ie...>]

On 10/28/2017 12:51 PM, Eric S. Raymond wrote:
> Hans-Bernhard Bröker <HBB...@t-...>:
>> That's really 3.6 beta340, one of a rather long series of releases called,
>> by their full name, "pre-3.6 beta {number}", or just "beta{number}" for
>> short.  These were maintained by Dave Denholm and Lars Hecking, in a
>> non-public CVS.   Some important milestones in that sequence even got their
>> own "patchlevel" sub-releases.
>>
>> [BTW, I finally dug through my collection of old HDs and found quite a
>> number of them as tarballs, e.g. betas 178, 213, 231, 340 and 347, and also
>> an apparently genuine 3.7.0].
> 
> On the one hand, that 3.7.0 would be interesting - might help us tag the
> corresponding point in the repository.  On the other hand, I'm not getting
> a good feeling from Dan's reports on his attempts to disentangle that branch.
> We know it's corrupt - doesn't have the right tip state.
> 
> I'd be just as happy to drop it at this point
> 
> Sigh...you should probably sent us both copies of 3.7.0.
> 
>> Number 340 in that series became the start of our CVS repository, as the
>> initial vendor branch import.  Betas 343,344,345,346 and 347 were imported
>> onto the vendor branch, then Lars stopped using his private repository, so
>> there was no further use of the vendor branch.
> 
> Except it seems to have become confounded with the 3.7.x branch we have.

I've dug into the actual ./reconvert process a bit.  I'm bewildered by 
some fundamental things regarding the CVS repository on

:pserver:ano...@gn...:/cvsroot/gnuplot

I assume that this repository is the same that is used to get the CVS 
copy used in the ./reconvert process

cvssync -c gnuplot.cvs.sourceforge.net:/cvsroot/gnuplot gnuplot
mv gnuplot gnuplot-cvs

Those are the same repository, correct?

OK, I've been comparing the gnuplot-git repository against "cvs update 
-rGNUPLOT_3_7_0" and so on.  As I pointed out I've often gotten good 
matches in terms of code mods, with the exception being all these RCSid 
strings.

I see now how that "drd" is finding its way into the git repository. 
The cvssync gets the repository in archival form to the local system 
into gnuplot-cvs.  In there is the history for the ./alloc.c file, and 
I'll extract the pertinent references to "drd":

1.2
log
@Fix segfault.
@
text
@d2 1
a2 1
static char *RCSid = "$Id: alloc.c,v 1.12 1998/03/22 22:31:16 drd Exp $";
@


1.1
log
@Initial revision
@
text
@d2 1
a2 1
static char *RCSid = "$Id: alloc.c,v 1.11 1997/04/10 02:32:39 drd Exp $";


1.1.1.2
log
@Import of beta 343.
@
text
@d2 1
a2 1
static char *RCSid = "$Id: alloc.c,v 1.12 1998/03/22 22:31:16 drd Exp $";

Now, when I get the ./alloc.c file remotely from the server with "cvs" I 
get the following RCSid strings:

cvs update -r1.1 alloc.c
(yields)
static char *RCSid = "$Id: alloc.c,v 1.1 1998/04/15 19:16:27 lhecking 
Exp $";

cvs update -r1.2 alloc.c
(yields)
static char *RCSid = "$Id: alloc.c,v 1.2 1999/03/23 12:21:51 lhecking 
Exp $";

cvs update -r1.1.1.2 alloc.c
(yields)
static char *RCSid = "$Id: alloc.c,v 1.1.1.2 1998/04/15 19:21:56 
lhecking Exp $";

Why are these strings not matching what is listed in the gnuplot-cvs files?

Perhaps this has something to do with this

# In order not to introduce noise into tree comparisons, you must 
specify the
# argument 'finish' to do the part of the conversion that involves checking
# in the FAQ and patching out the RCS cookies
[snip]
#
# Also requires patch files COOKIEPATCH
#

Is there something about the file COOKIEPATCH possibly missing some 
fixes needed for files tucked away in the Attic?

I agree these RCSid aren't that important, but it sure would be a nice 
feeling to have a match between, say, "cvs update -rXYZ" and "git 
checkout XYZ" from remote locations.

Dan

Re: Update on the branch surgery

[Eric S. R. <es...@th...>]

Hans-Bernhard Bröker <HBB...@t-...>:
> That's really 3.6 beta340, one of a rather long series of releases called,
> by their full name, "pre-3.6 beta {number}", or just "beta{number}" for
> short.  These were maintained by Dave Denholm and Lars Hecking, in a
> non-public CVS.   Some important milestones in that sequence even got their
> own "patchlevel" sub-releases.
> 
> [BTW, I finally dug through my collection of old HDs and found quite a
> number of them as tarballs, e.g. betas 178, 213, 231, 340 and 347, and also
> an apparently genuine 3.7.0].

On the one hand, that 3.7.0 would be interesting - might help us tag the
corresponding point in the repository.  On the other hand, I'm not getting
a good feeling from Dan's reports on his attempts to disentangle that branch.
We know it's corrupt - doesn't have the right tip state.

I'd be just as happy to drop it at this point

Sigh...you should probably sent us both copies of 3.7.0. 

> Number 340 in that series became the start of our CVS repository, as the
> initial vendor branch import.  Betas 343,344,345,346 and 347 were imported
> onto the vendor branch, then Lars stopped using his private repository, so
> there was no further use of the vendor branch.

Except it seems to have become confounded with the 3.7.x branch we have.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Update on the branch surgery

[Eric S. R. <es...@th...>]

Petr Mikulik <mi...@ph...>:
> Those "beta 3xy" are patchlevels of gnuplot 3.6, not patchlevels of gnuplot
> 3.4.

Thanks, the annotations now reflect this.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Update on the branch surgery

[Eric S. R. <es...@th...>]

Daniel J Sebald <dan...@ie...>:
> >That diff is between two different revisions on the vendor branch (1.1.1.1
> >and 1.1.1.2, a.k.a. "Initial import of beta340" and "Import of 344").
> >Revision 1.1.1.2 is correct for all relased tags since 3.7.0, but I
> >suspect the general plan of "what happens on a vendor branch, stays on a
> >vendor branch", fails here.
> 
> Agreed.  This is a consequence of reposurgeon choosing to place that 3.7.x
> branch somewhere other than the trunk at (what I believe should be)
> 
> Date:   Thu Jan 14 19:35:52 1999 +0000
>     New file.
> 
> Lars created that branch, 1.1.1, near the start but didn't use it for much,
> apparently.  The branch-master is based on 1.1:
> 
> ----------------------------
> revision 1.1
> date: 1998/04/15 19:16:41;  author: lhecking;  state: Exp;
> branches:  1.1.1;
> Initial revision
> ----------------------------
> 
> I confirmed there are no comments "# HBB: revised etc." in the master branch
> which matches Lars' CVS repo.
> 
> So there are two things going on, basically the same principle.  Some
> differences in branch-3.7.7 hang around if rebasing after the fact. Some
> differences at the very beginning of the master branch get propagated. When
> we compare across the two branches trying to find the graft location for
> 3.7.x series, its a combination of differences.
> 
> Can Reposurgeon force branch points at the start, as opposed to doing
> rebasing?

I don't understrand your terminology.

It's not at all difficult to rebase a branch root, if that's what you're asking.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.