gnuplot-beta — Developers and beta testers of new gnuplot code

Documentation for special-filenames

[Lutz M. <lut...@gm...>]

I just noticed that the help text for “special-filenames” does not describe the use of ‘-‘, ‘’, pipes, or file descriptors. Instead those are all described in the “++” subtopic of “special-filenames”, which seems surprising to me. I am using 5.2.1 installed from MacPorts. Is that a bug in the documentation?

Best,

Lutz

Re: Time to start wrapping up the conversion.

[Ethan A M. <sf...@us...>]

[trying again]
My mailer ate the embedded traceback.


On Wednesday, November 1, 2017 2:22:43 PM PDT Eric S. Raymond wrote:
> I've uploaded a fresh version of the conversion machinery where you
> can fetch it at
> 
> wget http://www.catb.org/~esr/gnuplot-conversion.tar.gz

For me, reconvert dIes before finishing:


reposurgeon% # The 4.0 branch 
reposurgeon% blob <FUBAR 
reposurgeon% /#include gp_time.h/ mailbox_in --create <NEWCOMMIT 
reposurgeon% /#include gp_time.h/ assign v40root --singleton 
Traceback (most recent call last): 
 File "/usr/local/bin/reposurgeon", line 12393, in <module> 
   main() 
 File "/usr/local/bin/reposurgeon", line 12369, in main 
   interactive() 
 File "/usr/local/bin/reposurgeon", line 12358, in interactive 
   interpreter.cmdloop() 
 File "/usr/lib64/python2.7/cmd.py", line 141, in cmdloop 
   line = self.precmd(line) 
 File "/usr/local/bin/reposurgeon", line 7960, in precmd 
   line = self.set_selection_set(line) 
 File "/usr/local/bin/reposurgeon", line 7996, in set_selection_set 
   if self.chosen().named(self.line): 
 File "/usr/local/bin/reposurgeon", line 5750, in named 
   key=len, reverse=True): # longest name first 
TypeError: object of type 'NoneType' has no len()


Suggestions?


> This version contains a new script, 'forcepush'. After running
> reconvert, you can run forcepush to nuke the the SourceForge git
> repository and replace it with the new conversion.  (You must
> have Tcl and Expect installed for this to work.)
> 
> The script does assume that your local username is the sane as your
> SourceForge one, because I'm 'esr' both places.  If this is a
> problem for anyone, I'm sure we can modify the "nuke" and "forcepush"
> scripts to generalize them.
> The entire conversion and replacement process is now implemented and
> documented by the three scripts reconvert, nuke, and forcepush.  It
> can be run at any time by anyone with the right SourceForge
> credentials.
> 
> When doing repo conversions, I think it's important to show the work
> so all its assumptions are auditable.  Also, we are now far enough
> along in the process that if I have to run off to focus exclusively on
> something else, somebody (most likely Dan Sebald or HBB) would be able
> to finish the job.  

Sounds good.

> So, this is my major deliverable; the rest is polishing.

Thanks. Really, thanks.


> Things still to be done:
> 
> 1. The EMPTIES file in the conversion tarball needs to have real log
> messages filled in for all those "*** empty log message ***" instances
> (but don't touch the Check-Text headers; those are a safety measure
> in case the commit dates drift, they should prevent clobbering of the
> comment text in nearby commits).
>
> A relatively easy way to do this: bring up EMPTIES in your favorite
> editor, then run gitk on a repo conversion.  Looking at the diffs
> gitk displays per commit should make it easy to write a useful change
> comment.  Note that the topmost text box in the lower half of the
> gitk display is for text search; if you type "*** empty log message
> ***" into it, you can use the up and down arrows off to the left
> to quickly move between empty comments.

Not going to happen.  If I had nothing to say about a commit at the
time, I certainly don't have anything to say about it now.
Is there a command to replace all of these with the Jedi mind trick 
"This is not the commit you are looking for"
 
> If all of you mail your EMPTIES changes back to me I'll take care of
> collating them.  If the process runs long I might set up a
> public repo for the conversion stuff.
> 
> 2. The final disposition of the 3.7.x branch needs to be decided.  I
> think it is soaking up more effort than it is worth and should be
> dropped.  I'm pretty sure Ethan agrees.

Yes.

> We're almost done.  Let's get this wrapped up.

!!
	Ethan

Re: Time to start wrapping up the conversion.

[Ethan A M. <sf...@us...>]

On Wednesday, November 1, 2017 2:22:43 PM PDT Eric S. Raymond wrote:
> I've uploaded a fresh version of the conversion machinery where you
> can fetch it at
> 
> wget http://www.catb.org/~esr/gnuplot-conversion.tar.gz

For me, reconvert dIes before finishing:


reposurgeon% # The 4.0 branch 

Suggestions?


> This version contains a new script, 'forcepush'. After running
> reconvert, you can run forcepush to nuke the the SourceForge git
> repository and replace it with the new conversion.  (You must
> have Tcl and Expect installed for this to work.)
> 
> The script does assume that your local username is the sane as your
> SourceForge one, because I'm 'esr' both places.  If this is a
> problem for anyone, I'm sure we can modify the "nuke" and "forcepush"
> scripts to generalize them.
> The entire conversion and replacement process is now implemented and
> documented by the three scripts reconvert, nuke, and forcepush.  It
> can be run at any time by anyone with the right SourceForge
> credentials.
> 
> When doing repo conversions, I think it's important to show the work
> so all its assumptions are auditable.  Also, we are now far enough
> along in the process that if I have to run off to focus exclusively on
> something else, somebody (most likely Dan Sebald or HBB) would be able
> to finish the job.  

Sounds good.

> So, this is my major deliverable; the rest is polishing.

Thanks. Really, thanks.


> Things still to be done:
> 
> 1. The EMPTIES file in the conversion tarball needs to have real log
> messages filled in for all those "*** empty log message ***" instances
> (but don't touch the Check-Text headers; those are a safety measure
> in case the commit dates drift, they should prevent clobbering of the
> comment text in nearby commits).
>
> A relatively easy way to do this: bring up EMPTIES in your favorite
> editor, then run gitk on a repo conversion.  Looking at the diffs
> gitk displays per commit should make it easy to write a useful change
> comment.  Note that the topmost text box in the lower half of the
> gitk display is for text search; if you type "*** empty log message
> ***" into it, you can use the up and down arrows off to the left
> to quickly move between empty comments.

Not going to happen.  If I had nothing to say about a commit at the
time, I certainly don't have anything to say about it now.
Is there a command to replace all of these with the Jedi mind trick 
"This is not the commit you are looking for"
 
> If all of you mail your EMPTIES changes back to me I'll take care of
> collating them.  If the process runs long I might set up a
> public repo for the conversion stuff.
> 
> 2. The final disposition of the 3.7.x branch needs to be decided.  I
> think it is soaking up more effort than it is worth and should be
> dropped.  I'm pretty sure Ethan agrees.

Yes.

> We're almost done.  Let's get this wrapped up.

!!
	Ethan

Time to start wrapping up the conversion.

[<es...@th...>]

I've uploaded a fresh version of the conversion machinery where you
can fetch it at

wget http://www.catb.org/~esr/gnuplot-conversion.tar.gz

This version contains a new script, 'forcepush'. After running
reconvert, you can run forcepush to nuke the the SourceForge git
repository and replace it with the new conversion.  (You must
have Tcl and Expect installed for this to work.)

The script does assume that your local username is the sane as your
SourceForge one, because I'm 'esr' both places.  If this is a
problem for anyone, I'm sure we can modify the "nuke" and "forcepush"
scripts to generalize them.

The entire conversion and replacement process is now implemented and
documented by the three scripts reconvert, nuke, and forcepush.  It
can be run at any time by anyone with the right SourceForge
credentials.

When doing repo conversions, I think it's important to show the work
so all its assumptions are auditable.  Also, we are now far enough
along in the process that if I have to run off to focus exclusively on
something else, somebody (most likely Dan Sebald or HBB) would be able
to finish the job.  So, this is my major deliverable; the rest is polishing.

Things still to be done:

1. The EMPTIES file in the conversion tarball needs to have real log
messages filled in for all those "*** empty log message ***" instances
(but don't touch the Check-Text headers; those are a safety measure
in case the commit dates drift, they should prevent clobbering of the
comment text in nearby commits).

A relatively easy way to do this: bring up EMPTIES in your favorite
editor, then run gitk on a repo conversion.  Looking at the diffs
gitk displays per commit should make it easy to write a useful change
comment.  Note that the topmost text box in the lower half of the
gitk display is for text search; if you type "*** empty log message
***" into it, you can use the up and down arrows off to the left
to quickly move between empty comments.

If all of you mail your EMPTIES changes back to me I'll take care of
collating them.  If the process runs long I might set up a
public repo for the conversion stuff.

2. The final disposition of the 3.7.x branch needs to be decided.  I
think it is soaking up more effort than it is worth and should be
dropped.  I'm pretty sure Ethan agrees.

We're almost done.  Let's get this wrapped up.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Where rights secured by the Constitution are involved, there can be no
rule making or legislation which would abrogate them.
        -- Miranda vs. Arizona, 384 US 436 p. 491

Re: Roundup of GNUPLOT conversion issues

[Eric S. R. <es...@th...>]

Hans-Bernhard Bröker <HBB...@t-...>:
> >>Only when we switched to automake, dropped all generated files from the
> >>repository, and started using the 'prepare' script instead, did tarball
> >>contents start to be really different from fresh working copies of their
> >>corresponding CVS tags.
> >
> >Can you identify when that switchover was?
> >
> The change to automake happened very soon after the directory structure
> change, on 1999-03-28.  'prepare' exists since 2001-06-09T19:15:55.  The
> drop of auto-generated files was on 2003-01-08 (as per state 'dead' in the
> Attic/,v files, and a commit mail in in the gnuplot-cvs archives).
> 
> The distance between fresh working copies and tarballs really only jumped up
> at the 2003-01-08 step, though.

Thanks.  I figure that where we can verify a tag-for-tag match of the
CVS content with the git content that's the best thing to do, but there
might come up cases where we need to identify a tag location by matching
against tarball content, in which case this will be good to know.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: State of the CVS conversion

[Ethan A M. <sf...@us...>]

On Wednesday, November 1, 2017 10:20:28 AM PDT Eric S. Raymond wrote:
> I've uploaded a fresh version of rhe conversion machinery where you
> can fetch it at
> 
> wget http://www.catb.org/~esr/gnuplot-conversion.tar.gz
> 
> Problems solved in this release:
> 
> * Changelog scanning catches the cases Bastian was worried about.
>   It's probably done.
> 
> Remaining issues:
> 
> * The 3.7.x branch is still a mess, with CVS tip state not matching
>   the git tip state.

	Since it is very unlikely that anyone needs to work from git tip
	for 3.7, this doesn't seem like a big deal.

> * Empty log messages still need to be filled in.

	Why?  

		Ethan

Re: Roundup of GNUPLOT conversion issues

[Hans-Bernhard B. <HBB...@t-...>]

Am 01.11.2017 um 14:34 schrieb Eric S. Raymond:
> Hans-Bernhard Bröker <HBB...@t-...>:

>> Only when we switched to automake, dropped all generated files from the
>> repository, and started using the 'prepare' script instead, did tarball
>> contents start to be really different from fresh working copies of their
>> corresponding CVS tags.
> 
> Can you identify when that switchover was?
> 
The change to automake happened very soon after the directory structure 
change, on 1999-03-28.  'prepare' exists since 2001-06-09T19:15:55.  The 
drop of auto-generated files was on 2003-01-08 (as per state 'dead' in 
the Attic/,v files, and a commit mail in in the gnuplot-cvs archives).

The distance between fresh working copies and tarballs really only 
jumped up at the 2003-01-08 step, though.

State of the CVS conversion

[<es...@th...>]

I've uploaded a fresh version of rhe conversion machinery where you
can fetch it at

wget http://www.catb.org/~esr/gnuplot-conversion.tar.gz

Problems solved in this release:

* Changelog scanning catches the cases Bastian was worried about.
  It's probably done.

Remaining issues:

* The 3.7.x branch is still a mess, with CVS tip state not matching
  the git tip state.

* Empty log messages still need to be filled in.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Where rights secured by the Constitution are involved, there can be no
rule making or legislation which would abrogate them.
        -- Miranda vs. Arizona, 384 US 436 p. 491

Re: Roundup of GNUPLOT conversion issues

[Eric S. R. <es...@th...>]

Hans-Bernhard Bröker <HBB...@t-...>:
> >>Please note that the tip of the 3.7.* branch is not coincident with the
> >>3.7.3 tag or tarball.  Some further changes were made on the branch after
> >>3.7.3.
> >
> >I do not see any in gitspace. How would I retrieve or list them in CVS?
> 
> The pedestrian way would be to just check out a working copy on the branch,
> and one of the last release tag from the same branch, and compare those.  In
> a generated reposurgeon Makefile setup:

OK, I have verified that I have those post-3.7.3 commits in the conversion.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Roundup of GNUPLOT conversion issues

[Eric S. R. <es...@th...>]

Hans-Bernhard Bröker <HBB...@t-...>:
> Am 29.10.2017 um 20:31 schrieb Daniel J Sebald:
> 
> >I don't think the exact tarballs are there.  I sort of have a recollection
> >of whenever a release was to be made, the tarball required some extra
> >effort on someone's part.
> 
> Not back then they didn't.  The tarballs can be generated directly from a
> fresh working copy, just by running
> 
> 	./configure && make dist
> 
> But all that effectively does is make a version-named copy of the source
> tree without "CVS" directories and .cvsignore files, and tarball that.
> 
> Any actual differences between that and the official release tarball are
> down to either
> 
> a) auto-tools of different versions being re-run at 'make' time, or
> b) last-minute fixes applied directly to the tarball, manually
> 
> Only when we switched to automake, dropped all generated files from the
> repository, and started using the 'prepare' script instead, did tarball
> contents start to be really different from fresh working copies of their
> corresponding CVS tags.

Can you identify when that switchover was?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: One branchpoint discrepancy

[Eric S. R. <es...@th...>]

Daniel J Sebald <dan...@ie...>:
> Those number of branches suggests to me the first step in this process is to
> build a branch/linked-list for all individual files, i.e., the file.ext,v
> files as the source.  Then to build a branch means "merging" all those in a
> meaningful way to create the git branch--something involving "clique"
> determines the root of the branch.  Having reached that point, then the git
> branch is attached to its parent, i.e., branch point.

That is correct.  For branch-merging purposes, a "clique" is a set of file
mods with identical metadata (committer and comment) and dates within a certain
time fuzz in seconds of each other (default 300 seconds).

The logic walks down branches from tip to base trying to identify cliques
which it makes into changesets.

> The issue is that there is no consideration in this clique process that
> includes the symbol "version stamps" identifying cross-branch file use.

That is also correct.  

>                I really like the C++ structure of the code, e.g., the way
> that CVS numbers are stored as little objects for which A-to-B comparisons
> can be done easily.  It's just a question of wanting to set about doing
> that.

Yeah, thank Keith for that, then me.  He went partway down the road of making
those things objectlets; I saw where he was going and continued it to its
logical conclusion.  I have no doubt he would have done the same, but he
abandoned the code pretty immediately and in a semi-unfinished state
as soon as he got the X repos moved.

As a result, the core algorithm and overall architecture are Keith's, but the
prettiness is often me.  I did it in self-defense - refactoring the code
to make it readable was the only way I could find to understand it.

> However, the sense in which branch_merge() considers "merge" isn't the same
> notion as a git "merge".

Yes.  I should think about renaming so the term "merge" isn't used, to avoid
that confusion.  Perhaps "collate"?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: One branchpoint discrepancy

[Daniel J S. <dan...@ie...>]

On 10/31/2017 02:02 PM, Eric S. Raymond wrote:
> Daniel J Sebald <dan...@ie...>:
>>> I rather doubt any good will come from that.  Nobody understands that
>>> utility anywhere near well enough to be able to modify it on short notice
>>> without risking total break-down.  At this point it is, essentially,
>>> magic.
>>
>> It doesn't look like a very big program.
> 
> No, it doesn't.  What used to be one large lump of incomprehensibility
> has shrunk in size and apparent complexity as I carved pieces off the
> outside, narrowing their interfaces to the central core.
> 
> Those outside bits are indeed pretty small and comprehensible, now.
> The most notable success was a clean refactor of the front-end code
> for digesting RCS masters that enabled it to use thread-per-master
> parallelism with a re-entrant parser.  That's how the program went
> from respectably fast to ridiculously fast.
> 
> Unfortunately, there remain two knots of mystery in the code that
> nobody has been able to crack.  The greater one one is the actual
> changeset generation in merge.c - the black hole of incomprehension is
> the functions merge_to_changesets() and merge_branches(), which solve
> the general changeset-synthesis problem HBB accurately described.  The
> lesser mystery is cvs_master_patch_vendor_branch(), which is probably
> where our problem is.

Yes, I started out looking at merge.c and merge_branches().  It walks 
backward from the start of heads of file branches stitching them 
together to create a git branch.  I printed out the number of branches:

find . -name '*,v' -print | cvs-fast-export --reposurgeon > /dev/null
MERGED SOME BRANCHES 1257
MERGED SOME BRANCHES 338
MERGED SOME BRANCHES 407
cvs-fast-export: warning - branch point branch-pre-3-7-1 -> import-1.1.1 
matched by date
MERGED SOME BRANCHES 672
cvs-fast-export: warning - branch point branch-4-2-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 740
cvs-fast-export: warning - branch point branch-4-4-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 795
cvs-fast-export: warning - branch point branch-4-6-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 805
cvs-fast-export: warning - branch point branch-5-0-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 789
MERGED SOME BRANCHES 553
cvs-fast-export: warning - branch point column -> import-1.1.1 matched 
by date
MERGED SOME BRANCHES 544
cvs-fast-export: warning - branch point branch-4-0-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 529
cvs-fast-export: warning - branch point pm3d -> import-1.1.1 matched by date
MERGED SOME BRANCHES 507
cvs-fast-export: warning - branch point axis_branch_base -> import-1.1.1 
matched by date
cvs-fast-export: no commitids before 2017-10-31T04:47:09Z.

Those number of branches suggests to me the first step in this process 
is to build a branch/linked-list for all individual files, i.e., the 
file.ext,v files as the source.  Then to build a branch means "merging" 
all those in a meaningful way to create the git branch--something 
involving "clique" determines the root of the branch.  Having reached 
that point, then the git branch is attached to its parent, i.e., branch 
point.

The issue is that there is no consideration in this clique process that 
includes the symbol "version stamps" identifying cross-branch file use. 
(Instead it is solely based upon date, which means cvs-fast-export 
concludes so many of these branches go back to the import/vendor branch 
as the branchpoint.  See warnings above about dates matching.)  The 
information about branches there, all neatly saved in the branch C++ 
structures; it's just not used.  I really like the C++ structure of the 
code, e.g., the way that CVS numbers are stored as little objects for 
which A-to-B comparisons can be done easily.  It's just a question of 
wanting to set about doing that.

However, the sense in which branch_merge() considers "merge" isn't the 
same notion as a git "merge".


> Also, fast actually matters.  cvs2git probably comes the closest to
> cvs-fast-export in terms of correctness and robustness across weird
> cases.  But it's so slow that the kind of iterative refinement I'm
> doing by repeatedly tweaking the reconvert script is not really
> practical - I belive you noticed 45 minutes of lag, as opposed to 20
> seconds on my desktop.  That's a problem that's getting worse over
> time, because the small, clean CVS repos have already been moved.
> Increasingly it's only the large, old, nasty ones that are left.

Yes, cvs2git is slow, sort of brute force creating some giant blob.

Dan

Re: The first repository branch, "Windows linestyle fix."

[Eric S. R. <es...@th...>]

Hans-Bernhard Bröker <HBB...@t-...>:
> Am 29.10.2017 um 20:01 schrieb Daniel J Sebald:
> >I've experimented with something called cvs2git, which I believe does a
> >pretty good job of constructing branches and merges, but has some
> >drawbacks.
> [...]
> >It's major drawbacks are:
> >
> >1) No corrected author info.
> >
> >2) All of the RCSid strings are replaced by empty string, e.g.,
> 
> That's not a drawback; it's a necessary consequence of moving to git. One
> way or another, all these strings (technically: RCS-style keywords) have to
> go.

There's a late step in reconvert that does exactly that.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: One branchpoint discrepancy

[Daniel J S. <dan...@ie...>]

On 10/31/2017 12:11 PM, Eric S. Raymond wrote:
> Hans-Bernhard Bröker <HBB...@t-...>:
>> Am 30.10.2017 um 20:30 schrieb Daniel J Sebald:
>>
>>> I'd sort of like to put effort into the right place.  I've cloned the
>>> cvs-fast-export utility and I'm willing to help on matters, so if its
>>> possible I wonder if we could enhance the merging aspect of that utility.
>>
>> I rather doubt any good will come from that.  Nobody understands that
>> utility anywhere near well enough to be able to modify it on short notice
>> without risking total break-down.  At this point it is, essentially, magic.
> 
> Sadly, HBB is correct.  The way I modified Keith's original code was by
> working from the outside in - there is a hard core, now in merge.c, that nobody
> understands.

I've looked at the code.  Very clean and well-written, actually, but I 
see the limitations.

Dan

Re: Roundup of GNUPLOT conversion issues

[Eric S. R. <es...@th...>]

Daniel J Sebald <dan...@ie...>:
> >That would be slightly wrong because 3.7.0 is not on the branch.  It's on
> >the trunk.
> 
> This is correct.  GNUPLOT_RELEASE_3_7_0 is on the trunk while the other
> GNUPLOT_RELEASE_3_7_x are on the branch.  I'm attaching screenshots of the
> cvs2git tree and the cvs-fast-export tree after I've made the rebase with
> reposurgeon.  The GNUPLOT_RELEASE_3_7_0 synthetic commit was done by
> cvs-fast-export, not reposurgeon.  So, cvs-fast-export got the tag/branch
> correct but didn't realize that's where the pertinent branch should go.

Right, because that commit represents a broken (incomplete) tag set.
cvs-fast-export ignores those as branch point candidates because
there's no way to know that the checkout at that point is correct.

Which actually gives me an idea for a feature.  I could add a switch to
whitelist incomplete tags.

Here is a useful thing you can do while I try to fix the changelog scanning.
Modify reconvert so it doesn't delete that one branchlet, reparent the
3.7.x branch onto it, and compare the result at tip.

If there are no differences other than RCS IDs, I would whitelist the tag.
At that point we could consider the 3.7.x branch salvaged. 
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: One branchpoint discrepancy

[Eric S. R. <es...@th...>]

Daniel J Sebald <dan...@ie...>:
> >I rather doubt any good will come from that.  Nobody understands that
> >utility anywhere near well enough to be able to modify it on short notice
> >without risking total break-down.  At this point it is, essentially,
> >magic.
> 
> It doesn't look like a very big program.

No, it doesn't.  What used to be one large lump of incomprehensibility
has shrunk in size and apparent complexity as I carved pieces off the
outside, narrowing their interfaces to the central core.

Those outside bits are indeed pretty small and comprehensible, now.
The most notable success was a clean refactor of the front-end code
for digesting RCS masters that enabled it to use thread-per-master
parallelism with a re-entrant parser.  That's how the program went
from respectably fast to ridiculously fast.

Unfortunately, there remain two knots of mystery in the code that
nobody has been able to crack.  The greater one one is the actual
changeset generation in merge.c - the black hole of incomprehension is
the functions merge_to_changesets() and merge_branches(), which solve
the general changeset-synthesis problem HBB accurately described.  The
lesser mystery is cvs_master_patch_vendor_branch(), which is probably
where our problem is.

Over the last five years about six pretty sharp hackers have tried
to comprehend these well enough to substantially modify them.  The two
who came closest to succeeding were probably Lawrence Hygate and
myself, but all of us ultimately failed. The central merge code and
the vendor-branch handling are not much less of a black box than when
Keith Packard wrote them.

Yes, this seriously sucks.  It means that when cvs-fast-export fails
there isn't much recourse.  But...I triaged several CVS lifters when I
was qualifying front ends for reposurgeon; I ended up adopting this
one (and heavily modifying it) because it sucked the least. Which is
to say it handles the largest range of cases without crashing or
spewing nonsense.

Also, fast actually matters.  cvs2git probably comes the closest to
cvs-fast-export in terms of correctness and robustness across weird
cases.  But it's so slow that the kind of iterative refinement I'm
doing by repeatedly tweaking the reconvert script is not really
practical - I belive you noticed 45 minutes of lag, as opposed to 20
seconds on my desktop.  That's a problem that's getting worse over
time, because the small, clean CVS repos have already been moved.
Increasingly it's only the large, old, nasty ones that are left.

When I wrote about this adverse-selection problem in 2014 I actually
cited GNUPLOT as an example:

     http://esr.ibiblio.org/?p=6216

So...Daniel, the problem is really difficult, and not one I think we
can afford to block the GNUPLOT conversion on when CVS support has a
hard drop-dead date.

That said, you might bring a new perspective to the problem.  I ran
out of possibilities years back; if you can actually improve the
vendor-branch handling I will be *very* impressed.

> Maybe that is what cvs2git is doing.  As I said, it takes 45 minutes. But
> gosh the tree-structure and tags of cvs-fast-export matches cvs2git so well.
> I think it is a simple matter of cvs-fast-export not recognizing it has to
> do a merge in those half dozen locations due to a cross-branch reference to
> a version number.  If reposurgeon could make those connections, it would be
> nice, but I don't think reposurgeon works that way, just rebasing.

That is correct.  By the time reposurgeon sees the stream, the information
required to do vendor-branch surgery is gone.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: One branchpoint discrepancy

[Eric S. R. <es...@th...>]

Hans-Bernhard Bröker <HBB...@t-...>:
> Am 30.10.2017 um 20:30 schrieb Daniel J Sebald:
> 
> > I'd sort of like to put effort into the right place.  I've cloned the
> >cvs-fast-export utility and I'm willing to help on matters, so if its
> >possible I wonder if we could enhance the merging aspect of that utility.
> 
> I rather doubt any good will come from that.  Nobody understands that
> utility anywhere near well enough to be able to modify it on short notice
> without risking total break-down.  At this point it is, essentially, magic.

Sadly, HBB is correct.  The way I modified Keith's original code was by
working from the outside in - there is a hard core, now in merge.c, that nobody
understands. 

> That won't work at all.  The primary conflict is that CVS has individual
> branch structures for every member file, whereas git branches the entire
> repository.  Those two concepts don't mix and match.  BETA_344_989422 may be
> the right join point for this particular file, but that's most likely the
> _only_ file for which that's the case.  Basically every single tag ever made
> in CVS can contain one or more file joins from the vendor branch onto the
> trunk.  Some are still waiting to happen.
> 
> Normal CVS repositories would have every single file starting off at 1.1.
> I.e. the first tag would be on 1.1 revisions of every file, and all
> development would start from there.  The conversion tools have no problem at
> all with this set-up.
> 
> But our repository was started by a "cvs import", and received some further
> imports after that, and that changes everything.  It means that all our
> original files started at revision 1.1.1.1, and progressed along that
> 1.1.1.* branch, until they were first modified.  None of them ever got a tag
> on it 1.1 revision --- 1.1. was really never used for anything.
> 
> Every time a file that was on the vendor branch until that point (and
> remember, for some files that still hasn't happened today!) is checked in,
> that particular file is essentially merged over from the vendor branch onto
> the trunk --- but in the archive this merge appears as an ordinary check-in
> of a revision 1.2.
>
> E.g. even though in a RCS revision tree, it appears like this:
> 
> 	1.1 --*---> 1.2
> 	       \
> 	        +-> 1.1.1.1 --> 1.1.1.2
> 
> the real sequence of active revisions for gnuplot.rot is:
> 
> 	1.1.1.1 --> 1.1.1.2 --> 1.2
> 
> Other files have different sequences, and different points in time at which
> they made their transition 1.1.1.1 --> 1.1.1.2, or 1.1.1.{n} --> 1.2.  Some
> even have branches inside the vendor branch (1.1.1.2.2.1)
> 
> The vendor branch may best be grafted into the trunk _before_ the 1.1
> initial import, and replace that entirely.  I.e. we might imagine that the
> above sequence was transformed into
> 
> 	1.0 --> 1.1 --> 1.2
> 
> Files with more stuff going on in their vendor branch would have to dip into
> negative numbers, i.e.
> 
> 	1.1.1.1 --> 1.1.1.2 +-> 1.1.1.3 --> 1.1.1.4 --> 1.2 ...
> 	                    |
> 	                    +-> 1.1.1.2.2.1
> 
> would (imaginatively) turn into
> 
> 	1.(-2)  --> 1.(-1)  +-> 1.0 ------> 1.1 ------> 1.2 ...
>  	                    |
>                             +-> 1.(-1).2.1
> 
> Realistically all the revision numbers in the entire archive would have to
> be shifted up such that the chain really does start at 1.1:
> 
> 	1.1  -----> 1.2 --*---> 1.3 ------> 1.4 ------> 1.5 ...
>  	                   \
>                             +-> 1.2.2.1
> 
> Unfortunately, the way RCS ,v files are organized, this shift can only be
> performed by parsing and re-encoding every revision on the vendor branch.
> (The direction the diffs are recorded is from the head all the way down to
> 1.1, and from there _up_ along the vendor branch).  And because of the way
> CVS uses RCS ,v files, every one of them has to be transformed individually.
> 
> _That_ is the transformation that needs to be done in order for conversion
> tools not to have any problems with the vendor branch.  And because the
> transformation differs for every RCS archive, it has to be done either
> directly on the CVS repository, or the importer has to pretend it had
> happened that way.
> 
> Let me reiterate: to the best of my understanding, no process working on an
> already converted git repository has any realistic chance to perform this
> operation correctly.  It has to be done on the CVS import side.

This is good analysis.  You have described the problem more clearly than
*I* undetstood it.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: One branchpoint discrepancy

[Daniel J S. <dan...@ie...>]

On 10/30/2017 05:52 PM, Hans-Bernhard Bröker wrote:
> Am 30.10.2017 um 20:30 schrieb Daniel J Sebald:
> 
>>  I'd sort of like to put effort into the right place.  I've cloned the
>> cvs-fast-export utility and I'm willing to help on matters, so if its 
>> possible I wonder if we could enhance the merging aspect of that utility.
> 
> I rather doubt any good will come from that.  Nobody understands that 
> utility anywhere near well enough to be able to modify it on short 
> notice without risking total break-down.  At this point it is, 
> essentially, magic.

It doesn't look like a very big program.


>> Yes, there is a bit of ambiguity in merges with CVS, it just isn't 
>> 100% accurate to identify exactly what the programmer had checked out 
>> when compiling and subsequently did a checkin.
> 
> Not really, because there _are_ no recorded merges in CVS.  If merges 
> happen, they do so in somebody's working copy.  To the repository, they 
> only ever appear as check-ins, without any indication whether the new 
> content was created by some kind of merge, or by just writing it manually.

I didn't state that right; I should have said "implied merges" or 
"psuedo-merges", i.e., the merges placed in the git translation by the 
conversion tool.  And the way something like cvs2git has an implied 
history for constructing those merges comes from the fact Lars placed 
what looks like three dozen or more tags in the work.  Without all those 
tags, it would be as you described below, simply a trail of numbers.


>> There might be some ambiguity about the state of other files at this 
>> point, but I would say simply recall the BETA_344_989422 state from 
>> CVS and call that the state for the merge and generate all git diffs 
>> accordingly, perhaps that's not the exact methodology.
> 
> That won't work at all.  The primary conflict is that CVS has individual 
> branch structures for every member file, whereas git branches the entire 
> repository.  Those two concepts don't mix and match.  BETA_344_989422 
> may be the right join point for this particular file, but that's most 
> likely the _only_ file for which that's the case.

It's not.  I believe there are several files that fall in that same 
category.  I can search for them, to verify, but that's a lot of work. 
But in terms of principle, generally BETA_344_989422 can appear in 
multiple files in the same way.  I sent screenshots of what cvs2git 
produces for merges, and it seems to produce a good job of matching CVS 
updates to various versions.  Here are the sorts of statements cvs2git 
is making:

"This commit was generated by cvs2svn to compensate for changes in r48, 
which included commits to RCS files with non-trunk default branches."

     BETA_344_989422:1.1.1.2  (this tag is in the trunk, but it's 
referencing a version that is in a branch different than the trunk)
     BETA_344:1.1.1.2         (this tag is in the branch)
     BETA_343_980416:1.1.1.1



>  Basically every 
> single tag ever made in CVS can contain one or more file joins from the 
> vendor branch onto the trunk.  Some are still waiting to happen.

That's correct, one or more.  And in every one of those merges back to 
the trunk (in cvs2git) there are probably a half dozen files that have 
that same pattern of being referenced by BETA_344_989422.  It's a 
subgroup of at least all these files I listed previously:

amiga.c
Copyright
ctrl87.c
ctrl87.h
demo/gnuplot.rot
docs/gpcard.tex
docs/old/ChangeLog.old  [This is RCSid diff, does not count]
docs/old/makefile.r     [Ditto]
docs/old/README.3p5     [Ditto]
fnproto.h
intergra.x11
os9.c
win/wgnuplib.c
win/wprinter.c
win/wresourc.h

and then there are a bunch of ancillary changes that are being made to 
many other files that keep the psuedo-merge in sync.


> Normal CVS repositories would have every single file starting off at 
> 1.1.  I.e. the first tag would be on 1.1 revisions of every file, and 
> all development would start from there.  The conversion tools have no 
> problem at all with this set-up.
> 
> But our repository was started by a "cvs import", and received some 
> further imports after that, and that changes everything.  It means that 
> all our original files started at revision 1.1.1.1, and progressed along 
> that 1.1.1.* branch, until they were first modified.  None of them ever 
> got a tag on it 1.1 revision --- 1.1. was really never used for anything.
> 
> Every time a file that was on the vendor branch until that point (and 
> remember, for some files that still hasn't happened today!) is checked 
> in, that particular file is essentially merged over from the vendor 
> branch onto the trunk --- but in the archive this merge appears as an 
> ordinary check-in of a revision 1.2.
> 
> E.g. even though in a RCS revision tree, it appears like this:
> 
>      1.1 --*---> 1.2
>             \
>              +-> 1.1.1.1 --> 1.1.1.2
> 
> the real sequence of active revisions for gnuplot.rot is:
> 
>      1.1.1.1 --> 1.1.1.2 --> 1.2
> 
> Other files have different sequences, and different points in time at 
> which they made their transition 1.1.1.1 --> 1.1.1.2, or 1.1.1.{n} --> 
> 1.2.  Some even have branches inside the vendor branch (1.1.1.2.2.1)

Correct, but the psuedo-merges don't really care about all the different 
times, just all the files associated with a particular tag, in this case 
BETA_344_989422.  I'm not sure, but I think cvs2git in this case makes a 
rough estimate of the time which it assigns to the merge, like maybe 
halfway between the latest age of any file and the next modification. 
When a psuedo-merge took place exactly isn't important, just that it 
fits sequentially in the proper location (i.e., it must have happened 
between A and B is the important part).


> The vendor branch may best be grafted into the trunk _before_ the 1.1 
> initial import, and replace that entirely.  I.e. we might imagine that 
> the above sequence was transformed into
> 
>      1.0 --> 1.1 --> 1.2

Yes, there are multiple ways to imagine this, but I think from a 
conversion tool's standpoint it has to pick the scenario, e.g., we're 
going to assume this is a master branch and then follow all these 
branches and merges according to the tags.  It's similar to what I said 
early on that even with git, once the heads of branches are merged, it's 
sort of difficult to figure out in hindsight which particular branch was 
associated with which head at the time.


> Files with more stuff going on in their vendor branch would have to dip 
> into negative numbers, i.e.
> 
>      1.1.1.1 --> 1.1.1.2 +-> 1.1.1.3 --> 1.1.1.4 --> 1.2 ...
>                          |
>                          +-> 1.1.1.2.2.1
> 
> would (imaginatively) turn into
> 
>      1.(-2)  --> 1.(-1)  +-> 1.0 ------> 1.1 ------> 1.2 ...
>                           |
>                              +-> 1.(-1).2.1
> 
> Realistically all the revision numbers in the entire archive would have 
> to be shifted up such that the chain really does start at 1.1:
> 
>      1.1  -----> 1.2 --*---> 1.3 ------> 1.4 ------> 1.5 ...
>                          \
>                              +-> 1.2.2.1
> 
> Unfortunately, the way RCS ,v files are organized, this shift can only 
> be performed by parsing and re-encoding every revision on the vendor 
> branch.  (The direction the diffs are recorded is from the head all the 
> way down to 1.1, and from there _up_ along the vendor branch).  And 
> because of the way CVS uses RCS ,v files, every one of them has to be 
> transformed individually.

Maybe that is what cvs2git is doing.  As I said, it takes 45 minutes. 
But gosh the tree-structure and tags of cvs-fast-export matches cvs2git 
so well.  I think it is a simple matter of cvs-fast-export not 
recognizing it has to do a merge in those half dozen locations due to a 
cross-branch reference to a version number.  If reposurgeon could make 
those connections, it would be nice, but I don't think reposurgeon works 
that way, just rebasing.


> _That_ is the transformation that needs to be done in order for 
> conversion tools not to have any problems with the vendor branch.  And 
> because the transformation differs for every RCS archive, it has to be 
> done either directly on the CVS repository, or the importer has to 
> pretend it had happened that way.
> 
> Let me reiterate: to the best of my understanding, no process working on 
> an already converted git repository has any realistic chance to perform 
> this operation correctly.  It has to be done on the CVS import side.

But I said that.  cvs-fast-export has access to all that original 
information; it's the place this sort of thing should be addressed.

I suggest trying cvs2git and then explore the various changesets for the 
repository, see if it makes sense, and keep in mind this idea that the 
abundance of tags from Lars is the added information that's giving these 
merges an implied structure.

I'm willing to help, I identified all the branch points and agree with 
what cvs2git is producing in terms of psuedo-merges, tested against CVS 
etc..  I'm willing to look into cvs-fast-export (as I see it, git-wise, 
merges and branches are very similar except the branch case is like 
having one of the bases of a merge be empty, so there might not need to 
be too much code-writing).  But I didn't make the call on 
cvs-fast-export, so I think the decision of what to do here is up to 
someone else at this point.

Dan

Re: One branchpoint discrepancy

[Hans-Bernhard B. <HBB...@t-...>]

Am 30.10.2017 um 20:30 schrieb Daniel J Sebald:

>  I'd sort of like to put effort into the right place.  I've cloned the
> cvs-fast-export utility and I'm willing to help on matters, so if its 
> possible I wonder if we could enhance the merging aspect of that utility.

I rather doubt any good will come from that.  Nobody understands that 
utility anywhere near well enough to be able to modify it on short 
notice without risking total break-down.  At this point it is, 
essentially, magic.

> Yes, there is a bit of ambiguity in merges with CVS, it just isn't 100% 
> accurate to identify exactly what the programmer had checked out when 
> compiling and subsequently did a checkin.

Not really, because there _are_ no recorded merges in CVS.  If merges 
happen, they do so in somebody's working copy.  To the repository, they 
only ever appear as check-ins, without any indication whether the new 
content was created by some kind of merge, or by just writing it manually.

> There might be some ambiguity about the state of other files at this 
> point, but I would say simply recall the BETA_344_989422 state from CVS 
> and call that the state for the merge and generate all git diffs 
> accordingly, perhaps that's not the exact methodology.

That won't work at all.  The primary conflict is that CVS has individual 
branch structures for every member file, whereas git branches the entire 
repository.  Those two concepts don't mix and match.  BETA_344_989422 
may be the right join point for this particular file, but that's most 
likely the _only_ file for which that's the case.  Basically every 
single tag ever made in CVS can contain one or more file joins from the 
vendor branch onto the trunk.  Some are still waiting to happen.

Normal CVS repositories would have every single file starting off at 
1.1.  I.e. the first tag would be on 1.1 revisions of every file, and 
all development would start from there.  The conversion tools have no 
problem at all with this set-up.

But our repository was started by a "cvs import", and received some 
further imports after that, and that changes everything.  It means that 
all our original files started at revision 1.1.1.1, and progressed along 
that 1.1.1.* branch, until they were first modified.  None of them ever 
got a tag on it 1.1 revision --- 1.1. was really never used for anything.

Every time a file that was on the vendor branch until that point (and 
remember, for some files that still hasn't happened today!) is checked 
in, that particular file is essentially merged over from the vendor 
branch onto the trunk --- but in the archive this merge appears as an 
ordinary check-in of a revision 1.2.

E.g. even though in a RCS revision tree, it appears like this:

	1.1 --*---> 1.2
	       \
	        +-> 1.1.1.1 --> 1.1.1.2

the real sequence of active revisions for gnuplot.rot is:

	1.1.1.1 --> 1.1.1.2 --> 1.2

Other files have different sequences, and different points in time at 
which they made their transition 1.1.1.1 --> 1.1.1.2, or 1.1.1.{n} --> 
1.2.  Some even have branches inside the vendor branch (1.1.1.2.2.1)

The vendor branch may best be grafted into the trunk _before_ the 1.1 
initial import, and replace that entirely.  I.e. we might imagine that 
the above sequence was transformed into

	1.0 --> 1.1 --> 1.2

Files with more stuff going on in their vendor branch would have to dip 
into negative numbers, i.e.

	1.1.1.1 --> 1.1.1.2 +-> 1.1.1.3 --> 1.1.1.4 --> 1.2 ...
	                    |
	                    +-> 1.1.1.2.2.1

would (imaginatively) turn into

	1.(-2)  --> 1.(-1)  +-> 1.0 ------> 1.1 ------> 1.2 ...
  	                    |
                             +-> 1.(-1).2.1

Realistically all the revision numbers in the entire archive would have 
to be shifted up such that the chain really does start at 1.1:

	1.1  -----> 1.2 --*---> 1.3 ------> 1.4 ------> 1.5 ...
  	                   \
                             +-> 1.2.2.1

Unfortunately, the way RCS ,v files are organized, this shift can only 
be performed by parsing and re-encoding every revision on the vendor 
branch.  (The direction the diffs are recorded is from the head all the 
way down to 1.1, and from there _up_ along the vendor branch).  And 
because of the way CVS uses RCS ,v files, every one of them has to be 
transformed individually.

_That_ is the transformation that needs to be done in order for 
conversion tools not to have any problems with the vendor branch.  And 
because the transformation differs for every RCS archive, it has to be 
done either directly on the CVS repository, or the importer has to 
pretend it had happened that way.

Let me reiterate: to the best of my understanding, no process working on 
an already converted git repository has any realistic chance to perform 
this operation correctly.  It has to be done on the CVS import side.

Re: Release 5.2.1

[Ethan A M. <sf...@us...>]

On Monday, October 30, 2017 12:30:22 PM PDT Daniel J Sebald wrote:
> On 10/28/2017 02:17 PM, sfeam via gnuplot-beta wrote:
> > It is not clear when the cvs->git conversion process will settle down
> > to
> > the point where I am comfortable preparing an incremental release
> > from the new git repository.  Rather than waiting for that, I would
> > rather package up 5.2.1 from a final cvs snapshot plus a couple of
> > last-minute fixes that I will maintain on the side until the git
> > conversion completes.

> Updating CVS is fine, as the ./reconvert script gathers the latest
> version of the code.  

I wish someone had made that clear earlier.  
I asked about a freeze and thought that the response was "yes,
a freeze would be good".   OK, I'll go ahead and commit the 10-12
changesets that I applied locally to prepare the 5.2.1 tarball.

Maybe then it should be repackaged as 5.2.2?   There has been 
at least one request for a fix-to-the-fix since I packaged up 5.2.1. 

> Don't want to make the first thing you do under git be a release.

Exactly.  Also I worry that we may turn up additional glitches in the
conversion once serious develoment resumes.

	Ethan

> 
> Dan

Re: One branchpoint discrepancy

[Daniel J S. <dan...@ie...>]

On 10/30/2017 12:09 PM, Eric S. Raymond wrote:
> Daniel J Sebald <dan...@ie...>:
[snip]
>> The interesting thing is that just like cvs2git, cvs-fast-export chose to
>> tag
>>
>> [gnuplot-4-6-alpha]  update Bruce Ravel's contact info
>>
>> but failed to make that branch connection.  Anyway, this necessitates the
>> attached change to ./reconvert.
>>
>> Dan
> 
> Thanks, applied to my reconvert master.

OK, thanks.

I'm pretty sure the one major thing we have to resolve is this group of 
CVS checkins at the very start of the repository.  All those changes 
have to be merged into the master branch.  If we can address that, all 
will work out with no need for adjustments.

I'd sort of like to put effort into the right place.  I've cloned the 
cvs-fast-export utility and I'm willing to help on matters, so if its 
possible I wonder if we could enhance the merging aspect of that utility.

Yes, there is a bit of ambiguity in merges with CVS, it just isn't 100% 
accurate to identify exactly what the programmer had checked out when 
compiling and subsequently did a checkin.  But I can see the logic that 
cvs2git is following and there is some sense to it.

RULE: Basically, a merge should be declared whenever a symbol that is 
assigned to a higher-level branch (let's say trunk) references a version 
number that is associated with a lower-level branch.  Let's use 
gnuplot.rot as an example, as that is the one that Eric has applied a 
shim for.  The change associated with that shim is version 1.1.1.2:

1.1.1.2
log
@Import of beta 344.
@
text
@a0 6
# HBB: revised open-ended animation routine. Used to just turn
etc.

OK, so let's look at the CVS gnuplot.rot,v file for the symbols and 
version 1.1.1.2.  I'm including a screenshot of the qgit display of 
cvs-fast-export's output at that important would-be merge point.  Here 
are the contiguous three symbols in which 1.1.1.2 first appears, from 
the gnuplot.rog,v file:

	BETA_344_989422:1.1.1.2
	BETA_344:1.1.1.2
	BETA_343_980416:1.1.1.1

Refer to the screenshot, and notice that BETA_344 is associated with the 
branch (red), and then the first symbol after that which references 
1.1.1.2 and *appears in the trunk* (look at the screenshot, black) is 
the synthetic commit BETA_344_989422:1.1.1.2.  OK, so by the rule listed 
above, this should be declared a merge.  Now, gnuplot.rot is not the 
only file that falls in this changeset, as there are probably several 
other files in which a branch is first referenced by BETA_344_989422.

There might be some ambiguity about the state of other files at this 
point, but I would say simply recall the BETA_344_989422 state from CVS 
and call that the state for the merge and generate all git diffs 
accordingly, perhaps that's not the exact methodology.  But the general 
idea is those changes from the branch have to get merged in at some 
point if they are used by the trunk, and their first use in the trunk is 
the logical place to create a merge.  (Furthermore, in this case, but 
not of general relevance, is the fact that the comments for changesets 
surrounding BETA_344_989422 suggest that indeed things are being "merged".)

So, to summarize, looking at that screenshot PNG, we should have a merge 
connecting BETA_344 (the first reference to gnuplot.rot's 1.1.1.2 in the 
*branch*) to BETA_344_989422 (the first reference to gnuplot.rot's 
1.1.1.2 in the *trunk*).  That merge might sweep in other files, but 
whenever the rule hypothesis for any file appears, there has to be a merge.

I'll be away for most of the day, but think over the amount of effort it 
would be to address this in cvs-fast-export source code.  I'm willing to 
help if I can.  Otherwise, maybe there is some other creative way in 
reposurgeon to force a merge of the branch modifications.  (For example, 
I imagined rebasing the master on top of that red branch with Import XYZ 
changesets, but again that's effort that really isn't the best way to 
solve this.)

Dan

Re: Release 5.2.1

[Daniel J S. <dan...@ie...>]

On 10/28/2017 02:17 PM, sfeam via gnuplot-beta wrote:

> It is not clear when the cvs->git conversion process will settle down to
> the point where I am comfortable preparing an incremental release
> from the new git repository.  Rather than waiting for that, I would rather
> package up 5.2.1 from a final cvs snapshot plus a couple of last-minute
> fixes that I will maintain on the side until the git conversion completes.

Updating CVS is fine, as the ./reconvert script gathers the latest 
version of the code.  Don't want to make the first thing you do under 
git be a release.

Dan

Re: One branchpoint discrepancy

[Eric S. R. <es...@th...>]

Daniel J Sebald <dan...@ie...>:
> I've skimmed through branchpoints for both cvs-fast-export/reposurgeon
> (i.e., current ./reconvert) and cvs2git.  I found one disagreement, the
> branch-4-6-stable branchpoint.
> 
> In cvs2git branch-4-6-stable emanates from
> 
>     update Bruce Ravel's contact info
>     sfeam<>
>     11/22/11 11:12 PM
> 
> In fast-cvs-export/reposurgeon we've chosen
> 
>     Try to allow for coordinate offset caused by scrolling (Firefox, Opera,
> chrome?)
>     Ethan A Merritt<xxxxxx@xxxxxx>
>     Peter<ploxxxx@pixxxx>
> 
> which is one changeset prior to "update Bruce Ravel's contact info".
> 
> The interesting thing is that just like cvs2git, cvs-fast-export chose to
> tag
> 
> [gnuplot-4-6-alpha]  update Bruce Ravel's contact info
> 
> but failed to make that branch connection.  Anyway, this necessitates the
> attached change to ./reconvert.
> 
> Dan

Thanks, applied to my reconvert master.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

My work is funded by the Internet Civil Engineering Institute: https://icei.org
Please visit their site and donate: the civilization you save might be your own.

Re: Short vectors are not plotted

[sfeam <sf...@us...>]

On Sunday, 29 October 2017 16:02:11 Dima Kogan wrote:
> sfeam <sf...@us...> writes:
> 
> >> I.e. we convert the vector start/end to integer terminal coords; short
> >> vectors become 0-vectors in this representation, so the
> >> terminal-specific arrow() function plots nothing.
> >>
> >> This is a bug: checking for a 0-vector should happen before we convert
> >> to integer pixel coordinates.
> >
> > I am not following the logic.
> >
> > If it is a 0-length vector then isn't it correct to not draw it?
> > Even if you keep the floating point representation for a bit longer,
> > at the time you convert to integer terminal coordinates it will
> > again become 0-length and thus not drawn.  What am I missing?
> >
> > Is the idea that you want to draw an arrowhead even if the
> > shaft length is zero?
> 
> Right. It's not actually a length-0 vector, but its length is < 1 pixel
> long. We should still draw an arrowhead, and the arrowhead should point
> in the correct direction. Once we convert to integers, then even if we
> wanted to draw something, we can't do it, since the direction has been
> lost.

I don't at all like the idea of mixing (double) and (int) coordinates in the
terminal API.  Let's try to find a different solution.

First off, I think your patch only handles a subset of arrows.
It modifies the handling of "plot ... with vectors" but not the code that
handles "splot ... with vectors" or "set arrow ...".

That last one is interesting because rather than the in-line coordinate
transform and clipping code used by plot_vectors() and plot3d_vectors()
it goes through a much simpler routine draw_clip_arrow(). 
And it looks to me at first glance that the bulk of the former two code
blocks can be replaced by a call to draw_clip_arrow() also.
That at least brings all the arrow-drawing onto a common path.

Next I note that most of the callers of draw_clip_arrow are already
working with (double) coordinates. Converting the remaining ones
should be relatively simple.

At that point I think you are mostly home free. Now all you need to
do is add a flag or a new set of enums to the 5th parameter of 
do_arrow() that signals "draw only the arrowhead".   In places where
you want to guarantee that the arrowhead is drawn even if the vector
length approaches zero, you can call term->arrow() twice.
The first call is either exactly like now or perhaps we would clear
all the arrowhead flags.
The second call we dummy up a unit length vector in the desired
direction  but set the "arrowhead only" flag.

And that should do it except for the oddball terminals that have
private arrow-drawing code.  Leave that to a second round of cleanup
after the generic code path is working.

	Ethan


> 
> Here's a (hopefully convincing) thought: let's say you're looking at a
> plot of a vector field. Then you zoom out repeatedly. I would expect the
> plotted vector lengths to get shorted as you zoom out. The arrowheads
> maybe would get smaller too, depending on how they're plotted. But the
> transitions should be smooth. What currently happens is that as you zoom
> out, some subset of the vectors disappears. As you zoom out more, some
> OTHER subset disappears. This subset could be entirely different, and
> some vectors actually come back. Eventually they all disappear, but as
> they do so, sometimes they're plotted in an intermediate state where the
> arrowhead IS plotted, but its direction is completely wrong.
> 
> Try with the example attached in the last email; use the mouse wheel to
> zoom out.
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

One branchpoint discrepancy

[Daniel J S. <dan...@ie...>]

I've skimmed through branchpoints for both cvs-fast-export/reposurgeon 
(i.e., current ./reconvert) and cvs2git.  I found one disagreement, the 
branch-4-6-stable branchpoint.

In cvs2git branch-4-6-stable emanates from

     update Bruce Ravel's contact info
     sfeam<>
     11/22/11 11:12 PM

In fast-cvs-export/reposurgeon we've chosen

     Try to allow for coordinate offset caused by scrolling (Firefox, 
Opera, chrome?)
     Ethan A Merritt<xxxxxx@xxxxxx>
     Peter<ploxxxx@pixxxx>

which is one changeset prior to "update Bruce Ravel's contact info".

The interesting thing is that just like cvs2git, cvs-fast-export chose 
to tag

[gnuplot-4-6-alpha]  update Bruce Ravel's contact info

but failed to make that branch connection.  Anyway, this necessitates 
the attached change to ./reconvert.

Dan