Re: One branchpoint discrepancy

[Daniel J S. <dan...@ie...>]

On 10/31/2017 02:02 PM, Eric S. Raymond wrote:
> Daniel J Sebald <dan...@ie...>:
>>> I rather doubt any good will come from that.  Nobody understands that
>>> utility anywhere near well enough to be able to modify it on short notice
>>> without risking total break-down.  At this point it is, essentially,
>>> magic.
>>
>> It doesn't look like a very big program.
> 
> No, it doesn't.  What used to be one large lump of incomprehensibility
> has shrunk in size and apparent complexity as I carved pieces off the
> outside, narrowing their interfaces to the central core.
> 
> Those outside bits are indeed pretty small and comprehensible, now.
> The most notable success was a clean refactor of the front-end code
> for digesting RCS masters that enabled it to use thread-per-master
> parallelism with a re-entrant parser.  That's how the program went
> from respectably fast to ridiculously fast.
> 
> Unfortunately, there remain two knots of mystery in the code that
> nobody has been able to crack.  The greater one one is the actual
> changeset generation in merge.c - the black hole of incomprehension is
> the functions merge_to_changesets() and merge_branches(), which solve
> the general changeset-synthesis problem HBB accurately described.  The
> lesser mystery is cvs_master_patch_vendor_branch(), which is probably
> where our problem is.

Yes, I started out looking at merge.c and merge_branches().  It walks 
backward from the start of heads of file branches stitching them 
together to create a git branch.  I printed out the number of branches:

find . -name '*,v' -print | cvs-fast-export --reposurgeon > /dev/null
MERGED SOME BRANCHES 1257
MERGED SOME BRANCHES 338
MERGED SOME BRANCHES 407
cvs-fast-export: warning - branch point branch-pre-3-7-1 -> import-1.1.1 
matched by date
MERGED SOME BRANCHES 672
cvs-fast-export: warning - branch point branch-4-2-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 740
cvs-fast-export: warning - branch point branch-4-4-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 795
cvs-fast-export: warning - branch point branch-4-6-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 805
cvs-fast-export: warning - branch point branch-5-0-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 789
MERGED SOME BRANCHES 553
cvs-fast-export: warning - branch point column -> import-1.1.1 matched 
by date
MERGED SOME BRANCHES 544
cvs-fast-export: warning - branch point branch-4-0-stable -> 
import-1.1.1 matched by date
MERGED SOME BRANCHES 529
cvs-fast-export: warning - branch point pm3d -> import-1.1.1 matched by date
MERGED SOME BRANCHES 507
cvs-fast-export: warning - branch point axis_branch_base -> import-1.1.1 
matched by date
cvs-fast-export: no commitids before 2017-10-31T04:47:09Z.

Those number of branches suggests to me the first step in this process 
is to build a branch/linked-list for all individual files, i.e., the 
file.ext,v files as the source.  Then to build a branch means "merging" 
all those in a meaningful way to create the git branch--something 
involving "clique" determines the root of the branch.  Having reached 
that point, then the git branch is attached to its parent, i.e., branch 
point.

The issue is that there is no consideration in this clique process that 
includes the symbol "version stamps" identifying cross-branch file use. 
(Instead it is solely based upon date, which means cvs-fast-export 
concludes so many of these branches go back to the import/vendor branch 
as the branchpoint.  See warnings above about dates matching.)  The 
information about branches there, all neatly saved in the branch C++ 
structures; it's just not used.  I really like the C++ structure of the 
code, e.g., the way that CVS numbers are stored as little objects for 
which A-to-B comparisons can be done easily.  It's just a question of 
wanting to set about doing that.

However, the sense in which branch_merge() considers "merge" isn't the 
same notion as a git "merge".


> Also, fast actually matters.  cvs2git probably comes the closest to
> cvs-fast-export in terms of correctness and robustness across weird
> cases.  But it's so slow that the kind of iterative refinement I'm
> doing by repeatedly tweaking the reconvert script is not really
> practical - I belive you noticed 45 minutes of lag, as opposed to 20
> seconds on my desktop.  That's a problem that's getting worse over
> time, because the small, clean CVS repos have already been moved.
> Increasingly it's only the large, old, nasty ones that are left.

Yes, cvs2git is slow, sort of brute force creating some giant blob.

Dan