Repository conversion

[<es...@th...>]

Something unfortunate is going on with my list subscription.  Mailman
tells me I'm subscribed, but I'm not seeing list mail; I'm having
to read this thread through the archive interface.

Ethan A Merritt writes:
>Can you summarize in what ways your conversion would be different
>from the automated cvs->git conversion service offered by SourceForge
>(which so far the project has declined to use)?

Before going into the details, I should make clear that I did not
specify a target VCS in my offer because I can up-convert to either
git or hg very easily - gnuplot can choose either.  I retain a
lingering preference for hg myself, but I advise going with git 
anyway - it presents the lowest entry to the largest number of
potential contributors.

I don't know what SF uses for its conversions; the service is a recent
invention and Google isn't turning up documentation on it for me.
There aren't many possibilities - maintained and documented tools for
this purpose are thin on the ground. I fear they are probably using
cvsps called by git-cvsimport, because that is what the git suite
ships.

Unfortunately, cvsps is so irreparably broken that I, acting as its last
maintainer, end-of-lifed it a couple of months ago. Mojca Miklavec can
confirm what a mess it is; she tried doing  a test conversion and got
horrible results.  The git devs don't realize how bad the situation
is.

Through an odd set of circumstances, last year I wound up as the
maintainer of both cvsps and Keith Packard's parsecvs code, which I
rewrote significantly and renamed cvs-fast-export.  After EOLing
cvsps, I shifted my effort to cvs-fast-import, which is what Mojca
Miklavec has been doing her most recent test conversions with.

cvsps and cvs-fast-export are two of only three possibilities SF
might be using.  The third is cvs2git, which has some problems of
its own that I am trying to help its maintainer fix.  

All three of these tools, by themselves, are too weak to produce 
a really high-quality conversion.  The things they don't do include:

1. Lifting CVS version references in comments into a form that
will still be usable in git or hg.

2. Complete coalescence of CVS change cliques into changesets.  They
often only do this partially, one reason being that the default
merge window is set too low.

3. Mapping of .cvsignores to .gitignores.  My newer versions of
cvs-fast-export do this but neither cvs2git nor the cvsps version in
the git suite does.

4. Cleaning up conversion artifacts and junk branches. I can't be
more specific about this in advance because each CVS repo tends to
have its own unique set of strange malformations.  Fortunately, 
gnuplot's history seems exceptionally clean and I expect relatively
few problems.

5. (Optional) Canonicalizing change comments to git summary +
continuation form.

Completing these tasks requires human judgment. The only way to get a
really good conversion is to follow up application of a batch
converter with skilled hand-editing, and reposurgeon is the only
existing tool for that.  It's an interpreter for a domain-specific
language built around repository-editing primitives. 

(As previously noted, I am currently in the process of lifting Emacs's
history from bzr to git with reposurgeon.  That is an exceptionally
large and messy conversion, far more complex than I expect gnuplot's
to be.)

I have done about half a dozen large conversions before; the most
recent completed one was GNU troff.  You can read about my procedures
in the DVCS Migration HOWTO:

   http://www.catb.org/esr/dvcs-migration-guide.html

Ideally, I team with an inside developer who knows the project's
history and idiosyncracies and is highly motivated to make the
conversion succeed.  For this project, Mojca Miklavec has claimed that
role - she has already supplied one of the key pieces of metadata, a
map from developers' CVS usernames to full names and email addresses.

If the project elects to go ahead, our work product would be a lift
script - that is, a file containing surgical instructions to
reposurgeon - which would be available for review before the actual
conversion day.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Where rights secured by the Constitution are involved, there can be no
rule making or legislation which would abrogate them.
        -- Miranda vs. Arizona, 384 US 436 p. 491