Re: New spin of repository conversion

[Daniel J S. <dan...@ie...>]

On 10/18/2017 05:10 PM, Eric S. Raymond wrote:
> "Bastian Märkisch" <bma...@we...>:
>> Thank you for the new version.  Overall the attribution algorithm
>> seems to work remarkably well! There are a number of cases though
>> were it fails.
>>
>> Here's an example (picked more or less randomly):
>> https://sourceforge.net/p/gnuplot/git-main/ci/26f769db9e954052b11dbc8f242fc35a3cfd6880/
>> This one is attributed to Ethan, although the change is by me.
>> The reason seems to be that the Changelog entry is inserted below
>> an entry by Ethan on the same day.  I think that is not an uncommon
>> case, in fact I quickly found around a dozen such cases.  Can this
>> be taken into account or do we have to check manually?
> 
> Inserting an entry *below* the previous one defeats my naive algorithm, which
> simply assumed that the topmost entry in the ChangeLog is the most recent and
> most relevant one.
> 
> I'm open to suggestions about a better algorithm, but...if it's not to assume
> that the topmost entry is the right one, what rule should be used?  Under
> what circumstances do attributions get inserted below that?
> 
> You may have to fix these manually.

See the code associated with the utility here:

https://sourceforge.net/p/gnuplot/patches/_discuss/thread/e85f41d6/97ca/attachment/git_changelog_author.cc

Basically,

1) Find first change in changelog
2) Search backward to find the author info
3) Heuristics:
    a) Author info can't be a *subtraction*, i.e., a '-' in first column
       (that indicates wholesale swap of ChangeLog file or
        corrected author info)
    b) If it is only the ChangeLog that was modified, do nothing
       (that indicates the committer simply modified like a date
        or typo in the ChangeLog, i.e., the scenario Peter
        described)

The above may not be exactly what the code is doing, but it is close, 
with nuance.

The search requires, of course, that backward from the first change 
there is going to be adequate author information.  I.e., that the 
ChangeLog message isn't going to be so long that the author info is 
dropped.  That is why I used

git log --format='commit <%cE!%cIZ>' -p --unified=50 ChangeLog > CL.diff

That is, give me 50 lines prior to the first diff hunk, with the 
assumption that no one has written a ChangeLog message 50 lines long.


>> Another idea concerning the CVS conversion:  at the early stages of
>> the gnuplot CVS repo, files were moved from the top directory to
>> sub-directories. That is they were deleted and added to CVS again.
>> Hence the history before the move is kind of lost.  While not super
>> important, I wonder if this could be easily ammended now?
> 
> Can you be more specific about the time this happened, and in what way the
> git conversion fails to reflect the old history?
> 
> The way cvs-fast-export works *may* already solve this problem.  It scans
> masters in the attic directory - it has to,  because that's where deleted
> files go. In fact attic files are treated pretty much as though they were
> at their original, pre-attic locations, so if a commit clique is found by
> matching time and comment it does not matter that some of the file
> revisions are in the attic and some are not.

Most VCS's didn't "move" files, up until git, et al.  And I'm not 
exactly sure even git technically has an action "move".

Dan