ChangeLog mining doesn't work

[<es...@th...>]

Alas, automated ChangeLog mining to fill in author slots doesn't work
well enough to be usable.

The algorithm I ended up implementing  digests the ChangeLogs into a
set of tuples each consiring of a date stamp (with no time part), a
contributor ID, and a set of paths.  It then walks through the entry
list, looking for commits that match the path set.  Then it filters
for commits close in time to the entry timestamp.

"Close" is by default 12 hours to either side of the commit stamp.  The
window can be extended on the future side to allow for delay in
merging patches.

$ reposurgeon "read ." "changelogs"
reposurgeon: Fills 123 of 49627 authorship slots from 4915 ChangeLog entries
$ reposurgeon "read ." "changelogs 36"
reposurgeon: Fills 468 of 49627 authorship slots from 4915 ChangeLog entries
$ reposurgeon "read ." "changelogs 72"
reposurgeon: Fills 498 of 49627 authorship slots from 4915 ChangeLog entries
$ reposurgeon "read ." "changelogs 120"
reposurgeon: Fills 515 of 49627 authorship slots from 4915 ChangeLog entries

The argument of changelogs is a count of hours to extend the
closeness window by. As expected, increasing the window yields
more matches.

However, even with a very long window the match rate never goes above
1% of commits. That is noise level.

I think the reason is hinted at by the large disparity (about 10:1)
between commit cliques and ChangeLog entries.  What this tells us is
that a typical ChangeLog entry corresponds not to one commit clique
but to several.  There's no algorithmic way to know what the boundaries
are.

If such annotations are going to be made they will need a human eye
and hand comparing at each of 4916 ChangeLog entries against the
commit history.

That could be done with reposurgeon; I ran some numbers and it's
probably about 80 hours of hand-work. No thanks...
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

A true libertarian supports free enterprise, opposes big business;
supports local self-government, opposes the nation-state; supports the
National Rifle Association, opposes the Pentagon.	-- Edward Abbey