[Lxr-dev] [ lxr-Bugs-3520299 ] Incorrect DB purge with --reindexall

[SourceForge.net <no...@so...>]

Bugs item #3520299, was opened at 2012-04-22 05:56
Message generated for change (Tracker Item Submitted) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3520299&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Database interface
Group: current cvs
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Incorrect DB purge with --reindexall

Initial Comment:
Applies to all versions of LXR, probably since 0.9.3 in 2004!

Statements delete_releases and delete_files have implicit interrelatioship: a "release" is some alias to a base "revision" file. A base "revision" can thus only be deleted when all "releases" pointing to it have been destroyed. In the present implementation, "releases" are first deleted. Then base "revisions" are scanned: at that time, there is not any more "releases" thus the delete statement for table 'files' can't succeed since it can't compared the files' fileid to releases' (which no longer exist).

This leaves the 'files' records in the DB. In the case of the kernel, that amounts to more than 37000 items per version.

When reindexing, it is very likely that the new "revision" for the file will be different (it is almost certain with the VCSes, only slightly probable with plain files). This means the orphaned "revisions" will not be reused and new records will be created. DB grows uselessly and performance is impacted.

Even without that, the whole process is too slow (same order of magnitude as the indexing itself!) since it involves a selective query with references to several tables.

The first issue ('files' table) could be resolved with a reference count.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3520299&group_id=27350