How it Works

First the tool queries the Perforce SCM for how many changes have already been processed. It then retrieves all available changelists after this point. It then gathers information about each filechange (file-revision-action). This includes how many lines the file has, diffs between previous revisions, where the file was branched from, etc. After this data has been gathered, it proceeds to process all files it has determined to be text files.

How it Chooses Files

One of the hard problems in attributing ownership is first determining which files are interesting to look at. One would think you could just look to see if a file was marked text or not, however this fails for unicode files, and times when users simply marked the file incorrectly. The current process for determining which files should be tracked is primarily based around the extension of the file.

• The Extension
  
  
  If the extension is not recognized, the user is prompted if this extension is a "text" file type. The result of this interaction is written to the extension_types table. Various metrics are employed to ignore "backup" auxillary extensions (i.e. .backup, or .orig), files that begin with a dot, and files that have no extension to properly distinguish between then.

• Nulls and Upper ASCII
  
  
  When printing files we keep track of if the file has any nulls and how many upper ASCII the file contains. This is used to determine if the file may be binary, regardless of our extension rules. If we determine the file may be binary we prompt the user if the file is actually binary or not as some valid text files may have embedded japanese and/or nulls. Depending on the user's response we write an entry to either the binary_files table, or text_files table. This entry is actually a user editable regular expression which allows for the entry to take affect across branches.

• Big Files
  
  
  If the file is bigger that 2.5MB, we also prompt the user to see if this file should be excluded from the viable files or not. This is to primarly catch log files which cause needlessly long diffs. Based on the user response the file pattern regex is written to either big_files, or binary_files.

Based on all the above factors files are either automatically excluded, or included to avoid futher user interaction.

How Lines are Determined

Since are are various scenarios which one can encounter files which do not have their EOLs normalized this is the first step performed. A simple regex replace on "\r\n|\n\r|\r" to "\n" is performed. This is paramount for determining ownership across revisions.

How User Diffs & Ownerships are Calculated

Attributing ownership and differences to a user is dependent on the FileAction (file and action against a file). The following illustrates how it is calculated based on FileAction.

• Add/Delete: If the file was added/deleted, we add to the user diffs table (linediffs) that the user added/deleted all the lines the file has/used to have. Furthermore, if the file was added, and the same as the revision-2, we carry the ownership information from the revision-2 file to the added file, otherwise we mark the whole fileas owned by the user who added the file.

• Branch: If the file was branched, we first check to see if there are any diffs between the target and source (diffFromTo). If there are diffs these diffs are used to determine ownership, otherwise the ownership is carried from the source of the branched file.

• Edit: Similarly if the file was edited, we use the diffs between the file and the previous revision of the file, as well as the ownership of the previous revision of the file edited to determine ownership of the file after the edit.

• Integrate/Import: If the file was integrated or imported, we first do a diff between the file and the previous revision of the file (diffMinus) to determine what changed. We then create a temporary ownership table based on the previous revision, apply the diffs to obtain an ownership with the right number of lines, and mark the diff'd regions as owned by an invalid user. We then look through all the diffs in the following manner:

  • If a diff was an add or a change, we go line by line through the diff to see if if any line matches any of the diffs of the source of the integration to the target (diffFromTo). If any line of a diffMinus matches a diff of diffFromTo the line is added to linediffs as user edit, and the line is marked in the ownership table as owned by the user of the integrate. Otherwise the ownership of the source of the integrate is propagated to the target of the integrate.

  • If a diff was a delete, we go line by line through the diff to see if any line does not intersect with diffFromToMinus (diff of source of integrate, to revision before target of integrate), or of the intersection is not an "add" region. If so, then the line is marked as a userDiff. This is to verify the integrate cause the region to be removed.

At the end of these operations the collated user diffs are written to the linediffs table, and the updated ownership table is written to ownerships. There are also subtilties like when the associated file is not available, or has been purged. Generally in these cases the files are marked with unknown ownership.