#394 Post-Python 3: Transliterated values and Web pages are not always de-duplicated during Publisher Merges

[Ahasuerus]

Transliterated values and Web pages are not always de-duplicated during Publisher Merges. It can only happen when merging 3 or more publishers. The scenario is as follows below.

The "kept" publisher record is always the publisher record with the lowest Publisher ID. Suppose the second and the third publisher records have transliterated values and/or Web pages which they share, but the first publisher record does not have. Given this scenario, the shared values will be associated with the first record twice. This happens because mod/ua_merge.py checks the values which were associated with the first record before the merge started with the values associated with the second and the third publisher records. It works correctly for the second publisher record, but not for the third one.

This problem can be fixed by changing the code to add each value which was added to the first record's data to the in-memory representations of the first publisher record's data. A one-time cleanup script will likely be needed to clean up pre-existing bad records.