If you add the same URL more than once, the favicon is also added more than once.
Is it possible to implement a process that prevents duplicate entries?
e.g. by hashing the icons?
There should already be a process to prevent this from happening. I know it is imperfect but I have previously only seen problems when comparing the output from 32bit machines and 64bit machines. I can't explain why that happens but would have expected it to only occur once (when swapping between 32bit and 64bit or vice versa) so that there are at most two copies of each icon and once swapping between 32 and 64 bit ends I would expect deleted duplicates to remain deleted (provided that the "correct" duplicate is deleted).
So if you have recently swapped between 32bit and 64bit windows you might be seeing this same issue. Or maybe there is something else that can cause the comparison between the downloaded icon file and the stored icon to fail?