Search Results for "algorithm"

NO LONGER MAINTAINED, NO LONGER SUPPORTED Manifest Maker is a graphical Python application which takes a file or group of files and creates a plain text manifest list of each item. The manifest includes the file name (including directory structure) as well as a checksum of the file.

...This is basically a reimplementation of dupfinder as: it doesn't compile on my computer, the executable won't work either, the projects seems dead and I didn't wanna work through the original code. So Look'n'Feel should be somewhat the same, though some changes have been made. Algorithm is simple and suitable for large file sets with few differences: basic set of duplicate candidates is determined by file size, then candidates are compared byte-wise. So for files which don't start to differ by growing at the tail it reduces read operations greatly compared to hash-based comparision (i.e. test case: 40GB of files, 16GB RAM, many differences: initial compare (uncached by OS) around 300sec, subsequent compare (relevant file parts cached by OS) ca. 2sec) Note: this program is not related to lsdup

The project will develop an in – memory object cache utilizing the LRU (least-recently-used) algorithm. The cache will store the objects that are most accessed within a fixed size memory pool. Thus system is keep statistics and calculates tradeoffs.