md5cp / Blog: Recent posts

I think I have all of the bugs fixed now for version 1.0...Revision 34 is the one you'll want to pick up.

It looks like revision 31 of version 1.0 is the release that's going to work. Small bugs here and there were found by traversing the full production data tree (and copying into a non-production location). Just shy of 123,000 files, bugs were found to mishandle files that weren't named correctly (or didn't match the --field mappings). They've been handled. Performance is good; doing the initial scan of those 123K files took 69 minutes. The same run with the old PHP script took over 5 hours to complete. ... read more

There's a bug in the --dest directory if you don't specify a / at the end of the path. It will work, but the directory it creates will have all sorts of interesting crud at the end. Clearly, there's a string termination bug somewhere.

During testing, version 0.4 processed 79183 files in a little over 12 minutes. The files were a mix of small text files and larger .ps and .png files. The test was conducted using the --nocopy option, so it largely exercised the SQLite capabilities and the file I/O. Nonetheless, I'm pretty happy with the performance.

As a benchmark, the PHP script upon which this is based performed that task is about 100 minutes. Admittedly, PHP isn't as fast, and I used MySQL instead of SQLite, so there was a network and socket I/O component that added overhead.