|
From: David K. <dav...@al...> - 2003-09-23 14:52:55
|
Hi people! I am getting bitten by the fact that a slow writer using FTP to upload files to our server for processing using Babeldoc via the DirectoryScanner sometimes, due to timing obviously, causes the read file to be incomplete, thereby causing parsing/transformation errors and so forth. (Long sentence, yeah I know... ;-p ) To fix this, I have looked at various alternatives, including potential ftp daemon settings (nope), kernel module (not very nice), the fam thing (from SGI, old Perl module, didn't work) to avoid the read from taking place until the file was written completely. After failing enough at the above trials...I turned to our beloved Babeldoc instead, for a dirty(?) little hack: I extended the DirectoryScanner to take another config parameter 'minLastModified', default 0, specififying the least number of milliseconds that must have passed since the file's last modification timestamp, as reported by the File instance. I am working under the assumption that the modification timestamp is indeed correctly updated during the ftp upload, and it seems to be the case, at least on my Red Hat 7.2 installation where I am testing this. Using <scanner_name>.minLastModified = 5000 I have yet to see any failed reads so far. But then again, I have only tested it for about an hour or so... :-) I'll know better tomorrow what the actual effect will be. Diff (unified) against DirectoryScanner.java (per current CVS) attached. Note: I haven't made any changes to the I18N stuff, and I do use a logInfo() here and there, for debugging, so please treat the patch as a proof-of-concept needing some (hopefully little) cleanup. Any hope of getting something like this into Babeldoc proper? (Oh, yeah, I haven't had the time to properly follow up on the last mail I sent about the SqlPreparedWriter and FileTransfer stages. Sorry about that. Still pondering if it makes sense to merge the Sql thing into the standard SqlWriter, since my prepared version has no support for multiple statements nor comments in it...I'll get back to the list on that in the (hopefully) near future.) Regards, David Kinnvall |