[Lessfs-users] Very slow performance lessfs 1.5.12 with hamsterdb 2.0.4.

[Dominic R. <do...@ti...>]

I am experimenting with lessfs on Ubuntu 12.04 using the configuration 
suggested by Mark at http://www.lessfs.com/wordpress/?p=707, but with 
lessfs 1.5.12 and hamsterdb 2.0.4 (and snappy 1.0.5). I have used Mark's 
lessfs.cfg file (http://www.lessfs.com/wordpress/wp-content/lessfs.cfg) 
but as I am testing with a single-processor VM I set MAX_THREADS=1. The 
machine has 1GB RAM.

I set about copying (rsync) some data (a lot of files, mostly quite 
small Word or pdf files) into the lessfs mount. Things go quickly for a 
while (and there are pauses every so often which I guess are when lessfs 
flushes data to disk) but then become very slow indeed (say 1 x 30K file 
per second), and never speed up again.

This seems similar to the problems in a previous posting 
(https://groups.google.com/forum/?fromgroups#!topic/lessfs/cA2M6Czc3Us) 
but that was where tokyocabinet was being used, and the suggestions 
relate to _BS configuration options which don't seem apply when using 
hamsterdb: man lessfs states: 'The hamsterdb database does not require 
the  _BS lines since hamsterdb does not use hash databases'. Does this 
apply to all *_BS configuration options, I presume so because none are 
set in Mark's lessfs.cfg?

Size of underlying datastore (sudo du -h --summarize /data): 589MB
Size of stored files (sudo du -h --summarize /mnt/less): 869M (62 min to 
calculate this!!!)
Number of stored files (sudo find /mnt/less -type f | wc -l): 22675

Can anyone suggest why lessfs is so very slow on my system and how I can 
improve it?

I have tried starting lessfs with '-o 
big_writes,max_read=131072,max_write=131072', or not: doesn't seem to 
make any difference to performance, maybe the setting BLKSIZE=131072 in 
lessfs.cfg means these settings are used anyway?

Thanks for any help

Dominic
-- 
*TimeDicer* <http://www.timedicer.co.uk>: Free File Recovery from Whenever