[Moosefs-users] 97% fsync for # of ops at I/O stats

[Markus K. <mar...@tu...>]

We are testing moosefs in combination with gridengine and getting about 97% 
fsync write operations. The only think which get written to moosfs is stdout 
stderr and at the end of a job a small file. The stdout/stderr output is max. 
10MB/hour and job. It seams that each output line gets a synced write 
operation.

Our chunkserver are mainly desktop hosts with an extra disk for the 
chunkserver (100-900GB). With 200 jobs running we see an average iowait of 
>20% in the hosts with the huger disks which makes the hosts unusable. Today 
i have installed an extra chunkserver with 2 2TB disks on our metalogger 
which reduces the iowait on the clients to about 5%. Nevertheless the 
metalogger host is now suffering 25% iowait. Together we have now 32 disks in 
usage for chunkservers.

Whats the best strategy to avoid this synced writes?
If I copy a huge file to moosefs I do not get a noticeable load on the 
clients.

We are using debian squeeze, kernel 3.2.0 and mossefs 1.6.24.
For chunkserver we use ext4 with mount options rw,noatime,nodiratime


Thanks
Markus Köberl
-- 
Markus Köberl
Graz University of Technology
Signal Processing and Speech Communication Laboratory
E-mail: mar...@tu...