Re: [Moosefs-users] MooseFS Recommendations

[Quenten G. <QG...@on...>]

Hi Elliot,

Thanks for the reply, I've been considering using 3TB Disks with ZFS in a 12/16 disk chassie with 2 x 6/8 disk raidz2 and using MFS for HA/speed only by using a replication factor of 2.

I guess the pros and cons so far i've worked out are,

In a distrubuted JBOD configuration which mfs seems to like, So I would have to use a replication factor of 3, I believe which would give us 3 x the High Avaliablity and higher recoverablity from bit rot/disk failure or something alike.

However In this example if we use a replication factor of 3 for example 2TB Of data, Using the MFS goal of 3 I need to store 6TB vs using a ZFS zpool I would only need to store 4TB, now multiply that by 10 and we are at 60TB for 20TB of data vs 40TB, which as you can imagine adds up very quickly.

I also would like to have a 2nd offiste replica at another datacentre and now we can muliply the sorage requirements for 20TB of data by 2. 120 vs 80 

Using the ZFS as I see it would protect us from Bit Rot/Bad Sectors & Failed Drives also reducing rebuild times as it would be handled by zfs and using MFS for high avaliblity and replication / speed (striping) what do you think?


On another note Are you using NFS or ISCSI targets? From the mfs share if I do a "dd if=/dev/zero of=ddfile bs=32k count=100000" i get around ~70mb/s however when I use iscsi or nfs i'm only getting 10-18mb/s. 

Our dev config which is 4 servers with 1 x 500gb sata drive and a 5th as metadata Server with ubuntu 11 with ext4 for our disk fs. Also Tried using FreeBSD (freenas w/pkg_add -r moosefs-client) which didn't seem to make any difference except I couldnt use NFS on that setup.

Cheers,
Quenten