Re: [MooseFS-Users] Xor replication

[Daniel F. B. <df...@it...>]

That's great news, space efficiency with no penalties.
We are starting with 4 servers, but the initial plan is to rapidly grow 
to 12 with a 3 goal, so perfect.
Keep us informed when there are something to test.

Thank again

Daniel


On 07/27/2016 09:50 AM, Jakub Kruszona-Zawadzki wrote:
> On 26 Jul, 2016, at 15:44, Daniel Feliciano Branco <df...@it...> wrote:
>
>> Hi all,
>>
>>   We are testing moosefs for our backup/storage service (brilliant so
>> far, thank you for sharing your hard work).
>>
>>   On version 3.0.1-1 (2015-01-28) it was added xor-replication but I'm
>> unable to find any information on the subject. Other then the hard code
>> on github.
> This was added just "for future use". Now we are working hard to bring "raid" feature in upcoming MooseFS 4.x.
>
>>   Is it possible to test this functionality? As I understand it, you can
>> have a raid5 (or 6) space efficiency and performance penalty, right?
>>
>>    Can any one shed some light on this issue?
>>
>>   Thank you
>>
>> Daniel Branco
> I hope that there will be no performance penalties in case of using "raid". Our approach is that files will be written using classic goal (few copies) and then after a few days automatically changed to "raid"-like method. This means that all temporary files will be stored in copies (such files occupy relatively small amount of space), but all the rest will be stored in more space-efficient "raid"-like method (and those files usually occupy large amount of space). In such approach I hope that we will have no performance penalties and will be able to save a lot of space.
>
> Our "raid" system is designed to have up to 9 checksum per 8 blocks. 8 blocks and 1 checksum will give safety similar to goal 2, 8 + 2 will give safety of goal 3 and so on. Probably the recommended method will be 8+2 (files occupy 10/8 = 125% of their space with safety of goal 3). Of course you will need at least 12 chunkservers to achieve that. Our "raid" is designed for rather "large" moosefs installations, because mainly in large (hundreds of TB) installations you need to save space. In small installations it doesn't really matter.
>
>> -- 
>> Serviço de Informática - http://si.itqb.unl.pt
>> Instituto de Tecnologia Quimica e Biológica
>> Universidade Nova de Lisboa - http://www.itqb.unl.pt
>> Avenida da República
>> Apartado 127
>> 2781-901 Oeiras
>> Portugal
>> Tel. (+351) 21 446 9470
>>
>>
>> ------------------------------------------------------------------------------
>> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
>> patterns at an interface-level. Reveals which users, apps, and protocols are
>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>> J-Flow, sFlow and other flows. Make informed decisions using capacity planning
>> reports.http://sdm.link/zohodev2dev
>> _________________________________________
>> moosefs-users mailing list
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users


-- 
Serviço de Informática - http://si.itqb.unl.pt
Instituto de Tecnologia Quimica e Biológica
Universidade Nova de Lisboa - http://www.itqb.unl.pt
Avenida da República
Apartado 127
2781-901 Oeiras
Portugal
Tel. (+351) 21 446 9470