From: a. <ai...@qq...> - 2011-04-14 15:52:13
|
Dear guys, We are plan to use moosefs as our store mechanism, which will be more than 5TB off-line command log. But now we have someconcern, Please find my queries below: ----- 1. is there any plan about resolving the single-point failure of master ? 2. The data will lost if master crash without dumping metadata from RAM into disk,is it right? 3. How can I know which data were lost if the question #2 happend? 4. Dose the master only caches metadata read-only or will update the data in RAM and flush to disk periodically? 5. Are the change logs based on the metadata.mfs.back or the data in RAM ? 6. Can I change the default dumpling behaviour for master metadata from per hour to user defined value? Such as 30 min? 7. How can we make sure no data lost at any time with HA(KeepAlived)+ DRBD + mfs solution ? 8. The mfsmetaloggr didn't re-connect automatically even though I had stoped and restarted the mfsmater, can only by manual? 9. which option in mfsmaster.cfg controls the rotation of changelog ? according to the size ? or each time when mfsmaster dump the metadata ? Any help will be appreciated. Looking forward your answer. Best Regard Bob Lin |
From: Michal B. <mic...@ge...> - 2011-04-15 12:18:57
|
Hi Bob! My replies are in the text. And for start please read this article: http://www.moosefs.org/news-reader/items/metadata-ins-and-outs.html From: ailms [mailto:ai...@qq...] Sent: Thursday, April 14, 2011 5:52 PM To: moosefs-users Subject: [Moosefs-users] Questions about MooseFS before putting it on the production env Dear guys, We are plan to use moosefs as our store mechanism, which will be more than 5TB off-line command log. But now we have someconcern, Please find my queries below: ----- 1. is there any plan about resolving the single-point failure of master ? [MB] Please check a project developed by Thomas Hatch: https://github.com/thatch45/mfs-failover It still needs some improvements but is a great way to test the failovers. And Thomas needs some beta testers. 2. The data will lost if master crash without dumping metadata from RAM into disk,is it right? [MB] Changelogs are saved continuously on the fly, there is very little probability you would lose your data upon crash of the master server 3. How can I know which data were lost if the question #2 happend? [MB] When you make a recovery process of metadata you would know if it was successful or not 4. Dose the master only caches metadata read-only or will update the data in RAM and flush to disk periodically? [MB] The changes to metadata are saved to changelogs continuously on the fly 5. Are the change logs based on the metadata.mfs.back or the data in RAM ? [MB] We can consider them as based on RAM 6. Can I change the default dumpling behaviour for master metadata from per hour to user defined value? Such as 30 min? [MB] As far as I remember, yes you can 7. How can we make sure no data lost at any time with HA(KeepAlived)+ DRBD + mfs solution ? [MB] We have not done such tests. Please first check a solution given in question 1. 8. The mfsmetaloggr didn't re-connect automatically even though I had stoped and restarted the mfsmater, can only by manual? [MB] Should have reconnected automatically. What was your environment? 9. which option in mfsmaster.cfg controls the rotation of changelog ? according to the size ? or each time when mfsmaster dump the metadata ? [MB] Please see "man mfsmaster.cfg” - you should have all the options listed Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 Any help will be appreciated. Looking forward your answer. Best Regard Bob Lin |
From: 颜. <rw...@12...> - 2011-04-16 02:19:11
|
Unfortunately, if the master crashed when some client is writing data to MFS, the data is lost absolutely. If we storage VM disk img file with MFS, when master crash, this img file was destroyed and u can't boot this vm with that img file. That means, we lost this vm permanently. 发件人: Michal Borychowski 发送时间: 2011-04-15 20:19:37 收件人: 'ailms' 抄送: 'moosefs-users' 主题: Re: [Moosefs-users] Questions about MooseFS before putting it onthe production env Hi Bob! My replies are in the text. And for start please read this article: http://www.moosefs.org/news-reader/items/metadata-ins-and-outs.html From: ailms [mailto:ai...@qq...] Sent: Thursday, April 14, 2011 5:52 PM To: moosefs-users Subject: [Moosefs-users] Questions about MooseFS before putting it on the production env Dear guys, We are plan to use moosefs as our store mechanism, which will be more than 5TB off-line command log. But now we have someconcern, Please find my queries below: ----- 1. is there any plan about resolving the single-point failure of master ? [MB] Please check a project developed by Thomas Hatch: https://github.com/thatch45/mfs-failover It still needs some improvements but is a great way to test the failovers. And Thomas needs some beta testers. 2. The data will lost if master crash without dumping metadata from RAM into disk,is it right? [MB] Changelogs are saved continuously on the fly, there is very little probability you would lose your data upon crash of the master server 3. How can I know which data were lost if the question #2 happend? [MB] When you make a recovery process of metadata you would know if it was successful or not 4. Dose the master only caches metadata read-only or will update the data in RAM and flush to disk periodically? [MB] The changes to metadata are saved to changelogs continuously on the fly 5. Are the change logs based on the metadata.mfs.back or the data in RAM ? [MB] We can consider them as based on RAM 6. Can I change the default dumpling behaviour for master metadata from per hour to user defined value? Such as 30 min? [MB] As far as I remember, yes you can 7. How can we make sure no data lost at any time with HA(KeepAlived)+ DRBD + mfs solution ? [MB] We have not done such tests. Please first check a solution given in question 1. 8. The mfsmetaloggr didn't re-connect automatically even though I had stoped and restarted the mfsmater, can only by manual? [MB] Should have reconnected automatically. What was your environment? 9. which option in mfsmaster.cfg controls the rotation of changelog ? according to the size ? or each time when mfsmaster dump the metadata ? [MB] Please see „man mfsmaster.cfg” – you should have all the options listed Kind regards Micha┅ Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wo┅oska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 Any help will be appreciated. Looking forward your answer. Best Regard Bob Lin |
From: Giovanni T. <me...@gi...> - 2011-04-16 08:30:09
|
On Sat, Apr 16, 2011 at 4:15 AM, 颜秉珩 <rw...@12...> wrote: > If we storage VM disk img file with MFS, when master crash, this img file > was destroyed and u can't boot this vm with that img file. > That means, we lost this vm permanently. Metalogger exists to have a recently consistent metadata backup. If you use metalogger's metadata, you should get your VM images immediately before the crash (in this specific situation, you probably need to restart VMs). -- Giovanni Toraldo http://gionn.net/ |
From: Thomas S H. <tha...@gm...> - 2011-04-16 14:53:20
|
I have been able to cleanly failover to a metalogger when the master crashes and keep vms running. My schedule will be opening up soon and I will FINALLY be able to clean up and finish my failover suite so that it just runs. -Thomas S Hatch On Sat, Apr 16, 2011 at 1:59 AM, Giovanni Toraldo <me...@gi...> wrote: > On Sat, Apr 16, 2011 at 4:15 AM, 颜秉珩 <rw...@12...> wrote: > > If we storage VM disk img file with MFS, when master crash, this img file > > was destroyed and u can't boot this vm with that img file. > > That means, we lost this vm permanently. > > Metalogger exists to have a recently consistent metadata backup. If > you use metalogger's metadata, you should get your VM images > immediately before the crash (in this specific situation, you probably > need to restart VMs). > > -- > Giovanni Toraldo > http://gionn.net/ > > > ------------------------------------------------------------------------------ > Benefiting from Server Virtualization: Beyond Initial Workload > Consolidation -- Increasing the use of server virtualization is a top > priority.Virtualization can reduce costs, simplify management, and improve > application availability and disaster protection. Learn more about boosting > the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > |