Menu
▾
▴
moosefs-users
|
From: Heiko S. <sch...@iu...> - 2011-02-24 08:42:13
|
Hello, we are currently investigating moosefs as a successor of our 200TB storage cfs. mfs-1.6.20, 2.6.36-gentoo-r5, x86_64, fuse 2.8.5 Everything is working fine. We have a question about the way mfs handles chunks. Is it possible to keep the chunks on a single chunkserver, instead of "load balance" them to all chunkservers ? Reason is that in case of a total unrecoverable loss of a single chunkserver we would loose some files completly. But that would be better to us than loosing some parts in all files. Incrementing the goal is not an option since the storage capacity is limited. Thanks and Regards Heiko |
|
From: Laurent W. <lw...@hy...> - 2011-02-24 08:54:24
|
On Thu, 24 Feb 2011 09:24:11 +0100 Heiko Schröter <sch...@iu...> wrote: > Hello, Hi, > > we are currently investigating moosefs as a successor of our 200TB storage cfs. > mfs-1.6.20, 2.6.36-gentoo-r5, x86_64, fuse 2.8.5 > Everything is working fine. > > We have a question about the way mfs handles chunks. > Is it possible to keep the chunks on a single chunkserver, instead of "load balance" them to all chunkservers ? Not that I know of. mfs is designed for reliability, and working with goal=1 is as bad as raid 0 when it comes to it. I don't think such a feature is planned, but I'm not part of mfs dev team :) Regards, -- Laurent Wandrebeck HYGEOS, Earth Observation Department / Observation de la Terre Euratechnologies 165 Avenue de Bretagne 59000 Lille, France tel: +33 3 20 08 24 98 http://www.hygeos.com GPG fingerprint/Empreinte GPG: F5CA 37A4 6D03 A90C 7A1D 2A62 54E6 EF2C D17C F64C |
|
From: Heiko S. <sch...@iu...> - 2011-02-24 09:08:59
|
Am Donnerstag 24 Februar 2011, um 09:54:14 schrieb Laurent Wandrebeck: > On Thu, 24 Feb 2011 09:24:11 +0100 > Heiko Schröter <sch...@iu...> wrote: > > > Hello, > Hi, > > > > we are currently investigating moosefs as a successor of our 200TB storage cfs. > > mfs-1.6.20, 2.6.36-gentoo-r5, x86_64, fuse 2.8.5 > > Everything is working fine. > > > > We have a question about the way mfs handles chunks. > > Is it possible to keep the chunks on a single chunkserver, instead of "load balance" them to all chunkservers ? > Not that I know of. mfs is designed for reliability, and working with > goal=1 is as bad as raid 0 when it comes to it. Thks for the reply but that was not my intention to ask. I try to keep the "pieces" (chunks) of a file on a single chunkserver. And yes, i'am quite clear about the risks when setting the goal=1. But this will affect the number of copies of the whole file, as far as i understand it. This what we have now with our lustre system anyway ;-) Our chunkserver (Raids) are generally running a hardware raid6 with 5 to 8TB per partition. I wouldn't mind if all the chunks are spread across theses partitions, as long as they stay on a single chunkserver. So, would it be possible to gather all chunks of a single file on a single chunkserver ? Regards Heiko |
|
From: Anh K. H. <ky...@vi...> - 2011-02-24 09:01:17
|
On Thu, 24 Feb 2011 09:24:11 +0100 Heiko Schröter <sch...@iu...> wrote: > we are currently investigating moosefs as a successor of our 200TB > storage cfs. mfs-1.6.20, 2.6.36-gentoo-r5, x86_64, fuse 2.8.5 > Everything is working fine. > > We have a question about the way mfs handles chunks. > Is it possible to keep the chunks on a single chunkserver, instead > of "load balance" them to all chunkservers ? As far as I know your requirement is far possible: there's no way to specify a chunk server for any files. The master does it job automatically. > Reason is that in case of a total unrecoverable loss of a single > chunkserver we would loose some files completly. But that would be > better to us than loosing some parts in all files. > Incrementing the goal is not an option since the storage capacity > is limited. How about your current goal (and your MFS setup)? IMHO you would get problem (as you described) when your goal is 1. If goal is 2, you can freely destroy (at most) 1 chunk server :P > ------------------------------------------------------------------------------ > Free Software Download: Index, Search & Analyze Logs and other IT > data in Real-Time with Splunk. Collect, index and harness all the > fast moving IT data generated by your applications, servers and > devices whether physical, virtual or in the cloud. Deliver > compliance at lower cost and gain new business insights. > http://p.sf.net/sfu/splunk-dev2dev > _______________________________________________ moosefs-users > mailing list moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > -- Anh Ky Huynh @ UTC+7 |
|
From: Michal B. <mic...@ge...> - 2011-02-25 10:21:52
|
Hi! If you set goal=1, all chunks belonging to the same file will go to the same chunk server. Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 -----Original Message----- From: Heiko Schröter [mailto:sch...@iu...] Sent: Thursday, February 24, 2011 10:09 AM To: moo...@li... Subject: Re: [Moosefs-users] Chunks Am Donnerstag 24 Februar 2011, um 09:54:14 schrieb Laurent Wandrebeck: > On Thu, 24 Feb 2011 09:24:11 +0100 > Heiko Schröter <sch...@iu...> wrote: > > > Hello, > Hi, > > > > we are currently investigating moosefs as a successor of our 200TB storage cfs. > > mfs-1.6.20, 2.6.36-gentoo-r5, x86_64, fuse 2.8.5 > > Everything is working fine. > > > > We have a question about the way mfs handles chunks. > > Is it possible to keep the chunks on a single chunkserver, instead of "load balance" them to all chunkservers ? > Not that I know of. mfs is designed for reliability, and working with > goal=1 is as bad as raid 0 when it comes to it. Thks for the reply but that was not my intention to ask. I try to keep the "pieces" (chunks) of a file on a single chunkserver. And yes, i'am quite clear about the risks when setting the goal=1. But this will affect the number of copies of the whole file, as far as i understand it. This what we have now with our lustre system anyway ;-) Our chunkserver (Raids) are generally running a hardware raid6 with 5 to 8TB per partition. I wouldn't mind if all the chunks are spread across theses partitions, as long as they stay on a single chunkserver. So, would it be possible to gather all chunks of a single file on a single chunkserver ? Regards Heiko ---------------------------------------------------------------------------- -- Free Software Download: Index, Search & Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Laurent W. <lw...@hy...> - 2011-02-25 10:27:18
|
On Fri, 25 Feb 2011 11:21:33 +0100 "Michal Borychowski" <mic...@ge...> wrote: > Hi! > > If you set goal=1, all chunks belonging to the same file will go to the same > chunk server. That one is good for a FAQ entry :) -- Laurent Wandrebeck HYGEOS, Earth Observation Department / Observation de la Terre Euratechnologies 165 Avenue de Bretagne 59000 Lille, France tel: +33 3 20 08 24 98 http://www.hygeos.com GPG fingerprint/Empreinte GPG: F5CA 37A4 6D03 A90C 7A1D 2A62 54E6 EF2C D17C F64C |
|
From: Davies L. <dav...@gm...> - 2011-04-21 11:18:31
|
Hi, you could modify default chunk size 64M to very high value, then a file has only one chunk, and in one chunk server. Davies 在 2011-2-24，下午4:24， Heiko Schröter 写道： > Hello, > > we are currently investigating moosefs as a successor of our 200TB storage cfs. > mfs-1.6.20, 2.6.36-gentoo-r5, x86_64, fuse 2.8.5 > Everything is working fine. > > We have a question about the way mfs handles chunks. > Is it possible to keep the chunks on a single chunkserver, instead of "load balance" them to all chunkservers ? > > Reason is that in case of a total unrecoverable loss of a single chunkserver we would loose some files completly. > But that would be better to us than loosing some parts in all files. > > Incrementing the goal is not an option since the storage capacity is limited. > > Thanks and Regards > Heiko > > ------------------------------------------------------------------------------ > Free Software Download: Index, Search & Analyze Logs and other IT data in > Real-Time with Splunk. Collect, index and harness all the fast moving IT data > generated by your applications, servers and devices whether physical, virtual > or in the cloud. Deliver compliance at lower cost and gain new business > insights. http://p.sf.net/sfu/splunk-dev2dev > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Michal B. <mic...@ge...> - 2011-04-21 11:24:54
|
Hi! But the default chunk size of 64MB is hard coded and we do not recommend to tamper with it as unforeseen things may happen. Best regards -Michal -----Original Message----- From: Davies Liu [mailto:dav...@gm...] Sent: Thursday, April 21, 2011 1:18 PM To: Heiko Schröter Cc: moo...@li... Subject: Re: [Moosefs-users] Chunks Hi, you could modify default chunk size 64M to very high value, then a file has only one chunk, and in one chunk server. Davies 在 2011-2-24，下午4:24， Heiko Schröter 写道： > Hello, > > we are currently investigating moosefs as a successor of our 200TB storage cfs. > mfs-1.6.20, 2.6.36-gentoo-r5, x86_64, fuse 2.8.5 > Everything is working fine. > > We have a question about the way mfs handles chunks. > Is it possible to keep the chunks on a single chunkserver, instead of "load balance" them to all chunkservers ? > > Reason is that in case of a total unrecoverable loss of a single chunkserver we would loose some files completly. > But that would be better to us than loosing some parts in all files. > > Incrementing the goal is not an option since the storage capacity is limited. > > Thanks and Regards > Heiko > > ------------------------------------------------------------------------------ > Free Software Download: Index, Search & Analyze Logs and other IT data in > Real-Time with Splunk. Collect, index and harness all the fast moving IT data > generated by your applications, servers and devices whether physical, virtual > or in the cloud. Deliver compliance at lower cost and gain new business > insights. http://p.sf.net/sfu/splunk-dev2dev > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users ------------------------------------------------------------------------------ Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X