Menu
▾
▴
moosefs-users
|
From: Florent B. <fl...@co...> - 2011-06-09 11:07:01
|
Hi all, I have a question about failover of mfsmaster. Which solution is the best, for now ? I don't know which solution to choose between : - drdb+heartbeat - mfsmetalogger+mfsrestore What do you think about it ? is there other ways to failover mfsmaster (active sync of RAM between 2 hosts ?) ? -- Florent Bautista ------------------------------------------------------------------------ Ce message et ses éventuelles pièces jointes sont personnels, confidentiels et à l'usage exclusif de leur destinataire. Si vous n'êtes pas la personne à laquelle ce message est destiné, veuillez noter que vous avez reçu ce courriel par erreur et qu'il vous est strictement interdit d'utiliser, de diffuser, de transférer, d'imprimer ou de copier ce message. This e-mail and any attachments hereto are strictly personal, confidential and intended solely for the addressee. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this message is strictly prohibited. ------------------------------------------------------------------------ |
|
From: Thomas S H. <tha...@gm...> - 2011-06-09 14:20:09
|
There are some solutions which involve synchronized virtual machines, many hyper-visors can do this. If you are planning on using the metalogger + mfsrestore there will always be a short failover lag. Personally, what I think would be the best option would be if the metalogger could populate the ram of a secondary read-only master. This is the same essential way that redis does it. The other option would be to set systems similar to mongo's HA solutions. Of course these solutions would require the MooseFS developers to implement them - I think that mfsmaster in-ram replication is something that MooseFS is very close to. Let us know what you decide on, there are a number of individuals using HA solutions for MooseFS. On Thu, Jun 9, 2011 at 5:06 AM, Florent Bautista <fl...@co...>wrote: > Hi all, > > I have a question about failover of mfsmaster. > > Which solution is the best, for now ? > > I don't know which solution to choose between : > > - drdb+heartbeat > - mfsmetalogger+mfsrestore > > What do you think about it ? is there other ways to failover mfsmaster > (active sync of RAM between 2 hosts ?) ? > > -- > > > Florent Bautista > ------------------------------ > > Ce message et ses éventuelles pièces jointes sont personnels, confidentiels > et à l'usage exclusif de leur destinataire. > Si vous n'êtes pas la personne à laquelle ce message est destiné, veuillez > noter que vous avez reçu ce courriel par erreur et qu'il vous est > strictement interdit d'utiliser, de diffuser, de transférer, d'imprimer ou > de copier ce message. > > This e-mail and any attachments hereto are strictly personal, confidential > and intended solely for the addressee. > If you are not the intended recipient, be advised that you have received > this email in error and that any use, dissemination, forwarding, printing, > or copying of this message is strictly prohibited. > ------------------------------ > > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > |
|
From: Steve <st...@bo...> - 2011-06-09 14:45:02
|
Is it at all possible that you could have more than one active master and they could be added or removed in the same way a chunk server can without affecting availability. Sharing workload ? -------Original Message------- From: Thomas S Hatch Date: 09/06/2011 15:20:07 To: Florent Bautista Cc: moo...@li... Subject: Re: [Moosefs-users] Failover of MFSMaster There are some solutions which involve synchronized virtual machines, many hyper-visors can do this. If you are planning on using the metalogger + mfsrestore there will always be a short failover lag. Personally, what I think would be the best option would be if the metalogger could populate the ram of a secondary read-only master. This is the same essential way that redis does it. The other option would be to set systems similar to mongo's HA solutions. Of course these solutions would require the MooseFS developers to implement them - I think that mfsmaster in-ram replication is something that MooseFS is very close to. Let us know what you decide on, there are a number of individuals using HA solutions for MooseFS. On Thu, Jun 9, 2011 at 5:06 AM, Florent Bautista <fl...@co...> wrote: Hi all, I have a question about failover of mfsmaster. Which solution is the best, for now ? I don't know which solution to choose between : - drdb+heartbeat - mfsmetalogger+mfsrestore What do you think about it ? is there other ways to failover mfsmaster (active sync of RAM between 2 hosts ?) ? -- Florent Bautista Ce message et ses éventuelles pièces jointes sont personnels, confidentiels et à l'usage exclusif de leur destinataire. Is vous n'êtes pas la personne à laquelle ce message EST destiné, veuillez noter que vous avez reçu ce courriel par erreur et qu'il vous EST strictement interdit d'utiliser, de diffuser, de transférer, d'imprimer ou de copier ce message. This e-mail and any attachments hereto are strictly personal, confidential and intended solely for the addressee. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this message is strictly prohibited. ----------------------------------------------------------------------------- EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users ----------------------------------------------------------------------------- EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Thomas S H. <tha...@gm...> - 2011-06-09 15:17:41
|
Unfortunately no. The problem is that the master holds all of the filesystem metadata, and if you had multiple master then you would need to guarantee that all masters have consistent data. This is a classic distributed database problem, and has still not been solved by projects like MySQL. There are few models out there for distributed master load, but the end result is almost always lower speed for the client, because more checks need to me put in place, or as is the case in a number of other distributed filesystems, a lot of the load is displaced to the client. I will agree that right now the master is the greatset point of failure and load issues in MooseFS. This is why I will still lobby for MooseFS to grate an active replicated master and clients that can query multiple mfsmasters to find the active master so that failover can happen without heartbeat and ucarp. But this solution is not in MooseFS, and I am unsure if it is going to be considered. Although, the metalogger is VERY close to doing this, I think that all they need are a few consistency checks and the ability to metalog to a master replica. On Thu, Jun 9, 2011 at 8:44 AM, Steve <st...@bo...> wrote: > > Is it at all possible that you could have more than one active master and > they could be added or removed in the same way a chunk server can without > affecting availability. Sharing workload ? > > > > > > > > > > -------Original Message------- > > > > From: Thomas S Hatch > > Date: 09/06/2011 15:20:07 > > To: Florent Bautista > > Cc: moo...@li... > > Subject: Re: [Moosefs-users] Failover of MFSMaster > > > > There are some solutions which involve synchronized virtual machines, many > hyper-visors can do this. > > If you are planning on using the metalogger + mfsrestore there will always > be a short failover lag. > > > > > > Personally, what I think would be the best option would be if the > metalogger > could populate the ram of a secondary read-only master. This is the same > essential way that redis does it. > > > > > > The other option would be to set systems similar to mongo's HA solutions. > > > > > > Of course these solutions would require the MooseFS developers to implement > them - I think that mfsmaster in-ram replication is something that MooseFS > is very close to. > > > > > > Let us know what you decide on, there are a number of individuals using HA > solutions for MooseFS. > > > > > > On Thu, Jun 9, 2011 at 5:06 AM, Florent Bautista <fl...@co...> > wrote: > > > > Hi all, > > > > I have a question about failover of mfsmaster. > > > > Which solution is the best, for now ? > > > > I don't know which solution to choose between : > > > > - drdb+heartbeat > > - mfsmetalogger+mfsrestore > > > > What do you think about it ? is there other ways to failover mfsmaster > (active sync of RAM between 2 hosts ?) ? > > > > > > -- > > > > > > Florent Bautista > > > > > > > > Ce message et ses éventuelles pièces jointes sont personnels, confidentiels > et à l'usage exclusif de leur destinataire. > > Is vous n'êtes pas la personne à laquelle ce message EST destiné, veuillez > noter que vous avez reçu ce courriel par erreur et qu'il vous EST > strictement interdit d'utiliser, de diffuser, de transférer, d'imprimer ou > de copier ce message. > > > > This e-mail and any attachments hereto are strictly personal, confidential > and intended solely for the addressee. > > If you are not the intended recipient, be advised that you have received > this email in error and that any use, dissemination, forwarding, printing, > or copying of this message is strictly prohibited. > > > > > > > > > > > ----------------------------------------------------------------------------- > > > EditLive Enterprise is the world's most technically advanced content > > authoring tool. Experience the power of Track Changes, Inline Image > > Editing and ensure content is compliant with Accessibility Checking. > > http://p.sf.net/sfu/ephox-dev2dev > > _______________________________________________ > > moosefs-users mailing list > > moo...@li... > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > > > > > > > > > > > > > > > > > > ----------------------------------------------------------------------------- > > > EditLive Enterprise is the world's most technically advanced content > > authoring tool. Experience the power of Track Changes, Inline Image > > Editing and ensure content is compliant with Accessibility Checking. > > http://p.sf.net/sfu/ephox-dev2dev > > > > > > > > > > > > _______________________________________________ > > moosefs-users mailing list > > moo...@li... > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > |
|
From: Steve <st...@bo...> - 2011-06-09 16:35:03
|
Ahh I see complex problem. A few mins googling and seeing what google use - they have 'chubby' -------Original Message------- From: Thomas S Hatch Date: 09/06/2011 16:16:35 To: Steve Cc: Florent Bautista; moo...@li... Subject: Re: [Moosefs-users] Failover of MFSMaster Unfortunately no. The problem is that the master holds all of the filesystem metadata, and if you had multiple master then you would need to guarantee that all masters have consistent data. This is a classic distributed database problem, and has still not been solved by projects like MySQL. There are few models out there for distributed master load, but the end result is almost always lower speed for the client, because more checks need to me put in place, or as is the case in a number of other distributed filesystems, a lot of the load is displaced to the client. I will agree that right now the master is the greatset point of failure and load issues in MooseFS. This is why I will still lobby for MooseFS to grate an active replicated master and clients that can query multiple mfsmasters to find the active master so that failover can happen without heartbeat and ucarp. But this solution is not in MooseFS, and I am unsure if it is going to be considered. Although, the metalogger is VERY close to doing this, I think that all they need are a few consistency checks and the ability to metalog to a master replica. On Thu, Jun 9, 2011 at 8:44 AM, Steve <st...@bo...> wrote: Is it at all possible that you could have more than one active master and they could be added or removed in the same way a chunk server can without affecting availability. Sharing workload ? -------Original Message------- From: Thomas S Hatch Date: 09/06/2011 15:20:07 To: Florent Bautista Cc: moo...@li... Subject: Re: [Moosefs-users] Failover of MFSMaster There are some solutions which involve synchronized virtual machines, many hyper-visors can do this. If you are planning on using the metalogger + mfsrestore there will always be a short failover lag. Personally, what I think would be the best option would be if the metalogger could populate the ram of a secondary read-only master. This is the same essential way that redis does it. The other option would be to set systems similar to mongo's HA solutions. Of course these solutions would require the MooseFS developers to implement them - I think that mfsmaster in-ram replication is something that MooseFS is very close to. Let us know what you decide on, there are a number of individuals using HA solutions for MooseFS. On Thu, Jun 9, 2011 at 5:06 AM, Florent Bautista <fl...@co...> wrote: Hi all, I have a question about failover of mfsmaster. Which solution is the best, for now ? I don't know which solution to choose between : - drdb+heartbeat - mfsmetalogger+mfsrestore What do you think about it ? is there other ways to failover mfsmaster (active sync of RAM between 2 hosts ?) ? -- Florent Bautista Ce message et ses éventuelles pièces jointes sont personnels, confidentiels et à l'usage exclusif de leur destinataire. Is vous n'êtes pas la personne à laquelle ce message EST destiné, veuillez noter que vous avez reçu ce courriel par erreur et qu'il vous EST strictement interdit d'utiliser, de diffuser, de transférer, d'imprimer ou de copier ce message. This e-mail and any attachments hereto are strictly personal, confidential and intended solely for the addressee. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this message is strictly prohibited. ----------------------------------------------------------------------------- EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users ----------------------------------------------------------------------------- EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Thomas S H. <tha...@gm...> - 2011-06-09 16:38:23
|
Right :) There are a lot of solutions, and none of them are easy to get right. But all in all I think that the MooseFS team is perfectly capable of figuring this one out, Jakob is crazy smart! On Thu, Jun 9, 2011 at 10:34 AM, Steve <st...@bo...> wrote: > > Ahh I see complex problem. > > > > A few mins googling and seeing what google use - they have 'chubby' > > > > > > > > > > -------Original Message------- > > > > From: Thomas S Hatch > > Date: 09/06/2011 16:16:35 > > To: Steve > > Cc: Florent Bautista; moo...@li... > > Subject: Re: [Moosefs-users] Failover of MFSMaster > > > > Unfortunately no. > > > > > > The problem is that the master holds all of the filesystem metadata, and if > you had multiple master then you would need to guarantee that all masters > have consistent data. > > > > > > This is a classic distributed database problem, and has still not been > solved by projects like MySQL. > > > > > > There are few models out there for distributed master load, but the end > result is almost always lower speed for the client, because more checks > need > to me put in place, or as is the case in a number of other distributed > filesystems, a lot of the load is displaced to the client. > > > > > > I will agree that right now the master is the greatset point of failure and > load issues in MooseFS. This is why I will still lobby for MooseFS to grate > an active replicated master and clients that can query multiple mfsmasters > to find the active master so that failover can happen without heartbeat and > ucarp. But this solution is not in MooseFS, and I am unsure if it is going > to be considered. Although, the metalogger is VERY close to doing this, I > think that all they need are a few consistency checks and the ability to > metalog to a master replica. > > > > > > On Thu, Jun 9, 2011 at 8:44 AM, Steve <st...@bo...> wrote: > > > > > > Is it at all possible that you could have more than one active master and > > they could be added or removed in the same way a chunk server can without > > affecting availability. Sharing workload ? > > > > > > > > > > > > > > > > > > > > -------Original Message------- > > > > > > > > From: Thomas S Hatch > > > > Date: 09/06/2011 15:20:07 > > > > To: Florent Bautista > > > > Cc: moo...@li... > > > > Subject: Re: [Moosefs-users] Failover of MFSMaster > > > > > > > > > > There are some solutions which involve synchronized virtual machines, many > > hyper-visors can do this. > > > > If you are planning on using the metalogger + mfsrestore there will always > > be a short failover lag. > > > > > > > > > > > > Personally, what I think would be the best option would be if the > metalogger > > > could populate the ram of a secondary read-only master. This is the same > > essential way that redis does it. > > > > > > > > > > > > The other option would be to set systems similar to mongo's HA solutions. > > > > > > > > > > > > Of course these solutions would require the MooseFS developers to implement > > them - I think that mfsmaster in-ram replication is something that MooseFS > > is very close to. > > > > > > > > > > > > Let us know what you decide on, there are a number of individuals using HA > > solutions for MooseFS. > > > > > > > > > > > > On Thu, Jun 9, 2011 at 5:06 AM, Florent Bautista <fl...@co...> > > wrote: > > > > > > > > Hi all, > > > > > > > > I have a question about failover of mfsmaster. > > > > > > > > Which solution is the best, for now ? > > > > > > > > I don't know which solution to choose between : > > > > > > > > - drdb+heartbeat > > > > - mfsmetalogger+mfsrestore > > > > > > > > What do you think about it ? is there other ways to failover mfsmaster > > (active sync of RAM between 2 hosts ?) ? > > > > > > > > > > > > -- > > > > > > > > > > > > Florent Bautista > > > > > > > > > > > > > > > > Ce message et ses éventuelles pièces jointes sont personnels, confidentiels > > et à l'usage exclusif de leur destinataire. > > > > > > Is vous n'êtes pas la personne à laquelle ce message EST destiné, veuillez > > noter que vous avez reçu ce courriel par erreur et qu'il vous EST > > > > strictement interdit d'utiliser, de diffuser, de transférer, d'imprimer ou > > de copier ce message. > > > > > > > > This e-mail and any attachments hereto are strictly personal, confidential > > and intended solely for the addressee. > > > > If you are not the intended recipient, be advised that you have received > > this email in error and that any use, dissemination, forwarding, printing, > > or copying of this message is strictly prohibited. > > > > > > > > > > > > > > > > > > > > > ----------------------------------------------------------------------------- > > > > > > > EditLive Enterprise is the world's most technically advanced content > > > > authoring tool. Experience the power of Track Changes, Inline Image > > > > Editing and ensure content is compliant with Accessibility Checking. > > > > http://p.sf.net/sfu/ephox-dev2dev > > > > _______________________________________________ > > > > moosefs-users mailing list > > > > moo...@li... > > > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ----------------------------------------------------------------------------- > > > > > > > EditLive Enterprise is the world's most technically advanced content > > > > authoring tool. Experience the power of Track Changes, Inline Image > > > > Editing and ensure content is compliant with Accessibility Checking. > > > > http://p.sf.net/sfu/ephox-dev2dev > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > moosefs-users mailing list > > > > moo...@li... > > > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > > > > > > > > > > |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X