Menu
▾
▴
moosefs-users
|
From: Mike <isp...@gm...> - 2011-06-21 03:02:09
|
What if someone wrote a patch for MooseFS that looked at the IP of the client, and the IP of the chunk servers that had the chunk the client wanted, and tried to pick the closest one? something like client = 10.1.1.1 chunkserver with copy#1 = 10.1.1.2 chunkserver with copy#2 = 10.1.1.20 chunkserver with copy#3 = 10.1.2.2 Those 3 chunk servers would be used in that order, since their IPs are closer to the client's IP (using a formula like a*256^3+b*256^2+c*256+d to calculate an integer? long int? based on an IP address). This way you could have "close" chunk servers respond to most of the requests from a client, but if no "close" server had the chunk you wanted, you could go to a "distant" one. Drop two IPs on the client's interface, and do some careful numbering, and you can even set preference on a machine on the same LAN. This might make for a really simple way to do "distant" archives that don't get used for reads unless they are the only source that's available, and other similar problems. writes would be a different problem. Thoughts? comments? |
|
From: rxknhe <rx...@gm...> - 2011-06-21 11:46:45
|
You mean 'rack-awareness' in a cluster, such as Hadoop. It doesn't have to be on IP based, that is too rigid setup in a large networks. Looks like MooseFS road map has some reference to a feature like 'Location awareness' http://www.moosefs.org/roadmap.html , so I would vote for that as a good feature to add in the future releases as well. On Mon, Jun 20, 2011 at 11:02 PM, Mike <isp...@gm...> wrote: > > What if someone wrote a patch for MooseFS that looked at the IP of the > client, and the IP of the chunk servers that had the chunk the client > wanted, and tried to pick the closest one? > > something like > > client = 10.1.1.1 > chunkserver with copy#1 = 10.1.1.2 > chunkserver with copy#2 = 10.1.1.20 > chunkserver with copy#3 = 10.1.2.2 > > Those 3 chunk servers would be used in that order, since their IPs are > closer to the client's IP (using a formula like a*256^3+b*256^2+c*256+d > to calculate an integer? long int? based on an IP address). > > This way you could have "close" chunk servers respond to most of the > requests from a client, but if no "close" server had the chunk you > wanted, you could go to a "distant" one. > > Drop two IPs on the client's interface, and do some careful numbering, > and you can even set preference on a machine on the same LAN. > > This might make for a really simple way to do "distant" archives that > don't get used for reads unless they are the only source that's > available, and other similar problems. > > writes would be a different problem. > > Thoughts? comments? > > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > |
|
From: Michal B. <mic...@ge...> - 2011-06-30 08:25:48
|
Hi! This could be quite difficult to keep "distances" in IP addresses. We introduced rack maps which should be easier in the mainenance. It would be soon available in the public version. Regards Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 -----Original Message----- From: Mike [mailto:isp...@gm...] Sent: Tuesday, June 21, 2011 5:02 AM To: moo...@li... Subject: [Moosefs-users] geographical fun with MooseFS What if someone wrote a patch for MooseFS that looked at the IP of the client, and the IP of the chunk servers that had the chunk the client wanted, and tried to pick the closest one? something like client = 10.1.1.1 chunkserver with copy#1 = 10.1.1.2 chunkserver with copy#2 = 10.1.1.20 chunkserver with copy#3 = 10.1.2.2 Those 3 chunk servers would be used in that order, since their IPs are closer to the client's IP (using a formula like a*256^3+b*256^2+c*256+d to calculate an integer? long int? based on an IP address). This way you could have "close" chunk servers respond to most of the requests from a client, but if no "close" server had the chunk you wanted, you could go to a "distant" one. Drop two IPs on the client's interface, and do some careful numbering, and you can even set preference on a machine on the same LAN. This might make for a really simple way to do "distant" archives that don't get used for reads unless they are the only source that's available, and other similar problems. writes would be a different problem. Thoughts? comments? ---------------------------------------------------------------------------- -- EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Florent B. <fl...@co...> - 2011-06-30 08:46:31
|
Hi Michal, Could we have an idea of when will be released the next public version and what improvements will be included ? Thank you a lot for your work. Florent Le 30/06/2011 10:24, Michal Borychowski a écrit : > Hi! > > This could be quite difficult to keep "distances" in IP addresses. We > introduced rack maps which should be easier in the mainenance. It would be > soon available in the public version. > > > Regards > Kind regards > Michał Borychowski > MooseFS Support Manager > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > Gemius S.A. > ul. Wołoska 7, 02-672 Warszawa > Budynek MARS, klatka D > Tel.: +4822 874-41-00 > Fax : +4822 874-41-01 > > > -----Original Message----- > From: Mike [mailto:isp...@gm...] > Sent: Tuesday, June 21, 2011 5:02 AM > To: moo...@li... > Subject: [Moosefs-users] geographical fun with MooseFS > > > What if someone wrote a patch for MooseFS that looked at the IP of the > client, and the IP of the chunk servers that had the chunk the client > wanted, and tried to pick the closest one? > > something like > > client = 10.1.1.1 > chunkserver with copy#1 = 10.1.1.2 > chunkserver with copy#2 = 10.1.1.20 > chunkserver with copy#3 = 10.1.2.2 > > Those 3 chunk servers would be used in that order, since their IPs are > closer to the client's IP (using a formula like a*256^3+b*256^2+c*256+d > to calculate an integer? long int? based on an IP address). > > This way you could have "close" chunk servers respond to most of the > requests from a client, but if no "close" server had the chunk you > wanted, you could go to a "distant" one. > > Drop two IPs on the client's interface, and do some careful numbering, > and you can even set preference on a machine on the same LAN. > > This might make for a really simple way to do "distant" archives that > don't get used for reads unless they are the only source that's > available, and other similar problems. > > writes would be a different problem. > > Thoughts? comments? > > ---------------------------------------------------------------------------- > -- > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Roger S. <rog...@un...> - 2011-07-07 13:36:36
|
How will this rack maps implemention be? Is is rack de-centric or co-centric? RS On 6/30/11 10:24 AM, Michal Borychowski wrote: > Hi! > > This could be quite difficult to keep "distances" in IP addresses. We > introduced rack maps which should be easier in the mainenance. It would be > soon available in the public version. > > > Regards > Kind regards > Michał Borychowski > MooseFS Support Manager > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > Gemius S.A. > ul. Wołoska 7, 02-672 Warszawa > Budynek MARS, klatka D > Tel.: +4822 874-41-00 > Fax : +4822 874-41-01 > > > -----Original Message----- > From: Mike [mailto:isp...@gm...] > Sent: Tuesday, June 21, 2011 5:02 AM > To: moo...@li... > Subject: [Moosefs-users] geographical fun with MooseFS > > > What if someone wrote a patch for MooseFS that looked at the IP of the > client, and the IP of the chunk servers that had the chunk the client > wanted, and tried to pick the closest one? > > something like > > client = 10.1.1.1 > chunkserver with copy#1 = 10.1.1.2 > chunkserver with copy#2 = 10.1.1.20 > chunkserver with copy#3 = 10.1.2.2 > > Those 3 chunk servers would be used in that order, since their IPs are > closer to the client's IP (using a formula like a*256^3+b*256^2+c*256+d > to calculate an integer? long int? based on an IP address). > > This way you could have "close" chunk servers respond to most of the > requests from a client, but if no "close" server had the chunk you > wanted, you could go to a "distant" one. > > Drop two IPs on the client's interface, and do some careful numbering, > and you can even set preference on a machine on the same LAN. > > This might make for a really simple way to do "distant" archives that > don't get used for reads unless they are the only source that's > available, and other similar problems. > > writes would be a different problem. > > Thoughts? comments? > > ---------------------------------------------------------------------------- > -- > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: i- <it...@it...> - 2011-07-07 15:08:15
|
Big up for this feature ! This REALLY is interresting. Le 07/07/2011 15:36, Roger Skjetlein a écrit : > How will this rack maps implemention be? Is is rack de-centric or > co-centric? > > RS > > On 6/30/11 10:24 AM, Michal Borychowski wrote: >> Hi! >> >> This could be quite difficult to keep "distances" in IP addresses. We >> introduced rack maps which should be easier in the mainenance. It would be >> soon available in the public version. >> >> >> -----Original Message----- >> From: Mike [mailto:isp...@gm...] >> Sent: Tuesday, June 21, 2011 5:02 AM >> To: moo...@li... >> Subject: [Moosefs-users] geographical fun with MooseFS >> >> >> What if someone wrote a patch for MooseFS that looked at the IP of the >> client, and the IP of the chunk servers that had the chunk the client >> wanted, and tried to pick the closest one? >> >> something like >> >> client = 10.1.1.1 >> chunkserver with copy#1 = 10.1.1.2 >> chunkserver with copy#2 = 10.1.1.20 >> chunkserver with copy#3 = 10.1.2.2 >> >> Those 3 chunk servers would be used in that order, since their IPs are >> closer to the client's IP (using a formula like a*256^3+b*256^2+c*256+d >> to calculate an integer? long int? based on an IP address). >> >> This way you could have "close" chunk servers respond to most of the >> requests from a client, but if no "close" server had the chunk you >> wanted, you could go to a "distant" one. >> >> Drop two IPs on the client's interface, and do some careful numbering, >> and you can even set preference on a machine on the same LAN. >> >> This might make for a really simple way to do "distant" archives that >> don't get used for reads unless they are the only source that's >> available, and other similar problems. >> >> writes would be a different problem. >> >> Thoughts? comments? |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X