Menu
▾
▴
Re: [MooseFS-Users] Separate network for chunk servers
|
From: Piotr R. K. <pio...@mo...> - 2018-06-06 21:21:28
|
Thanks for sharing! Peter -- Piotr Robert Konopelko | mobile: +48 601 476 440 MooseFS Client Support Team | moosefs.com <http://moosefs.com/> GitHub <https://github.com/moosefs/moosefs> | Twitter <https://twitter.com/moosefs> | Facebook <https://www.facebook.com/moosefs> | LinkedIn <https://www.linkedin.com/company/moosefs> > On 6 Jun 2018, at 8:26 PM, Wilson, Steven M <st...@pu... <mailto:st...@pu...>> wrote: > > > From: Wilson, Steven M <st...@pu... <mailto:st...@pu...>> > Sent: Monday, May 7, 2018 10:36 AM > To: MooseFS-Users > Subject: [MooseFS-Users] Separate network for chunk servers > > Hi, > > I'm considering implementing a dedicated network for our chunk servers to use soley for replication among themselves. By doing this, I hope to separate the chunk traffic from the clients from the replication traffic that takes place among the chunk servers. If my understanding is correct, this is not achieved by using the REMAP_* options in mfsmaster.cfg which only separates out the traffic to/from the master. > > If anyone else has done this, I'd be grateful to hear about your experience, especially in these two areas: > 1) what level of performance improvement was seen > 2) what needed to be done in the MooseFS configuration and OS networking to implement it > > Thanks! > Steve > > ===================================================================================== > > In case it's helpful to anyone else, I did find an easy way to shift traffic between chunk servers onto a separate network. On each chunk server I created a few iptables rules using NAT like the following (these would need to be repeated for each additional chunk server): > iptables -t nat -A OUTPUT -s $MY_IP/32 -d $OTHER_CS_IP/32 -j DNAT --to-destination $OTHER_CS_PRIV_IP -p tcp -m tcp --dport $CS_PORT > iptables -t nat -A POSTROUTING -s $MY_IP/32 -d $OTHER_CS_PRIV_IP/32 -j SNAT --to-source $MY_PRIV_IP > > I did some performance testing and found that, for fairly obvious reasons, this private chunk server network only helps when the network is near saturation and most of the I/O is writing to the file system (which generates inter-chunkserver traffic for producing redundant copies of the data). When this type of environment is simulated, I saw between 10 to 20% improvement in speed for writing files to the file system. > > Steve > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_________________________________________ <http://sdm.link/slashdot_________________________________________> > moosefs-users mailing list > moo...@li... <mailto:moo...@li...> > https://lists.sourceforge.net/lists/listinfo/moosefs-users <https://lists.sourceforge.net/lists/listinfo/moosefs-users> Best regards, Peter -- Piotr Robert Konopelko | mobile: +48 601 476 440 MooseFS Client Support Team | moosefs.com <http://moosefs.com/> GitHub <https://github.com/moosefs/moosefs> | Twitter <https://twitter.com/moosefs> | Facebook <https://www.facebook.com/moosefs> | LinkedIn <https://www.linkedin.com/company/moosefs> |
