Re: [Moosefs-users] How fast can you copy files to your Moosefs ?

[Michał B. <mic...@ge...>]

Your point B is closer to the real writing process. But the client doesn’t
wait with sending data. It sends new data before it receives confirmation
of writing previous data. Only removing from write queue takes place after
writing confirmation. 

 

The writing schema follows like this:

 

         send '1'

          |   /- send '2'

          |   |   /- send '3'

          |   |   |       /- remove '1' from buffer

          |   |   |       |   /- remove '2' from buffer

          v   v   v       v   v    /- remove '3' from buffer

Client: --+---+---+-------+---+---+------------

           \1  \2  \3    /1  /2  /3

CS1:    ----+111+222+333+---+---+--------------

             \1  \2  \3/1  /2 /3

CS2:    ------+111+222+333+--+-----------------

               \1  \2/1\3/2 /3

CS3:    --------+111+222+333+------------------

 

\X - sending data of block 'X'

/X - receiving status of block 'X'

 

-XXX- - writing data of block 'X'

 

 

For clarity we present here writing of only three blocks. So between goal=1
and goal=3 there is a constant difference independent of blocks quantity.

 

There are of course some “chokes” during writes to the disks. For goal=1
we have only delays connected to chokes of one disk, but for goal=3 we have
cumulated delays of all the disks. This is because each chunkserver writes
in parallel only one block - so each write “choke” delays the whole
process, independently of chunk server.

 

 

Kind regards

Michal Borychowski 

 

 

From: kuer ku [mailto:ku...@gm...] 
Sent: Wednesday, August 11, 2010 4:44 AM
To: Michał Borychowski
Subject: Re: [Moosefs-users] How fast can you copy files to your Moosefs ?

 

Hi, Michal,

 

With goal=3 data transmission looks like this: client <=> cs1 <=> cs2 <=>
cs3 

 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In such situations, when client finish write operation ?

 

A. when client finish writing to cs1, client can think it finish writing;
and cs1 will issue a writing operation to cs2 asynchronously; and cs2 does
it asynchronously too. 

In this case, when client finish writing, MFS have at least 1 copy of data,
and MFS will try to replicate more copies asynchronously.

 

B. client waiting write operation end util : cs1 finish writing, and cs2
finish writing, and cs3 finish writing. 

In this case, when client finish writing, MFS have 3 copies of data.

 

Michal, do you mean case A ?? In case A, to set a higher goal does not
decrease too much performance ( in client view).

 

 -- kuer

 

2010/8/9 Michał Borychowski <mic...@ge...>

This is not a normal behavior. With goal=3 data transmission looks like
this: client <=> cs1 <=> cs2 <=> cs3 (whereas order of chunkservers if
different for different chunks) and normally should work with speeds
similar to goal=1. For 100Mbit/s network we would expect 6-7MB/s). But
please check if you have full-duplex enabled. If you only have half-duplex,
speed can be substantially lower. 

 

As stated before, we definitely recommend 1Gbit/s. And please check if the
existing network is for sure properly configured.

 

 

Kind regards

Michał Borychowski 

 

 

From: Chen, Alvin [mailto:alv...@in...] 
Sent: Thursday, July 29, 2010 3:57 AM
To: Micha? Borychowski


Cc: moo...@li...

Subject: Re: [Moosefs-users] How fast can you copy files to your Moosefs ?

 

I mean setting goal to 3. If setting goal to 1, the writing speed is
9MBytes/sec for 100Mbps networking, but if setting goal to 3, the speed is
just 500KBytes/sec. 

 

For 100Mbps networking, the speed should reach 12.5Mbytes/sec, so if
setting goal to 1, 9Mbytes/sec is reasonable, but if setting goal to 3, the
speed should be around 3Mbytes/sec.

 

By the way, I just use scp to copy 4GB data file to the mount folder.

 

 

Best regards，Alvin Chen 
ICFS Platform Engineering Solution 
Flex Services (CMMI Level 3, IQA2005, IQA2008), Greater Asia Region 
Intel Information Technology 
Tel. 010-82171960 
inet.8-7581960 
Email. alv...@in... 

From: mic...@ge... [mailto:mic...@ge...] 
Sent: Wednesday, July 28, 2010 7:45 PM
To: Chen, Alvin
Cc: moo...@li...
Subject: RE: [Moosefs-users] How fast can you copy files to your Moosefs ?

 

First of all if you want better performance you should use gigabit network.

 

We have writes of about 20-30MiB/s (have a look here:
http://www.moosefs.org/moosefs-faq.html#average). 

 

You can also have a look here: http://www.moosefs.org/moosefs-faq.html#mtu
for some network tips.

 

PS. Talking about 3 copies do you mean setting goal=3 or copying 3 files
simultaneously?

 

 

Kind regards

Michal Borychowski 

 

MooseFS Support Manager

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gemius S.A.

ul. Wołoska 7, 02-672 Warszawa

Budynek MARS, klatka D

Tel.: +4822 874-41-00

Fax : +4822 874-41-01

 

From: Chen, Alvin [mailto:alv...@in...] 
Sent: Tuesday, July 27, 2010 10:52 AM
To: moo...@li...
Subject: [Moosefs-users] How fast can you copy files to your Moosefs ?

 

Hi guys,

 

I am a new user of moosefs. I have 3 chunk servers, one master server with
100Mbps network. And I just copy a 4GB files from one client machine to
Moosefs, and the copying speed can reach 9MB/s if just one copy, but the
copying speed is just 500KB/s if 3copies. 

 

 

How fast do your Moosefs can reach? Does anybody get better performance?

 

 

Best regards，Alvin Chen 
ICFS Platform Engineering Solution 
Flex Services (CMMI Level 3, IQA2005, IQA2008), Greater Asia Region 
Intel Information Technology 
Tel. 010-82171960 
inet.8-7581960 
Email. alv...@in... 


----------------------------------------------------------------------------
--
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users