Re: [MooseFS-Users] question about mfs reading speed slower than writing speed???

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi,Aleks,

According to your reply, I list out all the detail information about my
moosefs file system.  you can see below,

After several test, I find that , the read and write speed diff because of
the network latency.

When the client and mfs master in the same IDC, the reading and writing
speed ,almost the same.
but when the client and mfs master in different city, it show a result that
the reading speed much slow than the write speed.

can you explain why the network latency make such difference in reading and
writing ?
Also ,according to the test, seems that the reading speed never faster than
the write speed. it's strange, mostly, reading should be faster than
writing?

network topology:
----------------------------------------------------------

master:
10.153.136.230

master -metalogger:
10.153.136.227

chunkserver:
10.153.142.237
10.153.142.239
10.153.143.98

client:
same city:
10.135.32.170

another city:
10.149.131.111

mfs master , mfs metalogger, mfs chunk server are in same IDC.

client 10.135.32.170 in same city ,different IDC.

client 10.149.131.111 in other city.

all these server has ： 1000Mb/s speed.
------------------------------------------------------------
ethtool eth1
Settings for eth1:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000007 (7)

Network Latency Test:
--------------------------------------------------------------------------------
I do this test from the moosefs master server, ping all the other component.

mfs master ----> chunkserver
ping 10.153.142.237
PING 10.153.142.237 (10.153.142.237) 56(84) bytes of data.
64 bytes from 10.153.142.237: icmp_seq=1 ttl=61 time=0.102 ms
64 bytes from 10.153.142.237: icmp_seq=2 ttl=61 time=0.084 ms

mfs master -----> another city client
ping 10.149.131.111
PING 10.149.131.111 (10.149.131.111) 56(84) bytes of data.
64 bytes from 10.149.131.111: icmp_seq=1 ttl=55 time=27.0 ms
64 bytes from 10.149.131.111: icmp_seq=2 ttl=55 time=27.0 ms

mfs master -----------> same city client
ping 10.135.32.170
PING 10.135.32.170 (10.135.32.170) 56(84) bytes of data.
64 bytes from 10.135.32.170: icmp_seq=2 ttl=57 time=2.28 ms
64 bytes from 10.135.32.170: icmp_seq=3 ttl=57 time=2.27 ms

mfs version information:
-----------------------------------------------------------------------------------
the mfs version（current version):
./mfsmaster -v
version: 2.0.61-1

also, I do the test in
./mfsmaster -v
version: 1.6.27

./mfsmount -V
MFS version 2.0.61-1
FUSE library version: 2.8.3
fusermount version: 2.8.3

*client read and write test:*
-------------------------------------------------------------------------------------------

*Test1: same city client: 10.135.32.170*
read from mfs:
[root@TENCENT64 /mnt/mfs]# time dd if=/mnt/mfs/1-1.iso of=/dev/null

1024000+0 records in
1024000+0 records out
524288000 bytes (524 MB) copied, 12.8321 s, 40.9 MB/s

real    0m12.841s
user    0m0.092s
sys     0m0.364s
[root@TENCENT64 /mnt/mfs]# time dd if=/mnt/mfs/1-2.iso of=/dev/null
1024000+0 records in
1024000+0 records out
524288000 bytes (524 MB) copied, 10.148 s, 51.7 MB/s

real    0m10.154s
user    0m0.120s
sys     0m0.328s

write to mfs:
[root@TENCENT64 /mnt/mfs]# time dd if=/dev/zero of=/mnt/mfs/2-2.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 5.04111 s, 104 MB/s

real    0m5.047s
user    0m0.000s
sys     0m0.204s
[root@TENCENT64 /mnt/mfs]# time dd if=/dev/zero of=/mnt/mfs/2-3.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 5.46042 s, 96.0 MB/s

real    0m5.467s
user    0m0.000s
sys     0m0.208s

*another client in same city:*
read from mfs:
[root@TENCENT64 /mnt/mfs]# time cp /mnt/mfs/555-new.iso /data/

real    0m9.869s
user    0m0.000s
sys     0m0.669s

[root@TENCENT64 /mnt/mfs]# time cp /mnt/mfs/555.iso /data/

real    0m9.980s
user    0m0.004s
sys     0m0.687s

write to mfs:
[root@TENCENT64 /data]# time dd if=/dev/zero of=/mnt/mfs/1-1.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 5.30263 s, 98.9 MB/s

real    0m5.332s
user    0m0.000s
sys     0m0.349s
[root@TENCENT64 /data]# time dd if=/dev/zero of=/mnt/mfs/1-2.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 5.12036 s, 102 MB/s

real    0m5.128s
user    0m0.004s
sys     0m0.345s

*client in other city test:*

write to mfs:
[root@Tencent-SNG /mnt/mfs]# time dd if=/dev/zero of=/mnt/mfs/3-1.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 13.2458 s, 39.6 MB/s

real    0m13.332s
user    0m0.002s
sys     0m0.467s
[root@Tencent-SNG /mnt/mfs]# time dd if=/dev/zero of=/mnt/mfs/3-2.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 12.1528 s, 43.1 MB/s

real    0m12.211s
user    0m0.002s
sys     0m0.414s

read from mfs:

[root@Tencent-SNG /mnt/mfs]# time dd if=/mnt/mfs/1-1.iso of=/dev/null
1024000+0 records in
1024000+0 records out
524288000 bytes (524 MB) copied, 44.355 s, 11.8 MB/s

real    0m44.413s
user    0m0.202s
sys     0m0.653s
[root@Tencent-SNG /mnt/mfs]# time dd if=/mnt/mfs/1-2.iso of=/dev/null
1024000+0 records in
1024000+0 records out
524288000 bytes (524 MB) copied, 45.0923 s, 11.6 MB/s

real    0m45.151s
user    0m0.197s
sys     0m0.743s

*last , test the client and mfs master in the same server (means in same
IDC):*
write to mfs:

root@Tencent-SNG /mnt/mfs]# time dd if=/dev/zero of=/mnt/mfs/4-1.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 4.71052 s, 111 MB/s

real    0m4.712s
user    0m0.000s
sys     0m0.272s
[root@Tencent-SNG /mnt/mfs]# time dd if=/dev/zero of=/mnt/mfs/4-2.iso bs=1M
count=500
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 4.63003 s, 113 MB/s

real    0m4.632s
user    0m0.000s
sys     0m0.284s

read from mfs :

[root@Tencent-SNG /mnt/mfs]# time dd if=/mnt/mfs/1-1.iso of=/dev/null
1024000+0 records in
1024000+0 records out
524288000 bytes (524 MB) copied, 4.68863 s, 112 MB/s

real    0m4.690s
user    0m0.140s
sys     0m0.412s
[root@Tencent-SNG /mnt/mfs]# time dd if=/mnt/mfs/1-2.iso of=/dev/null
1024000+0 records in
1024000+0 records out
524288000 bytes (524 MB) copied, 4.68907 s, 112 MB/s

real    0m4.690s
user    0m0.120s
sys     0m0.412s

On Mon, Apr 13, 2015 at 2:57 AM, Aleksander Wieliczko <
ale...@mo...> wrote:

>  Hi Peter
>
> Which version of MooseFS are you using?
>
> Can you tell something more about your topology - configuration?
> Is your MooseFS components are in the same LAN ?
> Can you check latency between all MooseFS components?
>
>
> Best regards
> Aleksander Wieliczko
> Technical Support Engineer
> MooseFS.com <http://moosefs.com>
>
>
>  On 12.04.2015 09:52, Peter wrote:
>
>  Hi moosefs team,
>
>  Recently, I setup a moosefs filesystem.
> with the following topology:
>
>  Master server: 1
> Mesta logger server: 1
> chunkserver: 3
>
>  and each file will keep 3 copy.
>
>  base in this topology, I have run some test:
>
>  *and I find that write data to the mfs file system is much faster than
> the reading speed, this test result is confusing me. can you guys explain a
> little bit this for me?*
>
>
>  all the test file are the same size
>  /mnt/mfs --> moosefs file system.
> /data  --> local disk
>
> ---------------------------------------------------------------------------------------------
>
>  [root@TENCENT64 /mnt/mfs]# time cp 111.iso /mnt/mfs/
> cp: cannot stat `111.iso': No such file or directory
>
>  real    0m0.002s
> user    0m0.000s
> sys     0m0.000s
> [root@TENCENT64 /mnt/mfs]# time cp /data/111.iso /mnt/mfs/
>
>  real    0m5.065s
> user    0m0.000s
> sys     0m0.316s
> [root@TENCENT64 /mnt/mfs]# time cp 555.iso /data/
>
>  real    0m11.156s
> user    0m0.000s
> sys     0m0.780s
> [root@TENCENT64 /mnt/mfs]# ^C
> [root@TENCENT64 /mnt/mfs]# time cp 666.iso /data
>
>  real    0m11.165s
> user    0m0.004s
> sys     0m0.864s
> [root@TENCENT64 /mnt/mfs]# time cp /data/555.iso /mnt/mfs/555-new.iso
>
>  real    0m5.045s
> user    0m0.000s
> sys     0m0.300s
> [root@TENCENT64 /mnt/mfs]# ll -h
> total 3.0G
> -rw-r--r-- 1 root root 500M Apr 12 14:36 111.iso
> -rw-r--r-- 1 root root 500M Apr 12 15:33 555-new.iso
> -rw-r--r-- 1 root root 500M Apr 11 01:43 555.iso
> -rw-r--r-- 1 root root 500M Apr 11 01:46 666.iso
> -rw-r--r-- 1 root root 500M Apr 11 01:06 aaa.iso
> -rw-r--r-- 1 root root 500M Apr 11 01:39 bbb.iso
>
>
>  also, i use another client server to run the test , (the client server
> in other city)
>
>  we can see, the read speed is much more slow than the writing speed.
>
>  [root@Tencent-SNG /mnt/mfs]# time cp 111.iso /data/
>
>  real    0m44.736s
> user    0m0.005s
> sys     0m1.299s
> [root@Tencent-SNG /mnt/mfs]# time cp /data/222.iso ./
>
>  [root@Tencent-SNG /data]# time cp 333.iso /mnt/mfs/333-new.iso
>
>  real    0m12.432s
> user    0m0.008s
> sys     0m0.614s
>
>  Also, in case there is some kernel or linux version problem,
> I setup the moosefs master in centos 6.3 and centos 7.0.
> still have the same problem.
>
>  I also search the internet , and find that, the other also have the same
> problem.
>
>  Other people's performance test data:
> Block size 1M Filesize20G
> Client1 write：68.4MB/s  read：25.3MB/s
> Client2 write：67.5MB/s  read：24.7MB/s
>
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live exerciseshttp://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>
>
>
> _________________________________________
> moosefs-users mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>
>

Re: [MooseFS-Users] question about mfs reading speed slower than writing speed???

Fault tolerant, POSIX-compliant, Net Distributed Storage / File System

Re: [MooseFS-Users] question about mfs reading speed slower than writing speed???