Re: [Moosefs-users] chunkserver can't connect to masterserver

[l. <lwx...@16...>]

thanks for you help!

I think,Why not make the "block size of 64KiB" be a Parameter of the MFS system;  there will result of less waste of space for " small file ";
Many "hard-coded " values In MFS Can Changed to be a parameter of config file, So the system will be more flexible;

Regards
lwxian_aha 

2010-05-14 



lwxian_aha 



发件人： Michał_Borychowski 
发送时间： 2010-05-14  13:34:12 
收件人： 'lwxian_aha' 
抄送： 'moosefs-users' 
主题： RE: [Moosefs-users] chunkserver can't connect to masterserver 
 
 
 
From: lwxian_aha [mailto:lwx...@16...] 
Sent: Thursday, May 13, 2010 8:22 AM
To: Michał_Borychowski
Cc: moosefs-users
Subject: Re: [Moosefs-users] chunkserver can't connect to masterserver
 
with you suggestions, I have fixes the problem.thanks!!!
[MB] We are very happy that the patch has helped you!
   
On my product system ,there is a lots of small files (less then 10K), so there are about 5,000,000 chunks with only 1.4T DATA;
I make suggestions :
1, the first way   :you can separate the "registration packet " for serveral times ,every time limit to 500,000; so we will have no limit with the amount of chunks every chunkserver;
[MB] That’s what we thought of. Thank you for the suggestion.
 
2, the second way :  maybe you can store several files in one chunks(every chunk limits to 64M)? so you can reduce the amount of chunks?
[MB] It looks interesting but in case of a modification of a file in “the middle” of the chunk it would bring a big problem
 
 
Here ,I have another question:
on my chunkserver disk,I have see the smallest file of chunks is 70656, and my smallest source file is only about 4K; why the chunk file size is substantially exceeds the soursefile;
[MB] The system was initially designed for keeping large amounts (like several thousands) of very big files (of tens of gigabytes) and has a hard-coded chunk size of 64MiB and block size of 64KiB. That’s why even a small file will occupy 64KiB plus additionally 4KiB of checksums and 1KiB for the header. The whole transfer which takes place in the system is done in blocks of 64KiB. However it doesn’t have any impact on the performance (in a normal filesystem it is also usual to read ahead some superfluous data).
 
The issue regarding the occupied space is really more significant, but in our opinion it is still negligible. Let’s take 25 million files with a goal set to 2, so we have about 50 million of “last” chunks. If in every chunk we lose even 64KiB we will have an overall waste of 3.2TB which nowadays should not be a very big concern.
 
In your case 10,000 files with goal=2 would produce maximally 1.2 GB of wasted space. It is still perfect to keep source files on the MooseFS system if they are going to be served somewhere or just stored and developed there. 
 
If you need any further assistance please let us know.
 
Regards
Michał Borychowski 
 
 
thanks !!
 
2010-05-13 



lwxian_aha 



发件人： Michał_Borychowski 
发送时间： 2010-05-13  01:41:50 
收件人： 'lwxian_aha'; 'moosefs-users' 
抄送： 
主题： RE: [Moosefs-users] chunkserver can't connect to masterserver 
You have lots of chunks on your chunkservers. Probably the registration packet for the master is too big and the master server rejects it.
 
You need to change in the "matocsserv.c" file in "mfsmaster" folder this line:
#define MaxPacketSize 50000000
 
into this:
#define MaxPacketSize 200000000
 
After changing source you need to recompile the master server and restart it.
 
Generally on one chunkserver there are about 500,000 chunks. For this amount it is necessary to allocate 6 million bytes. Limit of 50 million seemed quite reasonable. But you have about 5,000,000 chunks on one chunkserver which demands about 60 million bytes for sending information about these chunks. And you exceed this limit. That's why we suggest to increase it to 200 million bytes. 
 
Let us know if it fixes your problem.
 
 
Kind regards
Micha?Borychowski 
 
 
 
 
From: lwxian_aha [mailto:lwx...@16...] 
Sent: Tuesday, May 11, 2010 5:34 AM
To: moosefs-users
Subject: [Moosefs-users] chunkserver can't connect to masterserver
 
today I have new trouble with my new MFS system,chunkserver can't connect to masterserver;
 
my MFS system  consist of one masterserver,three chunkserver,every chunkserver with 7.2T diskspace; about 1.4T 
data and about 7 million files ,every file with 2 copies;
MFS version is 1.6.11
OS version CENTOS 5.0
FS is ext3;
 
Following is the error message:
 
[root@localhost mfs]# tail /var/log/messages 
May 11 10:57:00 localhost mfsmaster[25802]: server 1 (ip: 192.168.10.23, port: 9422): usedspace: 33326616576 
(31.04 GiB), totalspace: 140025790464 (130.41 GiB), usage: 23.80%
May 11 10:57:00 localhost mfsmaster[25802]: total: usedspace: 33326616576 (31.04 GiB), totalspace: 140025790464 
(130.41 GiB), usage: 23.80%
May 11 10:57:03 localhost mfsmaster[25802]: CS(192.168.10.21) packet too long (56130653/50000000)
May 11 10:57:03 localhost mfsmaster[25802]: chunkserver disconnected - ip: 192.168.10.21, port: 0, usedspace: 0 
(0.00 GiB), totalspace: 0 (0.00 GiB)
May 11 10:57:08 localhost mfsmaster[25802]: CS(192.168.10.21) packet too long (56130653/50000000)
May 11 10:57:08 localhost mfsmaster[25802]: chunkserver disconnected - ip: 192.168.10.21, port: 0, usedspace: 0 
(0.00 GiB), totalspace: 0 (0.00 GiB)
May 11 10:57:13 localhost mfsmaster[25802]: CS(192.168.10.21) packet too long (56130653/50000000)
May 11 10:57:13 localhost mfsmaster[25802]: chunkserver disconnected - ip: 192.168.10.21, port: 0, usedspace: 0 
(0.00 GiB), totalspace: 0 (0.00 GiB)
May 11 10:57:18 localhost mfsmaster[25802]: CS(192.168.10.21) packet too long (56130653/50000000)
May 11 10:57:18 localhost mfsmaster[25802]: chunkserver disconnected - ip: 192.168.10.21, port: 0, usedspace: 0 
(0.00 GiB), totalspace: 0 (0.00 GiB)
 
 
 
what's happen ?need you help!
thanks a lot's
 
2010-05-05 
________________________________________
lwxian_aha