[Moosefs-users] I have problem ,Need help,thanks

[lwxian_aha <lwx...@16...>]

today I have new trouble with my new MFS system;
my MFS system  consist of one masterserver,three chunkserver,every chunkserver with 7.2T diskspace; about 1.4T data and about 7 million files 
,every file with 2 copies;

today my one chunkserver 's disk is damaged;
fist time ,the MFS filesystem is work ok; only problem is lots of file have only on copy;

but after about 2 hours , I found that another chunkserver is disappear from the console of mfscgiserv;
then, I restart the chunkserver :
[root@localhost mfs]# ./sbin/mfschunkserver 
working directory: /usr/local/mfs/var/mfs
lockfile created and locked
initializing mfschunkserver modules ...
scanning folder /data/mfsdisk1/ ...
scanning folder /newdata4/mfs/ ...
scanning folder /newdata3/mfs/ ...
/data/mfsdisk1/: 277654 chunks found
/newdata4/mfs/: 2209988 chunks found
/newdata3/mfs/: 2211619 chunks found
scanning complete
main server module: listen on *:9422
stats file has been loaded
mfschunkserver daemon initialized properly


it seems the chunkserver is start  ok,but it still disappear on the console of mfscgiserv;
and then I restart all masterserver and chunkserver ,but after this ,all chunkserver  disappear on the console of mfscgiserv;

bellow is some message:

chunk 1:

[root@localhost mfs]# ./sbin/mfschunkserver 
working directory: /usr/local/mfs/var/mfs
lockfile created and locked
initializing mfschunkserver modules ...
scanning folder /data/mfsdisk1/ ...
scanning folder /newdata4/mfs/ ...
scanning folder /newdata3/mfs/ ...
/data/mfsdisk1/: 277654 chunks found
/newdata4/mfs/: 2209988 chunks found
/newdata3/mfs/: 2211619 chunks found
scanning complete
main server module: listen on *:9422
stats file has been loaded
mfschunkserver daemon initialized properly
[root@localhost mfs]# tail /var/log/messages
May 10 20:36:00 localhost mfschunkserver[15720]: connected to Master
May 10 20:36:04 localhost mfschunkserver[15720]: testing chunk: /data/mfsdisk1/09/chunk_0000000000287409_00000001.mfs
May 10 20:36:10 localhost mfschunkserver[15720]: connecting ...
May 10 20:36:10 localhost mfschunkserver[15720]: connected to Master
May 10 20:36:14 localhost mfschunkserver[15720]: testing chunk: /newdata4/mfs/D1/chunk_000000000005EAD1_00000001.mfs
May 10 20:36:20 localhost mfschunkserver[15720]: connecting ...
May 10 20:36:20 localhost mfschunkserver[15720]: connected to Master
May 10 20:36:24 localhost mfschunkserver[15720]: testing chunk: /newdata3/mfs/86/chunk_0000000000061086_00000001.mfs
May 10 20:36:30 localhost mfschunkserver[15720]: connecting ...
May 10 20:36:30 localhost mfschunkserver[15720]: connected to Master


chunk 2:
[root@localhost mfs]# ./sbin/mfschunkserver 
working directory: /usr/local/mfs/var/mfs
lockfile created and locked
initializing mfschunkserver modules ...
scanning folder /data/mfsdisk1/ ...
scanning folder /newdata2/mfs/ ...
scanning folder /newdata1/mfs/ ...
/data/mfsdisk1/: 312506 chunks found
/newdata2/mfs/: 2182563 chunks found
/newdata1/mfs/: 2182481 chunks found
scanning complete
main server module: listen on *:9422
stats file has been loaded
mfschunkserver daemon initialized properly
[root@localhost mfs]# tail /var/log/messages
May 10 20:33:35 localhost mfschunkserver[25168]: connected to Master
May 10 20:33:44 localhost mfschunkserver[25168]: testing chunk: /newdata1/mfs/79/chunk_000000000008A979_00000001.mfs
May 10 20:33:45 localhost mfschunkserver[25168]: connecting ...
May 10 20:33:45 localhost mfschunkserver[25168]: connected to Master
May 10 20:33:54 localhost mfschunkserver[25168]: testing chunk: /data/mfsdisk1/58/chunk_000000000028A658_00000001.mfs
May 10 20:33:55 localhost mfschunkserver[25168]: connecting ...
May 10 20:33:55 localhost mfschunkserver[25168]: connected to Master
May 10 20:34:05 localhost mfschunkserver[25168]: testing chunk: /newdata2/mfs/D6/chunk_00000000000D05D6_00000001.mfs
May 10 20:34:05 localhost mfschunkserver[25168]: connecting ...
May 10 20:34:05 localhost mfschunkserver[25168]: connected to Master


master:
[root@localhost mfs]# ./sbin/mfsmaster 
working directory: /usr/local/mfs/var/mfs
lockfile created and locked
initializing mfsmaster modules ...
loading sessions ... ok
sessions file has been loaded
exports file has been loaded
loading metadata ...
loading objects (files,directories,etc.) ... ok
loading names ... ok
loading deletion timestamps ... ok
checking filesystem consistency ... ok
loading chunks data ... ok
connecting files and chunks ... ok
all inodes: 7476300
directory inodes: 19151
file inodes: 7457149
chunks: 7456858
metadata file has been loaded
stats file has been loaded
master <-> metaloggers module: listen on *:9419
master <-> chunkservers module: listen on *:9420
main master server module: listen on *:9421
mfsmaster daemon initialized properly
[root@localhost mfs]# tail /var/log/messages             # mass bellow message in the same time
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 158921: photos/47/74/82/r_163042.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 00000000004465CA (inode: 4353226 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 4353226: photos/46/45/10/857020.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 0000000000026B8D (inode: 158922 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 158922: photos/47/62/63/m_853671.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 00000000004465CB (inode: 4353227 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 4353227: photos/46/45/10/s_848116.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 0000000000026B8E (inode: 158923 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 158923: photos/47/62/63/739789.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 00000000004465CC (inode: 4353228 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 4353228: photos/46/91/73/936449.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 00000000000CBC48 (inode: 158924 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 158924: photos/47/74/82/s_171883.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 00000000004465CD (inode: 4353229 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 4353229: photos/46/45/10/l_848116.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 00000000000CBC49 (inode: 158925 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 158925: photos/47/74/82/m_156158.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 00000000004465CE (inode: 4353230 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * currently unavailable file 4353230: photos/46/91/73/m_928481.jpg
May 10 20:34:58 localhost mfsmaster[20168]: currently unavailable chunk 0000000000026B91 (inode: 158926 ; index: 0)
May 10 20:34:58 localhost mfsmaster[20168]: * current