Re: [Moosefs-users] Non-existing chunks hang the mfsmaster

[Michal B. <mic...@ge...>]

Hi!

 

How does your metadata file look like? What do you have inside
"/usr/local/var/mfs" (PREFIX/var/mfs) directory?

 

 

Regards

Michał 

 

 

From: Stas Oskin [mailto:sta...@gm...] 
Sent: Sunday, February 27, 2011 8:25 PM
To: moosefs-users
Subject: [Moosefs-users] Non-existing chunks hang the mfsmaster

 

Hi.

We got a very strange that happened on our test cluster.

After a power crash, the mfsmaster syslog is full of following errors:
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005313A2_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005393A2_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005493A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005293A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005513A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005313A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005113A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005093A3_00000001), so create it for future deletion

These errors appears all the time, and practically hang the mfsmaster.
mfscgi stops working (hangs), and mounts are aborting with following error:
error receiving data from mfsmaster: ETIMEDOUT (Operation timed out)
error receiving data from mfsmaster: ETIMEDOUT (Operation timed out)


Upgrading to .20 didn't help.

Any idea what this could be and how to resolve it?

Thanks.