[Moosefs-users] Chunk disappeared

[Jens K. S. <je...@me...>]

Hi all,

I'm running a small mfs setup with 3 chunkservers totalling approx. 18 
TB of capacity.

Today I ran this command on a directory with a lot of files that were 
previously goal 2:

   mfssetgoal -r 1 backups

I received this error message:

   backups: master query: receive error

Checking with mfsgetgoal it seems the command was executed fine as the 
goal was indeed 1 on the files I checked.

However when I went to the mfs web interface I found problems. Before 
running the command everything looked fine in the web inerface.

First I noticed that a lot of blocks had 2 valid copies but only goal 1. 
By refreshing I could see blocks going from the "blue" state to the 
"green" state. This was as expected.

Quite unexpected is the fact that I now had 1 chunk with 0 valid copies 
and goal 1. My understanding of this is that I have now irreversibly 
lost a chunk of data?

Is there any way I can find out which file(s) were completely or 
partially lost?

Another weird thing is that I now have 2 chunks with only 1 valid copy 
even though their goal is 2. They are shown in "orange" state in the web 
interface. How come this happened?

Is it normal when changing the goal of files? (i.e. a chunk that 
contains files that now have different goals)

It seems that moving the over-goal chunks to on-goal state is 
prioritized above moving my 2 under-goal chunks. Is this correct?

As I re-goaled about 10.000.000 chunks it will be a while before my 2 
under-goal chunks are replicated.

-- 
Jens Kristian Søgaard, Mermaid Consulting ApS,
je...@me...,
http://www.mermaidconsulting.com/