Re: [Moosefs-users] HELP!!! 250T Data lost!

[Davies L. <dav...@gm...>]

Hi,

I had try to recover it from metadata.mfs and metadata_ml.mfs.back, but
failed.

Because disk is full, mfsmaster had not dump all the metadata into disks,
had part of nodes in metadata.mfs, no names, no edges, no chunks.
No hope to recover from the broken metadata.mfs, it's too short.

The changelogs are also helpless, only in two days.

The only hope is try to undelete the previous metadata_ml.mfs.back from
metalogger machine, Chenggang had also failed, with extundelete and
etx3grep,
maybe some experts can archive this.

The final options is to GUESS the relation between files and chunks by
chunk id and size fo files, if data lost can not been afforded. Each chunk
is
combined with crc checksum and real data, if we know the relation between
files and chunksever, then we can get data back. Or we can contruct the
metadata according to the GUESS, the using mfsmaster to recover them.

Davies

On Wed, Mar 7, 2012 at 10:56 AM, Ken <ken...@gm...> wrote:

> Hi, chengang
>
> I think you should try more, and post detail here. Someone must resolve it.
> Maybe you will lost some data in last few minutes, but 250T should be
> saved.
>
> At first, BACKUP all files:
>   /var/lib/mfs/* on master
>   /var/lib/mfs/* on mfsmetalogger
>
> about restore error:
>
> file 'metadata.mfs.back' not found - will try 'metadata_ml.mfs.back'
>> instead
>> loading objects (files,directories,etc.) ... loading node: read error:
>> ENOENT (No such file or directory)
>> error
>> can't read metadata from file: .//metadata_ml.mfs.back
>>
>  How did you run mfsmetarestore?  add -d options?
> If stat(datapath + metadata_ml.mfs.back) fail, these error will occur.
> Maybe use strace will show why stat fail exactly.
>
> ps: I am in Beijing now and I can provide more help.
>
> HTH
>
> -Ken
>
>
>
> On Wed, Mar 7, 2012 at 10:21 AM, 陈钢 <yik...@gm...> wrote:
>
>> can not start mfsmaster with the file "78962688 Mar  6 17:18 metadata.mfs
>> "..
>> i tried that. :(
>>
>> 2012/3/7 Ricardo J. Barberis <ric...@da...>
>>
>> El Martes 06/03/2012, 陈钢 escribió:
>>> > on master
>>> [ ... ]
>>> > -rw-r-----  1 mfs  mfs   78962688 Mar  6 17:18 metadata.mfs
>>> > -rw-r--r--  1 root root         8 Jul  4  2011 metadata.mfs.empty
>>> > -rw-r-----  1 mfs  mfs       5984 Mar  6 12:00 sessions.mfs
>>> > -rw-r-----  1 mfs  mfs          0 Mar  6 16:46 sessions.mfs.tmp
>>> > -rw-r-----  1 mfs  mfs     131072 Mar  6 17:18 stats.mfs
>>>
>>> You have /var/lib/mfs/metadata.mfs on the master, it might not be corrupt
>>> after all?
>>>
>>> I'd suggest:
>>>
>>> - backup /var/lib/mfs to another disk/server (for later recovery if
>>> needed)
>>> - make sure you have free space in your main disk
>>> - then simply try to start mfsmaster
>>> - check mfs.cgi (web interface) if it looks OK
>>>
>>>
>>> BUT: if you can, wait for confirmation from Michał Borychowski first, in
>>> case
>>> what I'm telling you is not safe.
>>>
>>>
>>> (BTW: You have Reply-To set to che...@cp..., I don't know if that's
>>> intentional on your part)
>>>
>>> Hope it helps,
>>> --
>>> Ricardo J. Barberis
>>> Senior SysAdmin / ITI
>>> Dattatec.com :: Soluciones de Web Hosting
>>> Tu Hosting hecho Simple!
>>>
>>> ------------------------------------------
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>>
>> _______________________________________________
>> moosefs-users mailing list
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>>
>>
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>


-- 
 - Davies