Menu
▾
▴
Re: [Moosefs-users] HELP!!! 250T Data lost!
|
From: Olivier T. <Oli...@lm...> - 2012-03-08 13:17:40
|
Hi, Did you solve your problem ? I had few days ago a mfsmaster crash which went out of memory. When I tried to restart, it crashed saying that there was no metadata.mfs file. I tried "mfsmetarestore -a". It didn't work, saying that metadata.mfs.back was corrupted. There was a metadata.mfs.back file and a metadata.mfs.back.tmp file. metadata.mfs.back was half the size it should be. I restored from a daily backup the metadata.mfs file, then ran again 'mfsmetarestore -a', and this time, it worked. I could then start mfsmaster successfully. Did you tried that ? I mean, just restore the latest working metadata.mfs file ? HTH. Best regards, Olivier Le 07/03/12 04:20, 陈钢 a écrit : > In master`s metadata.mfs,i saw nodes part is complete, part of names, no free, > no chunks.. > > 2012/3/7 Davies Liu <dav...@gm... <mailto:dav...@gm...>> > > Hi, > > I had try to recover it from metadata.mfs and metadata_ml.mfs.back, but failed. > > Because disk is full, mfsmaster had not dump all the metadata into disks, > had part of nodes in metadata.mfs, no names, no edges, no chunks. > No hope to recover from the broken metadata.mfs, it's too short. > > The changelogs are also helpless, only in two days. > > The only hope is try to undelete the previous metadata_ml.mfs.back from > metalogger machine, Chenggang had also failed, with extundelete and etx3grep, > maybe some experts can archive this. > > The final options is to GUESS the relation between files and chunks by > chunk id and size fo files, if data lost can not been afforded. Each chunk is > combined with crc checksum and real data, if we know the relation between > files and chunksever, then we can get data back. Or we can contruct the > metadata according to the GUESS, the using mfsmaster to recover them. > > Davies > > On Wed, Mar 7, 2012 at 10:56 AM, Ken <ken...@gm... > <mailto:ken...@gm...>> wrote: > > Hi, chengang > > I think you should try more, and post detail here. Someone must resolve it. > Maybe you will lost some data in last few minutes, but 250T should be saved. > > At first, BACKUP all files: > /var/lib/mfs/* on master > /var/lib/mfs/* on mfsmetalogger > > about restore error: > > file 'metadata.mfs.back' not found - will try 'metadata_ml.mfs.back' > instead > loading objects (files,directories,etc.) ... loading node: read > error: ENOENT (No such file or directory) > error > can't read metadata from file: .//metadata_ml.mfs.back > > How did you run mfsmetarestore? add -d options? > If stat(datapath + metadata_ml.mfs.back) fail, these error will occur. > Maybe use strace will show why stat fail exactly. > > ps: I am in Beijing now and I can provide more help. > > HTH > > -Ken > > > > On Wed, Mar 7, 2012 at 10:21 AM, 陈钢 <yik...@gm... > <mailto:yik...@gm...>> wrote: > > can not start mfsmaster with the file "78962688 Mar 6 17:18 > metadata.mfs ".. > i tried that. :( > > 2012/3/7 Ricardo J. Barberis <ric...@da... > <mailto:ric...@da...>> > > El Martes 06/03/2012, 陈钢 escribió: > > on master > [ ... ] > > -rw-r----- 1 mfs mfs 78962688 Mar 6 17:18 metadata.mfs > > -rw-r--r-- 1 root root 8 Jul 4 2011 metadata.mfs.empty > > -rw-r----- 1 mfs mfs 5984 Mar 6 12:00 sessions.mfs > > -rw-r----- 1 mfs mfs 0 Mar 6 16:46 sessions.mfs.tmp > > -rw-r----- 1 mfs mfs 131072 Mar 6 17:18 stats.mfs > > You have /var/lib/mfs/metadata.mfs on the master, it might not > be corrupt > after all? > > I'd suggest: > > - backup /var/lib/mfs to another disk/server (for later recovery > if needed) > - make sure you have free space in your main disk > - then simply try to start mfsmaster > - check mfs.cgi (web interface) if it looks OK > > > BUT: if you can, wait for confirmation from Michał Borychowski > first, in case > what I'm telling you is not safe. > > > (BTW: You have Reply-To set to che...@cp... > <mailto:che...@cp...>, I don't know if that's > intentional on your part) > > Hope it helps, > -- > Ricardo J. Barberis > Senior SysAdmin / ITI > Dattatec.com :: Soluciones de Web Hosting > Tu Hosting hecho Simple! > > ------------------------------------------ > > > > ------------------------------------------------------------------------------ > Virtualization & Cloud Management Using Capacity Planning > Cloud computing makes use of virtualization - but cloud computing > also focuses on allowing computing to be delivered as a service. > http://www.accelacomm.com/jaw/sfnl/114/51521223/ > > _______________________________________________ > moosefs-users mailing list > moo...@li... > <mailto:moo...@li...> > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > ------------------------------------------------------------------------------ > Virtualization & Cloud Management Using Capacity Planning > Cloud computing makes use of virtualization - but cloud computing > also focuses on allowing computing to be delivered as a service. > http://www.accelacomm.com/jaw/sfnl/114/51521223/ > _______________________________________________ > moosefs-users mailing list > moo...@li... > <mailto:moo...@li...> > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > > -- > - Davies > > > > > ------------------------------------------------------------------------------ > Virtualization& Cloud Management Using Capacity Planning > Cloud computing makes use of virtualization - but cloud computing > also focuses on allowing computing to be delivered as a service. > http://www.accelacomm.com/jaw/sfnl/114/51521223/ > > > > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
