Menu
▾
▴
Re: [Moosefs-users] Purging missing chunks
|
From: Karl Z. <pv...@gm...> - 2012-06-13 13:37:43
|
Hi Olivier, Thanks for the response. The solution Chris gave of walking through the moose files worked and I was able to locate and remove all files quite quickly actually. The next time I run into this, which I hope is never. I will have 2 great and easy ways to take care of this. Thank you both so much for your help. Cheers, Karl 2012/6/13 Olivier Thibault <Oli...@lm...> > Hi Karl, > > Le 11/06/2012 18:45, Karl Zander a écrit : > > Hi Chris, >> >> Thanks for the fast response. I'll give it a shot but there must be a >> way that >> isn't so time consuming. My moosefs is 500TB and I'd imagine this >> operation >> could take days or a week. I'll get it started now and report back with >> my >> findings. >> > > As far as I understand, logs on the mfsmaster shows the missing chunk > immediatly followed by the files that are stored in this chunk. > Logs on my mfsmaster look like this : > Jun 12 09:29:09 manganese mfsmaster[18364]: currently unavailable chunk > 00000000003DDE38 (inode: 5122921 ; index: 0) > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu-**lmpt/75/f%252f/fusr/fshare/** > fxfig/fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu-**lmpt/0/f%252f/fusr/fshare/** > fxfig/fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu-**lmpt/49/f%252f/fusr/fshare/** > fxfig/fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu-**lmpt/56/f%252f/fusr/fshare/** > fxfig/fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu-**lmpt/64/f%252f/fusr/fshare/** > fxfig/fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu/0/**f%252f/fusr/fshare/fxfig/** > fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu/6/**f%252f/fusr/fshare/fxfig/** > fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/pc/ubuntu/8/**f%252f/fusr/fshare/fxfig/** > fLibraries/fFlowchart/**foffline_storage.fig > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 5122921: 2copies/backuppc/cpool/9/b/f/**9bf659facce54906ee6f3dd1b1794e**5e > Jun 12 09:29:09 manganese mfsmaster[18364]: currently unavailable chunk > 00000000000A1073 (inode: 928652 ; index: 0) > Jun 12 09:29:09 manganese mfsmaster[18364]: * currently unavailable file > 928652: 3copies/home_etu/20900601t/.**dbus/session-bus/** > 0ddb87ec776a0a55b80e2b184c0916**5c-1 > > So you can see here two missing chunks : 00000000003DDE38 and > 00000000000A1073, with their inode number, respectively 5122921 and 928652. > > What I did in such case was simply remove those files. (i know they have > been stored in other chunks lately). > Before deletion, i set the trashtime to 1 second so the files don't even > stay in the trash space. > > Automated way could then be something like : > > #!/bin/bash > > # Go to the mfs root > cd /mfsroot/ > > for f in `grep "currently unavailable file" mfsmaster.log | cut -d ' ' -f > 11-`; do > mfssettrashtime 1 $f > rm -f $f > done > > It may be necessary to escape some character in the file name (such as > space for example). > Before doing this, I would suggest to be sure that every chunkservers are > online and every disks/partitions of each chunkserver are available, in > order to be sure that listed missing files are really missing in the > moosefs space. > > Comments on this way of practice are welcome. > It worked very nice for me, but i am not sure it's the best way. > > Regards, > > Olivier > > > > > >> Regards, >> >> Karl >> >> On Jun 11, 2012 12:39 PM, "Chris Picton" <ch...@ec... >> <mailto:ch...@ec...>**> wrote: >> >> Hi Karl >> >> I use the following command to find files which have 0 copy chunks: >> >> find /mnt/mfs/root/ -type f | xargs mfscheckfile | grep -B1 ^0 | grep >> :$ | >> sed 's/:$//g' >> >> In my case, I have mounted the root of mfs on /mnt/mfs/root >> >> If the above finds no files, they have to be in trash (afaik) >> >> Chris >> >> On 2012/06/11 3:47 PM, Karl Zander wrote: >> >>> Hi MooseFS Users, >>> >>> I'm hoping someone has experienced this and can help me out. I have a >>> fairly large Moose and I lost 2 nodes due to hardware failure. I know >>> have some 30,000 missing chunks on my moosefs and I have no idea how >>> to >>> clear them. I've tried recursively checking each file with >>> mfsfileinfo >>> and I can't seem to find the missing chunks/files. I have files >>> listed on >>> the web interface that are missing but I've already deleted them and >>> the >>> trash time has already been exceeded. I'm finding this to be very >>> inefficient. Isn't there any easier way to delete chunks that are >>> offline >>> and never will be online again? >>> >>> Also, is there a better way to query the moose besides the web >>> interface? >>> I'd like to be able to call this information using scripts and the >>> mfs* >>> commands don't give me everything I need. I appreciate any help! >>> Thanks. >>> >>> MooseFS version 1.6.20-8 >>> >>> >>> Karl >>> >>> >>> ------------------------------**------------------------------** >>> ------------------ >>> Live Security Virtual Conference >>> Exclusive live event will cover all the ways today's security and >>> threat landscape has changed and how IT managers can respond. >>> Discussions >>> will include endpoint security, mobile security and the latest in >>> malware >>> threats.http://www.accelacomm.**com/jaw/sfrnl04242012/114/**50122263/<http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/> >>> >>> >>> ______________________________**_________________ >>> moosefs-users mailing list >>> moosefs-users@lists.**sourceforge.net<moo...@li...> <mailto: >>> moosefs-users@lists.**sourceforge.net<moo...@li...> >>> > >>> https://lists.sourceforge.net/**lists/listinfo/moosefs-users<https://lists.sourceforge.net/lists/listinfo/moosefs-users> >>> >> >> >> >> ------------------------------**------------------------------** >> ------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/**sfrnl04242012/114/50122263/<http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/> >> >> >> >> ______________________________**_________________ >> moosefs-users mailing list >> moosefs-users@lists.**sourceforge.net<moo...@li...> >> https://lists.sourceforge.net/**lists/listinfo/moosefs-users<https://lists.sourceforge.net/lists/listinfo/moosefs-users> >> >> > > -- > Olivier THIBAULT > Université François Rabelais - UFR Sciences et Techniques > Laboratoire de Mathématiques et Physique Théorique (UMR CNRS 7350) > Service Informatique de l'UFR > Parc de Grandmont > 37200 Tours - France > Email: olivier.thibault at lmpt.univ-tours.fr > Tel: (33)(0)2 47 36 69 12 > Fax: (33)(0)2 47 36 70 68 > > > > |
