Thread: [Extundelete-users] Issues with recovery
Status: Beta
Brought to you by:
necase
From: <ext...@li...> - 2012-08-09 09:54:54
|
Hi, I recently accidentally removed a folder with photos. Didn't notice at first and used rsync to pull more files from a remote host so the folder was recreated but only with a few new files. Noticed my mistyped command only after checking the folder afterwards. I do not have much experience of data recovery so I unmounted the device and did later remount it (and after reading about ext* recovery it seems like a bad idea). So next I did a complete image of the partition using dd so I won't risk loosing any more data. I do have a complete list of all the files I'm missing but it seems to be unable to restore any of them: # extundelete --restore-file DCIM/20101122_001.jpg sdd1.img WARNING: Extended attributes are not restored. Loading filesystem metadata ... 1172 groups loaded. Loading journal descriptors ... 29894 descriptors loaded. Failed to restore file DCIM/20101122_001.jpg Could not find correct inode number past inode 1843770. I tried --restore-all but it crashes all the time. ("apr" mentioned was/is a _folder_ in DCIM) # gdb --args extundelete --restore-all sdd1.img GNU gdb (Gentoo 7.4.1 p1) 7.4.1 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". For bug reporting instructions, please see: <http://bugs.gentoo.org/>... Reading symbols from /usr/bin/extundelete...Reading symbols from /usr/lib64/debug/usr/bin/extundelete.debug...done. done. (gdb) r Starting program: /usr/bin/extundelete --restore-all sdd1.img warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". WARNING: Extended attributes are not restored. Loading filesystem metadata ... 1172 groups loaded. Loading journal descriptors ... 29894 descriptors loaded. Searching for recoverable inodes in directory / ... 1201 recoverable inodes found. Looking through the directory structure for deleted files ... Unable to restore inode 1312744 (lost+found/20110724_030.jpg): No data found. [snip, lots of similar lines but _none_ match my missing files] Unable to restore inode 1327399 (lost+found/apr): No data found. Program received signal SIGSEGV, Segmentation fault. ext2fs_extent_get (extent=0x7fffffffd808, flags=<optimized out>, handle=0x840a30) at extent.c:379 379 path->left = path->entries; (gdb) bt #0 ext2fs_extent_get (extent=0x7fffffffd808, flags=<optimized out>, handle=0x840a30) at extent.c:379 #1 ext2fs_extent_get (handle=0x840a30, flags=<optimized out>, extent=0x7fffffffd808) at extent.c:275 #2 0x000000000040ad79 in local_block_iterate3 (fs=0x613090, inode=..., flags=<optimized out>, block_buf=0x0, func=<optimized out>, priv_data=0x7fffffffd9a8) at block.c:509 #3 0x0000000000406935 in pair_names_with (fs=0x613090, jfs=0x613090, inolist=std::vector of length 1178, capacity 8192 = {...}, ino=1327399, dirname="lost+found/apr", del=<optimized out>, parent_inos=std::vector of length 2, capacity 2 = {...}) at extundelete.cc:1239 #4 0x0000000000406dff in entry_iterate (entry=3, dirent=0x840d98, priv=0x7fffffffdbd8) at extundelete.cc:1162 #5 0x000000000040b414 in extundelete_process_dir_block (fs=0x613090, blocknr=0x7fffffffdc68, blockcnt=<optimized out>, ref_block=<optimized out>, ref_offset=<optimized out>, priv_data=0x7fffffffdc20) at block.c:1173 #6 0x0000000000406b97 in pair_names_with (fs=0x613090, jfs=0x613090, inolist=std::vector of length 1178, capacity 8192 = {...}, ino=<optimized out>, dirname="", del=<optimized out>, parent_inos=std::vector of length 1, capacity 1 = {...}) at extundelete.cc:1262 #7 0x0000000000407409 in restore_directory (fs=0x613090, jfs=0x613090, dirino=2, dirname="") at extundelete.cc:1369 #8 0x0000000000408dda in examine_fs (fs=0x613090) at extundelete.cc:1596 #9 0x0000000000403a1a in main (argc=1, argv=0x7fffffffeb68) at extundelete.cc:567 (gdb) p path $1 = (struct extent_path *) 0x110001800143838 Unrelated to extundelete is that ext3grep also crashes: # ext3grep --dump-names sdd1.img Running ext3grep version 0.10.2 WARNING: I don't know what EXT3_FEATURE_COMPAT_EXT_ATTR is. ext3grep: ext3grep.cc:119: void run_program(): Assertion `be2le(journal_super_block.s_header.h_magic) == 0xc03b3998U' failed. zsh: abort ext3grep --dump-names /mnt/share5/sdd1.img I'm not really sure how to proceed with this. Any hints regarding this? Since it's only a bunch of jpegs maybe I could look for all jpeg headers in the disk image and extract all files that way? (Not sure how fragmentation would affect this). Loosing the filenames is no issue. |
From: <ext...@li...> - 2012-08-09 23:34:06
|
On 08/09/12 ext...@li... wrote: > I recently accidentally removed a folder with photos. Didn't notice at > first and used rsync to pull more files from a remote host so the folder > was recreated but only with a few new files. > > I do not have much experience of data recovery so I unmounted the device > and did later remount it (and after reading about ext* recovery it seems > like a bad idea). So next I did a complete image of the partition using > dd so I won't risk loosing any more data. > Directory deleted, then created a new directory with the same name, and a umount and a new mount, before create the image copy ????? This is extremely difficult and you need a lot of luck. A file carving tool (eg. photorec) has a good chance to find some of the pictures. The file names can be generated subsequent from the metadata of the images. But such tools will restore also a lot of very old files, and not all are usable. > I do have a complete list of all the files I'm missing but it seems to > be unable to restore any of them: > > # extundelete --restore-file DCIM/20101122_001.jpg sdd1.img > Failed to restore file DCIM/20101122_001.jpg > Could not find correct inode number past inode 1843770. extundelete can not find these files under this name. extundelete uses only the current directory data blocks. But the directory name has been recreated, and very probably the old directory inode is also reused, maybe also the old directory data blocks are reused while writing the new files. I know only a single tool that can use copies of directory data blocks from the journal, ext4magic. But in your case, ext4magic needed very good old journal data, and is very complex and not easy to use. You can try the Manual Mode of the new ext4magic-rescue-TUI, and try to browse backward in time of the root directory, possible you can find the old deleted DCIM directory data and can restore your files. ext3grep and ext4 will not work, and I think extundelete has no chance to recover your files by name. > > I tried --restore-all but it crashes all the time. ("apr" mentioned > was/is a _folder_ in DCIM) > > # gdb --args extundelete --restore-all sdd1.img > Program received signal SIGSEGV, Segmentation fault. > ext2fs_extent_get (extent=0x7fffffffd808, flags=<optimized out>, > handle=0x840a30) at extent.c:379 > 379 path->left = path->entries; > (gdb) bt > #0 ext2fs_extent_get (extent=0x7fffffffd808, flags=<optimized out>, > handle=0x840a30) at extent.c:379 This is a very complicated libext2fs area, a lot more information needed to identify what is going wrong here. No chance to see a error in this gdb-log. robi |
From: <ext...@li...> - 2012-08-10 00:19:22
|
On 08/10/2012 01:53 AM, ext...@li... wrote: > On 08/09/12 ext...@li... wrote: > > I recently accidentally removed a folder with photos. Didn't notice at > > first and used rsync to pull more files from a remote host so the folder > > was recreated but only with a few new files. > > > > I do not have much experience of data recovery so I unmounted the device > > and did later remount it (and after reading about ext* recovery it seems > > like a bad idea). So next I did a complete image of the partition using > > dd so I won't risk loosing any more data. > > > > Directory deleted, then created a new directory with the same name, > and a umount and a new mount, before create the image copy ????? > > This is extremely difficult and you need a lot of luck. Yeah, I accidentally put a partial copy of the folder from a remote host, then I meant to delete the copy but accidentally removed it all ("rm DCIM" vs "rm DCIM/DCIM", followed by rsync). # rsync -av example.net:DCIM DCIM <-- wrong, puts folder in folder # rm -rf DCIM <-- this is where folder is deleted, the wrong one # rsync -av example.net:DCIM/ DCIM <-- correct command, notice trailing slash. Hence, old folder was removed and a new one was created. The remount was simply because I unmounted it from the filesystem and to a temporary location so no application would continue to write to it. > A file carving tool (eg. photorec) has a good chance to find some of the > pictures. > > The file names can be generated subsequent from the metadata of the images. > But such tools will restore also a lot of very old files, and not all > are usable. I have actually managed to recover all but 6 out of 6740 images by searching the disk image for the bytes 0xff 0xd8 0xff 0xe1, ignoring filesizes less than 75k, filtering by camera model from exif and renaming using timestamp. Seems to be kind of what PhotoRec is doing. > > # extundelete --restore-file DCIM/20101122_001.jpg sdd1.img > > Failed to restore file DCIM/20101122_001.jpg > > Could not find correct inode number past inode 1843770. > > extundelete can not find these files under this name. > extundelete uses only the current directory data blocks. > But the directory name has been recreated, and very probably > the old directory inode is also reused, maybe also the old directory > data blocks are reused while writing the new files. Makes sense. > I know only a single tool that can use copies of directory data blocks > from the journal, ext4magic. But in your case, ext4magic needed very good > old journal data, and is very complex and not easy to use. > > You can try the Manual Mode of the new ext4magic-rescue-TUI, > and try to browse backward in time of the root directory, possible you > can find > the old deleted DCIM directory data and can restore your files. > ext3grep and ext4 will not work, and I think extundelete has no chance > to recover your files by name. > > > I tried --restore-all but it crashes all the time. ("apr" mentioned > > was/is a _folder_ in DCIM) > > > > # gdb --args extundelete --restore-all sdd1.img > > Program received signal SIGSEGV, Segmentation fault. > > ext2fs_extent_get (extent=0x7fffffffd808, flags=<optimized out>, > > handle=0x840a30) at extent.c:379 > > 379 path->left = path->entries; > > (gdb) bt > > #0 ext2fs_extent_get (extent=0x7fffffffd808, flags=<optimized out>, > > handle=0x840a30) at extent.c:379 > > This is a very complicated libext2fs area, a lot more information needed to > identify what is going wrong here. No chance to see a error in this gdb-log. I can live with it not being able to recover any files, but it shouldn't crash. Is there is anything I can do to find the issue? I know my way around gdb and I write c, but I'm not familiar with the inner workings of ext[234]. |
From: <ext...@li...> - 2012-08-10 01:59:22
|
On 08/10/2012 01:53 AM, ext...@li... wrote: > I can live with it not being able to recover any files, but it shouldn't > crash. Is there is anything I can do to find the issue? I know my way > around gdb and I write c, but I'm not familiar with the inner workings > of ext[234]. The crash is within a multi-nested function and there are very large data structures and a lot of different and complex pointers. In this area it is very difficult and complicated because there are main differences between ext3 and ext4. And both filesystems are processed there. This must debugged step by step. It would need a dump of your file system and also depends on the exact version of "e2fsprogs" that you use. Is extremely time consuming, and the affected function in the libext2fs is not really public. It is called by a modified private function. This should do the extundelete developer itself. ;-) robi |