sleuthkit-users

[sleuthkit-users] filesystem recovery using sleuthkit

[Jim C. <jc...@di...>]

Im trying to recover contents of a 60GB Windows XP installation
after a grub accident.

A short 'how I got here' follows, then the Questions.

I recently screwed up (technical jargon) my dads winXP box
while trying to put grub on a usb-drive.
(grub went onto /dev/hda, but linux was on /dev/sda,
so it wouldnt boot anything)

I recovered using the recovery disk, tried the repair option:
fixmbr - to restore the MBR, and fixboot (I think)

These tools apparently trashed the root of C:\,
they showed me a mostly empty root-dir, with 4 (empty)
files with names that were evidently unreadable by the OS
(they were represnted by ?, all 4 of them)

with a knoppix disk and a 3 line perl prog,
I took 1GB at a time off it, and NFS'd it to my laptop,
then onto the USB drive  (knoppix dd wouldnt handle > 2GB files)
(knoppix 3.6 wouldnt mount the usb directly - some versioning
prob w the usb-* modules)

I catted all the 1gb chunks together, and have started autopsy-2.03
and sleuthkit-1.73 on it.

Evidence Locker: /mnt/u2/forensics
Start Time: Sun Jan  9 09:32:31 2005
Remote Host: localhost
Local Port: 9999

Open an HTML browser on the remote host and paste this URL in it:

    http://localhost:9999/autopsy

Keep this process running and use <ctrl-c> to exit
..//bin/fsstat: $Data not found while loading the MFT
..//bin/fsstat: Invalid FAT32 image (numroot != 0)
..//bin/fsstat: Invalid FAT32 image (numroot != 0)
..//bin/fsstat: $Data not found while loading the MFT
..//bin/fsstat: Invalid FAT32 image (numroot != 0)
..//bin/fsstat: getFAT: return FAT16 cluster too large


fsstat seems to be having trouble with this partition, and its unclear
to me what kind of filesystem is *really* there.

fdisk told me it was NTFS/HPFS
mount told me it was vfat
autopsy thinks its FAT16, would *not* accept choice of NTFS or FAT32

it appears that this is due to the root directory, which appears to
have been overwritten by the MS tools.  Ive inspected other
parts of the 60 GB, they appear to have the original content.

Qs:

is it common on MS OSs that partition type and fs type are different ?

is the choice of filesystem completely dictated by the root-directory ?
is the choice overridable ?
is there a reason its not automatic - since it seems to reject the 
'wrong' choices ?

is it possible to not care ?
in my case, I hope to find that the vast majority of directories
look to be uncorrupted - ie the directory entries are legal,
and point to files that match whatever metadata is stored in the 
directory entry itself,
and link to each other properly.
IOW, if 59.9 GB of data look consistent with each other,
then clearly the root folder is hosed, and is only thing to fix.


wrt:
Calculating MD5 of images/whole.img (this could take a while)
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+-

it would be nice to have some estimate of ' a while',
ie how big the ticks are,



my other attempts to extract some of the original filesystem
have also run afoul on this FAT16 choice:

..//bin/ils: getFAT: return FAT16 cluster too large
..//bin/dls: getFAT: return FAT16 cluster too large


Im gonna continue poking, 1st thru autopsy, then the sleuthkit tools,
but Im also contemplating trying to remove/zero out the root directory
so that something else in the partition has a chance to influence the mount.
Please advise me on this course of action b4 I do something (else) stupid

tia
jimc

Re: [sleuthkit-users] filesystem recovery using sleuthkit

[Brian C. <ca...@sl...>]

On Jan 9, 2005, at 12:50 PM, Jim Cromie wrote:
> Qs:
>
> is it common on MS OSs that partition type and fs type are different ?

No.  Windows uses the partition type to determine if it will mount the  
partition and how to mount it.

> is the choice of filesystem completely dictated by the root-directory ?

The other way around.  Each file system has its own way of storing the  
contents of a directory and FAT and NTFS are MUCH different.

> is the choice overridable ?
> is there a reason its not automatic - since it seems to reject the  
> 'wrong' choices ?

Autopsy does some sanity checks when you import the image so that you  
can more quickly determine when you select the wrong one (I am  
currently working on autodetect for the next version).   FAT and NTFS  
(and partition tables) are difficult because they all use the same  
signature value in the last two bytes of the first sector.  FYI -  
choosing just fat will autodetect which type of FAT.

> is it possible to not care ?
> in my case, I hope to find that the vast majority of directories
> look to be uncorrupted - ie the directory entries are legal,
> and point to files that match whatever metadata is stored in the  
> directory entry itself,
> and link to each other properly.

I have no clue what happened during the Windows recovery stuff.  TSK /  
Autopsy does not fix damaged file systems.  It only shows what is  
there.  I think you need to look into some FAT/NTFS fixing tools.

Just to confirm, you did import the partition image and not the disk  
image right?  The current version supports only partition images.  The  
'$Data not found while loading the MFT' is sometimes found when you try  
to process a disk image as NTFS (because they share the same magic  
value).


> wrt:
> Calculating MD5 of images/whole.img (this could take a while)
> +----+----+----+----+----+----+----+----+----+----+----+----+----+---- 
> +-
>
> it would be nice to have some estimate of ' a while',
> ie how big the ticks are,

The ticks are every few seconds, not a percentage.  Percentage would be  
nice, but difficult given the current design.

brian

Re: [sleuthkit-users] filesystem recovery using sleuthkit

[Jim C. <jc...@di...>]

Brian Carrier wrote:

>
> On Jan 9, 2005, at 12:50 PM, Jim Cromie wrote:
>
>> Qs:
>

>> is the choice of filesystem completely dictated by the root-directory ?
>
>
> The other way around.  Each file system has its own way of storing 
> the  contents of a directory and FAT and NTFS are MUCH different.
>
poorly stated question.  In your tool, which will work on an image file, 
there is no
partition type.  So Im assuming that the code looks for magic bytes in 
1st few sectors
of the image.  Id expect that magic would be part of the root directory.

So rephrasing;  does your tool have any mechanizm to look for a backup 
superblock ?
(they exist in ext2 anyway)


>> is the choice overridable ?
>> is there a reason its not automatic - since it seems to reject the  
>> 'wrong' choices ?
>
>
> Autopsy does some sanity checks when you import the image so that you  
> can more quickly determine when you select the wrong one (I am  
> currently working on autodetect for the next version).   

Ill check for the sourceforge CVS

> FAT and NTFS  (and partition tables) are difficult because they all 
> use the same  signature value in the last two bytes of the first 
> sector.  FYI -  choosing just fat will autodetect which type of FAT.
>
>> is it possible to not care ?
>> in my case, I hope to find that the vast majority of directories
>> look to be uncorrupted - ie the directory entries are legal,
>> and point to files that match whatever metadata is stored in the  
>> directory entry itself,
>> and link to each other properly.
>
>
> I have no clue what happened during the Windows recovery stuff.  TSK 
> /  Autopsy does not fix damaged file systems.  It only shows what is  
> there.  I think you need to look into some FAT/NTFS fixing tools.
>
:-(  my opensource options appear quite limited.
anybody know of anything that *might* help ?

> Just to confirm, you did import the partition image and not the disk  
> image right?  The current version supports only partition images.  
> The  '$Data not found while loading the MFT' is sometimes found when 
> you try  to process a disk image as NTFS (because they share the same 
> magic  value).
>
>
yes - I double checked that when I saw the NTFS 'superblock' in there.
the script was definitely using /dev/hda2

for my $chunk (0..60) {
    print "dd bs=1K if=/dev/hda2 of=part.$chunk skip=${chunk}M count=1M\n";
    print `dd bs=1K if=/dev/hda2 of=part.$chunk skip=${chunk}M count=1M`;
    print `ls`;
}

At this point, I think Im gonna try slicing off that FAT12 chunk,
much like Id do if the big image were the whole of /hda.

If it works, I will *certainly* report back.

>
> brian
>
thanks
Jim Cromie

Re: [sleuthkit-users] filesystem recovery using sleuthkit

[Brian C. <ca...@sl...>]

On Jan 10, 2005, at 11:24 AM, Jim Cromie wrote:

> Brian Carrier wrote:
>
>>
>> On Jan 9, 2005, at 12:50 PM, Jim Cromie wrote:
>>
>>> Qs:
>>
>
>>> is the choice of filesystem completely dictated by the 
>>> root-directory ?
>>
>>
>> The other way around.  Each file system has its own way of storing 
>> the  contents of a directory and FAT and NTFS are MUCH different.
>>
> poorly stated question.  In your tool, which will work on an image 
> file, there is no
> partition type.  So Im assuming that the code looks for magic bytes in 
> 1st few sectors
> of the image.  Id expect that magic would be part of the root 
> directory.

Nope.  Magic / sigs are all in the boot sector / superblock in the 
first few sectors.  The root directory has the same structure as a 
normal directory.

> So rephrasing;  does your tool have any mechanizm to look for a backup 
> superblock ?
> (they exist in ext2 anyway)

The current version does not.  I am currently adding it to v2.

brian

Re: [sleuthkit-users] filesystem recovery using sleuthkit

[Linux T. <lin...@ya...>]

--- Brian Carrier <ca...@sl...> wrote:
> Nope.  Magic / sigs are all in the boot sector /
> superblock in the 
> first few sectors.  The root directory has the same
> structure as a 
> normal directory.
> 

QUESTION - what is the magic signature for FAT12/16/32
and NTFS?  I can't find it.  So I guess I'm asking
what and where do you look Brian?


BACKUP SUPERBLOCK - use 'findsuper' to locate back up
superblock.  Works good for ext2/3 fs.


-lt



		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - now with 250MB free storage. Learn more.
http://info.mail.yahoo.com/mail_250

Re: [sleuthkit-users] filesystem recovery using sleuthkit

[Brian C. <ca...@sl...>]

On Feb 6, 2005, at 12:53 PM, Linux Tard wrote:

> --- Brian Carrier <ca...@sl...> wrote:
>> Nope.  Magic / sigs are all in the boot sector /
>> superblock in the
>> first few sectors.  The root directory has the same
>> structure as a
>> normal directory.
>>
>
> QUESTION - what is the magic signature for FAT12/16/32
> and NTFS?  I can't find it.  So I guess I'm asking
> what and where do you look Brian?

FAT12/16/32, NTFS, and DOS Partitions all have the same signature 
value, which is 0x55AA in the last two bytes of the first sector (there 
are also several data structures within FAT32 that use the same 
signature).   The new autodetect code in v2 first checks the signature 
value and then checks other values to narrow down NTFS versus FAT, such 
as valid cluster size and FAT size.  FAT and NTFS also have a label in 
the boot sector that typically contains "FAT12" or "NTFS", but this 
label is not required to exist or to be accurate, so it isn't a good 
test.

brian

Re: [sleuthkit-users] filesystem recovery using sleuthkit

[Rich T. <te...@ap...>]

You may have already tried, but I'll say it anyway -
Did you try booting with an XP boot disk and doing an
fdisk mbr???  That should return the mbr for windows
allowing you to boot....

Rich

--- Jim Cromie <jc...@di...> wrote:

> 
> Im trying to recover contents of a 60GB Windows XP
> installation
> after a grub accident.
> 
> A short 'how I got here' follows, then the
> Questions.
> 
> I recently screwed up (technical jargon) my dads
> winXP box
> while trying to put grub on a usb-drive.
> (grub went onto /dev/hda, but linux was on /dev/sda,
> so it wouldnt boot anything)
> 
> I recovered using the recovery disk, tried the
> repair option:
> fixmbr - to restore the MBR, and fixboot (I think)
> 
> These tools apparently trashed the root of C:\,
> they showed me a mostly empty root-dir, with 4
> (empty)
> files with names that were evidently unreadable by
> the OS
> (they were represnted by ?, all 4 of them)
> 
> with a knoppix disk and a 3 line perl prog,
> I took 1GB at a time off it, and NFS'd it to my
> laptop,
> then onto the USB drive  (knoppix dd wouldnt handle
> > 2GB files)
> (knoppix 3.6 wouldnt mount the usb directly - some
> versioning
> prob w the usb-* modules)
> 
> I catted all the 1gb chunks together, and have
> started autopsy-2.03
> and sleuthkit-1.73 on it.
> 
> Evidence Locker: /mnt/u2/forensics
> Start Time: Sun Jan  9 09:32:31 2005
> Remote Host: localhost
> Local Port: 9999
> 
> Open an HTML browser on the remote host and paste
> this URL in it:
> 
>     http://localhost:9999/autopsy
> 
> Keep this process running and use <ctrl-c> to exit
> ..//bin/fsstat: $Data not found while loading the
> MFT
> ..//bin/fsstat: Invalid FAT32 image (numroot != 0)
> ..//bin/fsstat: Invalid FAT32 image (numroot != 0)
> ..//bin/fsstat: $Data not found while loading the
> MFT
> ..//bin/fsstat: Invalid FAT32 image (numroot != 0)
> ..//bin/fsstat: getFAT: return FAT16 cluster too
> large
> 
> 
> fsstat seems to be having trouble with this
> partition, and its unclear
> to me what kind of filesystem is *really* there.
> 
> fdisk told me it was NTFS/HPFS
> mount told me it was vfat
> autopsy thinks its FAT16, would *not* accept choice
> of NTFS or FAT32
> 
> it appears that this is due to the root directory,
> which appears to
> have been overwritten by the MS tools.  Ive
> inspected other
> parts of the 60 GB, they appear to have the original
> content.
> 
> Qs:
> 
> is it common on MS OSs that partition type and fs
> type are different ?
> 
> is the choice of filesystem completely dictated by
> the root-directory ?
> is the choice overridable ?
> is there a reason its not automatic - since it seems
> to reject the 
> 'wrong' choices ?
> 
> is it possible to not care ?
> in my case, I hope to find that the vast majority of
> directories
> look to be uncorrupted - ie the directory entries
> are legal,
> and point to files that match whatever metadata is
> stored in the 
> directory entry itself,
> and link to each other properly.
> IOW, if 59.9 GB of data look consistent with each
> other,
> then clearly the root folder is hosed, and is only
> thing to fix.
> 
> 
> wrt:
> Calculating MD5 of images/whole.img (this could take
> a while)
>
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+-
> 
> it would be nice to have some estimate of ' a
> while',
> ie how big the ticks are,
> 
> 
> 
> my other attempts to extract some of the original
> filesystem
> have also run afoul on this FAT16 choice:
> 
> ..//bin/ils: getFAT: return FAT16 cluster too large
> ..//bin/dls: getFAT: return FAT16 cluster too large
> 
> 
> Im gonna continue poking, 1st thru autopsy, then the
> sleuthkit tools,
> but Im also contemplating trying to remove/zero out
> the root directory
> so that something else in the partition has a chance
> to influence the mount.
> Please advise me on this course of action b4 I do
> something (else) stupid
> 
> tia
> jimc
> 
> 
> 
>
-------------------------------------------------------
> The SF.Net email is sponsored by: Beat the
> post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> sleuthkit-users mailing list
>
https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> http://www.sleuthkit.org
> 

Re: [sleuthkit-users] filesystem recovery using sleuthkit

[Jim C. <jc...@di...>]

Rich Thompson wrote:

>You may have already tried, but I'll say it anyway -
>Did you try booting with an XP boot disk and doing an
>fdisk mbr???  That should return the mbr for windows
>allowing you to boot....
>
>Rich
>
>  
>
the XP recovery disk that shipped with the Dell
did not include fdisk IIRC, so I used what it did have;
fixmbr & fixboot.

These 2 tools each fixed something; they changed the error
message, but at the end of the boot attempt, it found
an empty FAT12 fs where the partition table told it to look.

At any rate, I re-installed XP, and have a 60 GB image
to play around with, and see if I can slice off that FAT12 slice,
or find an NTFS superblock later on the disk and copy it
onto the FAT12 sector.

If you know of any free (as beer) NTFS repair tools,
Im all ears.

thanks
jimc

>--- Jim Cromie <jc...@di...> wrote:
>
>  
>
>>Im trying to recover contents of a 60GB Windows XP
>>installation
>>after a grub accident.
>>
>>A short 'how I got here' follows, then the
>>Questions.
>>
>>I recently screwed up (technical jargon) my dads
>>winXP box
>>while trying to put grub on a usb-drive.
>>(grub went onto /dev/hda, but linux was on /dev/sda,
>>so it wouldnt boot anything)
>>
>>I recovered using the recovery disk, tried the
>>repair option:
>>fixmbr - to restore the MBR, and fixboot (I think)
>>
>>These tools apparently trashed the root of C:\,
>>they showed me a mostly empty root-dir, with 4
>>(empty)
>>files with names that were evidently unreadable by
>>the OS
>>(they were represnted by ?, all 4 of them)
>>
>>with a knoppix disk and a 3 line perl prog,
>>I took 1GB at a time off it, and NFS'd it to my
>>laptop,
>>then onto the USB drive  (knoppix dd wouldnt handle
>>    
>>
>>>2GB files)
>>>      
>>>
>>(knoppix 3.6 wouldnt mount the usb directly - some
>>versioning
>>prob w the usb-* modules)
>>
>>I catted all the 1gb chunks together, and have
>>started autopsy-2.03
>>and sleuthkit-1.73 on it.
>>
>>Evidence Locker: /mnt/u2/forensics
>>Start Time: Sun Jan  9 09:32:31 2005
>>Remote Host: localhost
>>Local Port: 9999
>>
>>Open an HTML browser on the remote host and paste
>>this URL in it:
>>
>>    http://localhost:9999/autopsy
>>
>>Keep this process running and use <ctrl-c> to exit
>>..//bin/fsstat: $Data not found while loading the
>>MFT
>>..//bin/fsstat: Invalid FAT32 image (numroot != 0)
>>..//bin/fsstat: Invalid FAT32 image (numroot != 0)
>>..//bin/fsstat: $Data not found while loading the
>>MFT
>>..//bin/fsstat: Invalid FAT32 image (numroot != 0)
>>..//bin/fsstat: getFAT: return FAT16 cluster too
>>large
>>
>>
>>fsstat seems to be having trouble with this
>>partition, and its unclear
>>to me what kind of filesystem is *really* there.
>>
>>fdisk told me it was NTFS/HPFS
>>mount told me it was vfat
>>autopsy thinks its FAT16, would *not* accept choice
>>of NTFS or FAT32
>>
>>it appears that this is due to the root directory,
>>which appears to
>>have been overwritten by the MS tools.  Ive
>>inspected other
>>parts of the 60 GB, they appear to have the original
>>content.
>>
>>Qs:
>>
>>is it common on MS OSs that partition type and fs
>>type are different ?
>>
>>is the choice of filesystem completely dictated by
>>the root-directory ?
>>is the choice overridable ?
>>is there a reason its not automatic - since it seems
>>to reject the 
>>'wrong' choices ?
>>
>>is it possible to not care ?
>>in my case, I hope to find that the vast majority of
>>directories
>>look to be uncorrupted - ie the directory entries
>>are legal,
>>and point to files that match whatever metadata is
>>stored in the 
>>directory entry itself,
>>and link to each other properly.
>>IOW, if 59.9 GB of data look consistent with each
>>other,
>>then clearly the root folder is hosed, and is only
>>thing to fix.
>>
>>
>>wrt:
>>Calculating MD5 of images/whole.img (this could take
>>a while)
>>
>>    
>>
>+----+----+----+----+----+----+----+----+----+----+----+----+----+----+-
>  
>
>>it would be nice to have some estimate of ' a
>>while',
>>ie how big the ticks are,
>>
>>
>>
>>my other attempts to extract some of the original
>>filesystem
>>have also run afoul on this FAT16 choice:
>>
>>..//bin/ils: getFAT: return FAT16 cluster too large
>>..//bin/dls: getFAT: return FAT16 cluster too large
>>
>>
>>Im gonna continue poking, 1st thru autopsy, then the
>>sleuthkit tools,
>>but Im also contemplating trying to remove/zero out
>>the root directory
>>so that something else in the partition has a chance
>>to influence the mount.
>>Please advise me on this course of action b4 I do
>>something (else) stupid
>>
>>tia
>>jimc
>>
>>
>>
>>
>>    
>>
>
>  
>