sleuthkit-users

[sleuthkit-users] Autopsy: Can't search in unallocated space of a partition

[Dennis <in...@ba...>]

Dear all, 

I am currently giving autopsy a test ride on one of my test images. I
use this test image in some of my forensic classes but I ran into a
problem.

My Setup 
   Windows 8 64 Bit 
   Autopsy V 3.0.6 

Image Details: t
   320 GB EWF Image

Case Setup / Activated Ingest Modules
   Recent Activities
   Hash Lookup
   EXIF Image Parser 
   Keyword Search 

And of course the checkbox for "process unallocated space" was
activated. 
 	
My Scenario
I know that a HTML fragment is available in the unallocated space of one
partition. This HTML fragment contains the string "secret secret".
Therefore, I just ran a search for the string secret but the search did
not yield any results in the unallocated space. 

I double checked that the string was inside the unallocated space by
mounting the image via fuse (DFF) and running the command 
	string -f -t d * | grep secret 
inside the NTFS unallocated folder. This resulted in roughly 20 - 30
hits. 

Question
Is this a known bug? Is the search in the unallocated space not yet
supported? How can I investigate what is going wrong?

Kind regards
Dennis  

Re: [sleuthkit-users] Autopsy: Can't search in unallocated space of a partition

[Dennis <in...@ba...>]

Hi, 

the image was created with FTK Imager (3.1). I did not activate
compression for the E01 image. 

Kind regards
Dennis 

Am Donnerstag, den 14.11.2013, 10:00 +0800 schrieb Notyor Buizines:
> what command did u use for taking image of hard disk?
> 
> 
> 
> On Thu, Nov 7, 2013 at 4:36 AM, Dennis <in...@ba...> wrote:
>         Dear all,
>         
>         I am currently giving autopsy a test ride on one of my test
>         images. I
>         use this test image in some of my forensic classes but I ran
>         into a
>         problem.
>         
>         My Setup
>            Windows 8 64 Bit
>            Autopsy V 3.0.6
>         
>         Image Details: t
>            320 GB EWF Image
>         
>         Case Setup / Activated Ingest Modules
>            Recent Activities
>            Hash Lookup
>            EXIF Image Parser
>            Keyword Search
>         
>         And of course the checkbox for "process unallocated space" was
>         activated.
>         
>         My Scenario
>         I know that a HTML fragment is available in the unallocated
>         space of one
>         partition. This HTML fragment contains the string "secret
>         secret".
>         Therefore, I just ran a search for the string secret but the
>         search did
>         not yield any results in the unallocated space.
>         
>         I double checked that the string was inside the unallocated
>         space by
>         mounting the image via fuse (DFF) and running the command
>                 string -f -t d * | grep secret
>         inside the NTFS unallocated folder. This resulted in roughly
>         20 - 30
>         hits.
>         
>         Question
>         Is this a known bug? Is the search in the unallocated space
>         not yet
>         supported? How can I investigate what is going wrong?
>         
>         Kind regards
>         Dennis
>         
>         
>         ------------------------------------------------------------------------------
>         November Webinars for C, C++, Fortran Developers
>         Accelerate application performance with scalable programming
>         models. Explore
>         techniques for threading, error checking, porting, and tuning.
>         Get the most
>         from the latest Intel processors and coprocessors. See
>         abstracts and register
>         http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
>         _______________________________________________
>         sleuthkit-users mailing list
>         https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
>         http://www.sleuthkit.org
> 
> 

Re: [sleuthkit-users] Autopsy: Can't search in unallocated space of a partition

[Dennis <in...@ba...>]

Simson, 

thanks for the reply. 

regarding 1) I know that the string is in the unallocated space of the
partition due to an EnCASE analysis of the image and I have double
checked this with the method outlined in my initial email using fuse,
string and grep command. 

regarding 2) I already had a look at bulk_extractor but I thought that
Autopsy should have found the string as well that was the reason for
starting this thread to investigate. 
Some thoughts - better to say wild guesses -  that I have: 
- character encoding problem (ANSI vs UTF8 vs UTF16 with its different
endianness)
- not fully index the unallocated space. 

Kind regards
Dennis 


Kind regards
Dennis 

Am Sonntag, den 17.11.2013, 06:54 -0500 schrieb Simson Garfinkel:
> Dennis —
> 
> 1. Perhaps the string is not in unallocated space.
> 
> 2. For your application below, bulk_extractor with ‘-f’ might give better results. I realize that your goal is to test Autopsy, but I’m not really sure why you want to do that.  bulk_extractor with identify_filenames.py will tell you which files the strings came from.
> 
> Simson
> 
> 
> On Nov 17, 2013, at 5:43 AM, Dennis <in...@ba...> wrote:
> 
> > Hi, 
> > 
> > the image was created with FTK Imager (3.1). I did not activate
> > compression for the E01 image. 
> > 
> > Kind regards
> > Dennis 
> > 
> > Am Donnerstag, den 14.11.2013, 10:00 +0800 schrieb Notyor Buizines:
> >> what command did u use for taking image of hard disk?
> >> 
> >> 
> >> 
> >> On Thu, Nov 7, 2013 at 4:36 AM, Dennis <in...@ba...> wrote:
> >>        Dear all,
> >> 
> >>        I am currently giving autopsy a test ride on one of my test
> >>        images. I
> >>        use this test image in some of my forensic classes but I ran
> >>        into a
> >>        problem.
> >> 
> >>        My Setup
> >>           Windows 8 64 Bit
> >>           Autopsy V 3.0.6
> >> 
> >>        Image Details: t
> >>           320 GB EWF Image
> >> 
> >>        Case Setup / Activated Ingest Modules
> >>           Recent Activities
> >>           Hash Lookup
> >>           EXIF Image Parser
> >>           Keyword Search
> >> 
> >>        And of course the checkbox for "process unallocated space" was
> >>        activated.
> >> 
> >>        My Scenario
> >>        I know that a HTML fragment is available in the unallocated
> >>        space of one
> >>        partition. This HTML fragment contains the string "secret
> >>        secret".
> >>        Therefore, I just ran a search for the string secret but the
> >>        search did
> >>        not yield any results in the unallocated space.
> >> 
> >>        I double checked that the string was inside the unallocated
> >>        space by
> >>        mounting the image via fuse (DFF) and running the command
> >>                string -f -t d * | grep secret
> >>        inside the NTFS unallocated folder. This resulted in roughly
> >>        20 - 30
> >>        hits.
> >> 
> >>        Question
> >>        Is this a known bug? Is the search in the unallocated space
> >>        not yet
> >>        supported? How can I investigate what is going wrong?
> >> 
> >>        Kind regards
> >>        Dennis
> >> 
> >> 
> >>        ------------------------------------------------------------------------------
> >>        November Webinars for C, C++, Fortran Developers
> >>        Accelerate application performance with scalable programming
> >>        models. Explore
> >>        techniques for threading, error checking, porting, and tuning.
> >>        Get the most
> >>        from the latest Intel processors and coprocessors. See
> >>        abstracts and register
> >>        http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> >>        _______________________________________________
> >>        sleuthkit-users mailing list
> >>        https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> >>        http://www.sleuthkit.org
> >> 
> >> 
> > 
> > 
> > 
> > ------------------------------------------------------------------------------
> > DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
> > OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
> > Free app hosting. Or install the open source package on any LAMP server.
> > Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
> > http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
> > _______________________________________________
> > sleuthkit-users mailing list
> > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> > http://www.sleuthkit.org
> 

Re: [sleuthkit-users] Autopsy: Can't search in unallocated space of a partition

[Brian C. <ca...@sl...>]

Hi Dennis,

Sorry for the late reply on this. I'm finally getting back to all of the e-mails that occurred during OSDFCon the prep time before it. 

It should have found the strings and I just verified it on a test image.  A couple of things to mention here:
- Autopsy does case insensitive, exact matches.  Meaning that if you search for "forensic", then it will not find "forensics". It will find "FORENSIC" though. We are going to change the behavior in the future to make these substring matches easier.  Currently, you need to make a regular expression search and do something like ".*forensic.*".  The ".*" are wild cards before and after the word. Not sure if that is related to what you are seeing or not.
- One thing to help debug is if the text was properly extracted.  If you know where the string is in unallocated space, then you can look at the virtual unallocated file that Autopsy/TSK created for that region of unallocated space.  The virtual files are located in the "$Unalloc" folder for each file system and have names of the following syntax:

Unalloc_InternalID_StartByteOffset_EndByteOffset

If you know the byte offset of the string relative to the start of the disk, then find the Unalloc file that it is located in and view its contents.  The "Strings" tab shows the output of running strings on the content and the "Text" tab shows you want is in the SOLR keyword index.

If none of these helped, let me know and we can proceed with more steps.

thanks,
brian


On Nov 6, 2013, at 3:36 PM, Dennis <in...@ba...> wrote:

> Dear all, 
> 
> I am currently giving autopsy a test ride on one of my test images. I
> use this test image in some of my forensic classes but I ran into a
> problem.
> 
> My Setup 
>   Windows 8 64 Bit 
>   Autopsy V 3.0.6 
> 
> Image Details: t
>   320 GB EWF Image
> 
> Case Setup / Activated Ingest Modules
>   Recent Activities
>   Hash Lookup
>   EXIF Image Parser 
>   Keyword Search 
> 
> And of course the checkbox for "process unallocated space" was
> activated. 
> 	
> My Scenario
> I know that a HTML fragment is available in the unallocated space of one
> partition. This HTML fragment contains the string "secret secret".
> Therefore, I just ran a search for the string secret but the search did
> not yield any results in the unallocated space. 
> 
> I double checked that the string was inside the unallocated space by
> mounting the image via fuse (DFF) and running the command 
> 	string -f -t d * | grep secret 
> inside the NTFS unallocated folder. This resulted in roughly 20 - 30
> hits. 
> 
> Question
> Is this a known bug? Is the search in the unallocated space not yet
> supported? How can I investigate what is going wrong?
> 
> Kind regards
> Dennis  
> 
> 
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models. Explore
> techniques for threading, error checking, porting, and tuning. Get the most 
> from the latest Intel processors and coprocessors. See abstracts and register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> _______________________________________________
> sleuthkit-users mailing list
> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> http://www.sleuthkit.org