Re: [sleuthkit-users] Problems Using fiwalk
Brought to you by:
carrier
Menu
▾
▴
Re: [sleuthkit-users] Problems Using fiwalk
|
From: Karl B. <kar...@gm...> - 2011-10-20 15:49:34
|
Simson, Sorry for the delay - things have been busy. I wasn't able to get the script to run, so did some other work to show what little I found was inconsequential.. But back to fiwalk and identify_filenames.py - looks like the version of identify_filenames.py I'm, running is from 10/6/2011: drwxr-xr-x 2 501 staff 360 2011-10-17 17:28 . drwxr-xr-x 9 501 staff 740 2011-10-17 17:26 .. -rw-r--r-- 1 501 staff 4487 2011-08-06 13:21 bulk_diff.py -rw-r--r-- 1 501 staff 3774 2011-09-30 00:26 bulk_extractor.py -rw-r--r-- 1 root root 4737 2011-10-17 17:28 bulk_extractor.pyc -rw-r--r-- 1 501 staff 3599 2011-03-10 00:53 cda2.py -rw-r--r-- 1 501 staff 850 2011-03-10 00:53 cda_test.py -rw-r--r-- 1 501 staff 8874 2011-03-10 00:53 cda_tool.py -rw-r--r-- 1 501 staff 48496 2011-09-22 02:21 dfxml.py -rw-r--r-- 1 root root 58373 2011-10-17 17:28 dfxml.pyc -rw-r--r-- 1 501 staff 4508 2011-08-20 18:44 fiwalk.py -rw-r--r-- 1 root root 6156 2011-10-17 17:28 fiwalk.pyc -rw-r--r-- 1 501 staff 10839 2011-10-06 01:10 identify_filenames.py -rw-r--r-- 1 501 staff 2697 2011-03-10 00:53 make_context_stop_list.py -rw-rw-rw- 1 root root 10437 2011-10-17 17:26 Makefile -rw-r--r-- 1 501 staff 161 2011-05-27 20:27 Makefile.am -rw-r--r-- 1 501 staff 9384 2011-10-07 19:34 Makefile.in -rw-r--r-- 1 501 staff 13575 2011-08-27 11:55 ttable.py As I have time, I want to see if I can get this to work, because I can definitely see the value in it and that it will be very helpful in future investigations. Thanks, Karl On Tue, Oct 18, 2011 at 1:10 PM, Simson Garfinkel <si...@ac...> wrote: > Hi, Karl. What version of identify_filenames.py are you using? Here is what > is on my development machine; I'll be pushing out a new version of > bulk_extractor in a few days.... > > 14:09 alphonse-mucha:~/domex/src/bulk_extractor/python$ ls -l *.py > -rw-r--r-- 1 simsong staff 4487 Aug 6 09:21 bulk_diff.py > -rw-r--r-- 1 simsong staff 3774 Sep 29 20:26 bulk_extractor.py > -rw-r--r-- 1 simsong staff 3599 Mar 9 2011 cda2.py > -rw-r--r-- 1 simsong staff 850 Mar 9 2011 cda_test.py > -rw-r--r-- 1 simsong staff 8874 Mar 9 2011 cda_tool.py > lrwxr-xr-x 1 simsong staff 28 Aug 4 09:01 dfxml.py@ -> > ../../fiwalk/python/dfxml.py > lrwxr-xr-x 1 simsong staff 29 Aug 4 09:01 fiwalk.py@ -> > ../../fiwalk/python/fiwalk.py > -rw-r--r-- 1 simsong staff 10968 Oct 12 06:38 identify_filenames.py > -rw-r--r-- 1 simsong staff 2697 Mar 9 2011 make_context_stop_list.py > -rw-r--r-- 1 simsong staff 2136 Oct 16 22:37 post_process_exif.py > lrwxr-xr-x 1 simsong staff 19 Aug 4 09:01 ttable.py@ -> > ../../lib/ttable.py > 14:09 alphonse-mucha:~/domex/src/bulk_extractor/python$ > > 14:09 alphonse-mucha:~/domex/src/bulk_extractor/python$ ls -l > ../../fiwalk/*.py ../../lib/*.py > -rw-r--r--@ 1 simsong staff 236 Sep 20 2008 > ../../fiwalk/checkversion.py > -rw-r--r--@ 1 simsong staff 9752 Nov 13 2009 ../../fiwalk/load_sql.py > -rw-r--r--@ 1 simsong staff 77863 Mar 28 2010 > ../../lib/BeautifulSoup.py > -rw-r--r-- 1 simsong staff 17968 Jan 4 2011 ../../lib/arff.py > -rw-r--r--@ 1 simsong staff 236 May 26 2009 > ../../lib/checkversion.py > -rw-r--r--@ 1 simsong staff 1048 Jan 19 2010 ../../lib/confirm.py > -rw-r--r-- 1 simsong staff 7589 Aug 12 18:49 ../../lib/domex.py > -rw-r--r--@ 1 simsong staff 1133 May 14 2008 ../../lib/hex.py > -rw-r--r-- 1 simsong staff 991 Aug 3 22:19 ../../lib/hexdump.py > -rw-r--r-- 1 simsong staff 7704 Jan 25 2011 ../../lib/histogram.py > -rw-r--r-- 1 simsong staff 1396 Oct 11 17:55 ../../lib/histogram2d.py > -rw-r--r--@ 1 simsong staff 7275 Dec 4 2009 ../../lib/logistic.py > -rwxr-xr-x@ 1 simsong staff 38401 Dec 4 2009 ../../lib/pstat.py* > -rw-r--r--@ 1 simsong staff 2174 Dec 2 2009 ../../lib/pylatex.py > -rw-r--r--@ 1 simsong staff 6278 Oct 10 23:55 ../../lib/statbag.py > -rwxr-xr-x@ 1 simsong staff 151400 Dec 4 2009 ../../lib/stats.py* > -rw-r--r--@ 1 simsong staff 147 May 26 2009 ../../lib/timer.py > -rw-r--r--@ 1 simsong staff 1575 Mar 17 2010 ../../lib/treegram.py > -rw-r--r--@ 1 simsong staff 13818 Oct 12 06:39 ../../lib/ttable.py > 14:10 alphonse-mucha:~/domex/src/bulk_extractor/python$ > > > > On Oct 18, 2011, at 11:23 AM, Karl Bernard wrote: > > After compiling, fiwalk ran without incident and created the XML file. When > I try to run identify_filenames.py I get the following errors. I thought > perhaps I'm getting the order of the inputs wrong, but when I tried them in > different order, I see a similar error. I also downloaded the latest > bulk_extractor and did configure, make, and make install. > > What are the proper inputs and switches for identify_filenames.py? > > One more question - does bulk_extractor only do allocated blocks, or > everything? I'm unable to map my hit back to an inode using tsk tools > (ifind, etc). This may be reasonable if the space is unallocated, but seems > problematic if it is allocated. Of course, I could be doing all my > calculations wrong.... > > FWIW - this is an HFS+ filesystem... > > Python3: > trying *identify_filenames.py fiwalk.xml featurefile outputfile* > root@sansforensics:/# python3 > /home/sansforensics/Downloads/bulk_extractor-1.1.0_alpha2/python/identify_filenames.py > /media/sdb1/2011image_fiwalk /media/6241-4B1A/2011image.dd/ccn.txt > /media/sdb1/2011image_out > File > "/home/sansforensics/Downloads/bulk_extractor-1.1.0_alpha2/python/identify_filenames.py", > line 13 > raise ImportError, 'This script requires the dfxml and fiwalk modules > for Python.' > ^ > SyntaxError: invalid syntax > > > Python2.6: > trying *identify_filenames.py fiwalk.xml featurefile outputfile* > root@sansforensics:/# python2.6 > /home/sansforensics/Downloads/bulk_extractor-1.1.0_alpha2/python/identify_filenames.py /media/sdb1/2011image_fiwalk /media/6241-4B1A/2011image.dd/ccn.txt > /media/sdb1/2011image_out > Traceback (most recent call last): > File > "/home/sansforensics/Downloads/bulk_extractor-1.1.0_alpha2/python/identify_filenames.py", > line 19, in <module> > which is in Python 2.7 or Python 3.0""" > ImportError: This script requires ArgumentParser > which is in Python 2.7 or Python 3.0 > > Python: > trying *identify_filenames.py fiwalk.xml featurefile* > root@sansforensics:/# python > ./identify_filenames.py /media/sdb1/2011image_fiwalk > /media/6241-4B1A/2011image.dd/ccn.txt > Traceback (most recent call last): > File "./identify_filenames.py", line 13, in <module> > raise ImportError, 'This script requires the dfxml and fiwalk modules > for Python.' > ImportError: This script requires the dfxml and fiwalk modules for Python. > > Any suggestions or comments are welcome. > > Thanks, > > Karl > > On Mon, Oct 17, 2011 at 10:56 AM, Simson Garfinkel <si...@ac...>wrote: > >> A progress report would require a two-pass scan. >> >> On Oct 17, 2011, at 11:49 AM, Karl Bernard wrote: >> >> FYI - fiwalk 0.6.15 compiled in the SIFT kit without incident and is >> running error-free at the moment. I'll let you know how it looks when it >> completes. I miss bulk_extractor's progress report ;) >> >> Karl >> >> On Fri, Oct 14, 2011 at 10:34 PM, Karl Bernard <kar...@gm...>wrote: >> >>> Simson, >>> >>> Thanks for the reply. I can't share the image file, it's part of an >>> ongoing investigation, but I'll try building fiwalk from source on Monday. >>> I'll let you know how it goes. >>> >>> Thanks again, >>> >>> Karl >>> On Oct 14, 2011 7:48 PM, "Simson Garfinkel" <si...@ac...> wrote: >>> >>>> Hi, Karl. I cannot speak for the version of fiwalk included on the SANS >>>> SIFT. However, if you wish to download the fiwalk from the afflib.orgwebsite and build it from sources, I'm happy to provide you with support. >>>> >>>> Is your file *HDDimage_ntfs_fiwalk available for testing?* >>>> >>>> identify_filenames requires the DFXML file. You can't do it with TSK >>>> tools alone; we will be integrating fiwalk into the SleuthKit distribution >>>> later this year. >>>> >>>> >>>> On Oct 14, 2011, at 2:46 PM, Karl Bernard wrote: >>>> >>>> I'm trying to use fiwalk from within the SANS SIFT kit 2.1 to process >>>> some images and create a "digital forensics XML file" and it's failing out >>>> with some sort of programmatic error: >>>> >>>> *root@sansforensics:~# fiwalk -I -X /media/sdb1/HDDimage_ntfs_fiwalk >>>> /media/sdb1/HDDimage.dd* >>>> *terminate called after throwing an instance of 'std::logic_error'* >>>> * what(): basic_string::_S_construct NULL not valid* >>>> *Aborted (core dumped)* >>>> >>>> >>>> FWIW - from the beginning of the XML file that it started to make, it >>>> looks like it's trying to say all ftype's are 0 - like >>>> this: <ftype>0</ftype> >>>> >>>> Any thoughts about what to try next? Anyone else successfully run fiwalk >>>> in the SIFT Kit? In the meantime, I'm going to break out my SANS 508 books >>>> and review the steps to resolve this directly in TSK ;) >>>> >>>> *Background:* >>>> Using AFFLIB's bulk_extractor (many thanks to Derrick Karpo and the many >>>> others that gave great suggestions) I've had great luck finding some hits >>>> for possible CC and SSN info on some imaged drives and now want to map the >>>> offsets back to file names/locations. >>>> >>>> Ran this on all images: >>>> *bulk_extractor -o /media/6241-4B1A/bulk_out1 -E accts >>>> /media/sdb1/HDDimage.dd* >>>> >>>> Found some hits like this (this one looks like part of a program): >>>> >>>> 79168205446 SSN: 123456789 >>>> NE_[vsCkn?n][RU]SSN: 123456789_[vsckN?N][RU]Su >>>> >>>> And now I want to see what file resides at offset 79168205446. >>>> >>>> Planned to use this (from: http://afflib.org/software/bulk_extractor): >>>> *identify_filenames.py* In the *bulk_extractor* feature file, each >>>> feature is annotated with the byte offset from the beginning of the image in >>>> which it was found. The program takes as input a *bulk_extractor* feature >>>> file and a DFXML file containing the locations of each file on the drive (produced >>>> with Garfinkel’s fiwalk program) and produces an annotated feature file >>>> that contains the offset, feature, and the file in which the feature was >>>> found. >>>> >>>> I know this can be done with TSK tools - but was hoping to use fiwalk >>>> instead... It looks like it could be a really great way to process these >>>> kinds of cases with a lot fewer headaches. >>>> >>>> References: >>>> http://afflib.org/software/fiwalk >>>> >>>> Any thoughts/suggestions are welcome, >>>> >>>> Karl Bernard >>>> ------------------------------------------------------------------------------ >>>> All the data continuously generated in your IT infrastructure contains a >>>> definitive record of customers, application performance, security >>>> threats, fraudulent activity and more. Splunk takes this data and makes >>>> sense of it. Business sense. IT sense. Common sense. >>>> >>>> http://p.sf.net/sfu/splunk-d2d-oct_______________________________________________ >>>> sleuthkit-users mailing list >>>> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users >>>> http://www.sleuthkit.org >>>> >>>> >>>> >> >> > > |
