Re: [sleuthkit-users] sleuthkit-users Digest, Vol 105, Issue 17

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi Alan.

There's no active development happening on fiwalk. On the other hand, there's no active development happening on many other SleuthKit tools. The issue is that fiwalk replicates tools that are being actively developed, like tsk_loaddb. In many cases it is faster to store things in the SQLite3 database than to generate and consume DFXML.

You are correct that DFXML is more applicable to mapping inside of files than the SQLite3 schema is. However, there are other XML standards as well, and my goal is to merge DFXML with one of those standards, rather than maintain a complete separate set of definitions.

> On Mar 27, 2015, at 5:59 PM, Alan Browne <ala...@gm...> wrote:
> 
> Simson
> Is the development of fiwalk still being continued. I use fiwalk to map a drive rather than tsk as I find it produces more info. If it is being further developed, would it be possible to map inside archived files as well
> Regards
> Alan
> 
> On 27 Mar 2015 21:33, <sle...@li... <mailto:sle...@li...>> wrote:
> Send sleuthkit-users mailing list submissions to
>         sle...@li... <mailto:sle...@li...>
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>
> or, via email, send a message with subject or body 'help' to
>         sle...@li... <mailto:sle...@li...>
> 
> You can reach the person managing the list at
>         sle...@li... <mailto:sle...@li...>
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of sleuthkit-users digest..."
> 
> 
> Today's Topics:
> 
>    1. Re: Attempting to use fiwalk (Simson Garfinkel)
>    2. Re: Attempting to use fiwalk (Jeff Scarborough)
>    3. Re: Attempting to use fiwalk (Simson Garfinkel)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Fri, 27 Mar 2015 14:58:30 -0400
> From: Simson Garfinkel <si...@ac... <mailto:si...@ac...>>
> Subject: Re: [sleuthkit-users] Attempting to use fiwalk
> To: Brian Carrier <ca...@sl... <mailto:ca...@sl...>>
> Cc: sle...@li... <mailto:sle...@li...>
> Message-ID: <E4E...@ac... <mailto:E4E...@ac...>>
> Content-Type: text/plain; charset=us-ascii
> 
> With the fiwalk rewrite, it's using standard Sleuthkit image processing.
> 
> However, Jeff, what are you using fiwalk for? What's your interest in DFXML?
> 
> Simson
> 
> 
> > On Mar 27, 2015, at 2:07 PM, Brian Carrier <ca...@sl... <mailto:ca...@sl...>> wrote:
> >
> > TSK commands should find the remaining files if you give it just the ".001" file.   Not sure about fiwalk's usage.
> >
> > Jeff, if you run tsk_gettimes on the image, then does it find all of them?
> >
> >
> >
> > On Mar 27, 2015, at 1:27 PM, Jeff Scarborough <jef...@gm... <mailto:jef...@gm...>> wrote:
> >
> >> I am a new user to SleuthKit and I am attempting to run fiwalk on an image and output a dfxml file.  The image is, I believe called a split raw since it is in the form of filename.001, filename.002, filename.003 etc.  I am having an issue with the command line to output the file.
> >>
> >> The below command is the example i usually run across.
> >>
> >> fiwalk -X path/report.xml path/image.raw
> >>
> >>
> >> I need to use fiwalk with split files.  I used the examples below with limited luck.
> >>
> >> fiwalk -X path/report.xml path/image.dd  -- this one said it had trouble opening the file
> >>
> >> fiwalk -X path/report.xml path/image.*  -- this one also has trouble
> >>
> >>
> >> The command line below seems to start the process but as far as I can see only processes the first file in the list and none of the others.
> >>
> >> fiwalk -X path/report.xml path/image.001
> >>
> >>
> >> Am I missing something in the command line that will process all of the files?
> >>
> >> I am using a virtual machine to run linux with SleuthKit installed and the image is on a USB drive.
> >>
> >> Thanks,
> >> Jeff Scarborough
> >> ------------------------------------------------------------------------------
> >> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> >> by Intel and developed in partnership with Slashdot Media, is your hub for all
> >> things parallel software development, from weekly thought leadership blogs to
> >> news, videos, case studies, tutorials and more. Take a look and join the
> >> conversation now. http://goparallel.sourceforge.net/_______________________________________________ <http://goparallel.sourceforge.net/_______________________________________________>
> >> sleuthkit-users mailing list
> >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>
> >> http://www.sleuthkit.org <http://www.sleuthkit.org/>
> >
> >
> > ------------------------------------------------------------------------------
> > Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> > by Intel and developed in partnership with Slashdot Media, is your hub for all
> > things parallel software development, from weekly thought leadership blogs to
> > news, videos, case studies, tutorials and more. Take a look and join the
> > conversation now. http://goparallel.sourceforge.net/ <http://goparallel.sourceforge.net/>
> > _______________________________________________
> > sleuthkit-users mailing list
> > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>
> > http://www.sleuthkit.org <http://www.sleuthkit.org/>
> 
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Fri, 27 Mar 2015 15:01:37 -0500
> From: Jeff Scarborough <jef...@gm... <mailto:jef...@gm...>>
> Subject: Re: [sleuthkit-users] Attempting to use fiwalk
> To: Simson Garfinkel <si...@ac... <mailto:si...@ac...>>,
>         sle...@li... <mailto:sle...@li...>
> Cc: Brian Carrier <ca...@sl... <mailto:ca...@sl...>>
> Message-ID:
>         <CAG...@ma... <mailto:vGg...@ma...>>
> Content-Type: text/plain; charset="utf-8"
> 
> Thank you all for your reply.
> 
> @Alex -- I believe you are correct in that fiwalk wants one file.
> Fortunately, Jason Wright had a workable idea for that.
> 
> @Brian Carrier -- Using tsk_gettimes on the image does seem to run through
> the data.  The process ran for several minutes before I stopped the
> program.  It seem the data would be more than what would be found in a
> single file.
> 
> @Simson Garfinkel -- I have a few drive images that I am attempting to
> extract data using Bulk Extractor.  According to a presentation you had
> given on Bulk Extractor, I am using fiwalk to extract DFXML data and will
> then run identify_filesnames.py in hopes of linking the data with the files.
> 
> @Jason Wright -- Thanks.  Using the affuse worked, once I had the commands
> down correctly.  Below are the commands I used for reference.
> 
> affuse path/to/image.001 /mnt/combine
> fiwalk -X report.xml /mnt/combine/image.001.raw
> 
> Thanks again,
> 
> Jeff Scarborough
> 
> On Fri, Mar 27, 2015 at 1:58 PM, Simson Garfinkel <si...@ac... <mailto:si...@ac...>> wrote:
> 
> > With the fiwalk rewrite, it's using standard Sleuthkit image processing.
> >
> > However, Jeff, what are you using fiwalk for? What's your interest in
> > DFXML?
> >
> > Simson
> >
> >
> > > On Mar 27, 2015, at 2:07 PM, Brian Carrier <ca...@sl... <mailto:ca...@sl...>>
> > wrote:
> > >
> > > TSK commands should find the remaining files if you give it just the
> > ".001" file.   Not sure about fiwalk's usage.
> > >
> > > Jeff, if you run tsk_gettimes on the image, then does it find all of
> > them?
> > >
> > >
> > >
> > > On Mar 27, 2015, at 1:27 PM, Jeff Scarborough <
> > jef...@gm... <mailto:jef...@gm...>> wrote:
> > >
> > >> I am a new user to SleuthKit and I am attempting to run fiwalk on an
> > image and output a dfxml file.  The image is, I believe called a split raw
> > since it is in the form of filename.001, filename.002, filename.003 etc.  I
> > am having an issue with the command line to output the file.
> > >>
> > >> The below command is the example i usually run across.
> > >>
> > >> fiwalk -X path/report.xml path/image.raw
> > >>
> > >>
> > >> I need to use fiwalk with split files.  I used the examples below with
> > limited luck.
> > >>
> > >> fiwalk -X path/report.xml path/image.dd  -- this one said it had
> > trouble opening the file
> > >>
> > >> fiwalk -X path/report.xml path/image.*  -- this one also has trouble
> > >>
> > >>
> > >> The command line below seems to start the process but as far as I can
> > see only processes the first file in the list and none of the others.
> > >>
> > >> fiwalk -X path/report.xml path/image.001
> > >>
> > >>
> > >> Am I missing something in the command line that will process all of the
> > files?
> > >>
> > >> I am using a virtual machine to run linux with SleuthKit installed and
> > the image is on a USB drive.
> > >>
> > >> Thanks,
> > >> Jeff Scarborough
> > >>
> > ------------------------------------------------------------------------------
> > >> Dive into the World of Parallel Programming The Go Parallel Website,
> > sponsored
> > >> by Intel and developed in partnership with Slashdot Media, is your hub
> > for all
> > >> things parallel software development, from weekly thought leadership
> > blogs to
> > >> news, videos, case studies, tutorials and more. Take a look and join the
> > >> conversation now.
> > http://goparallel.sourceforge.net/_______________________________________________ <http://goparallel.sourceforge.net/_______________________________________________>
> > >> sleuthkit-users mailing list
> > >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>
> > >> http://www.sleuthkit.org <http://www.sleuthkit.org/>
> > >
> > >
> > >
> > ------------------------------------------------------------------------------
> > > Dive into the World of Parallel Programming The Go Parallel Website,
> > sponsored
> > > by Intel and developed in partnership with Slashdot Media, is your hub
> > for all
> > > things parallel software development, from weekly thought leadership
> > blogs to
> > > news, videos, case studies, tutorials and more. Take a look and join the
> > > conversation now. http://goparallel.sourceforge.net/ <http://goparallel.sourceforge.net/>
> > > _______________________________________________
> > > sleuthkit-users mailing list
> > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>
> > > http://www.sleuthkit.org <http://www.sleuthkit.org/>
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> 
> ------------------------------
> 
> Message: 3
> Date: Fri, 27 Mar 2015 17:32:03 -0400
> From: Simson Garfinkel <si...@ac... <mailto:si...@ac...>>
> Subject: Re: [sleuthkit-users] Attempting to use fiwalk
> To: Jeff Scarborough <jef...@gm... <mailto:jef...@gm...>>
> Cc: Brian Carrier <ca...@sl... <mailto:ca...@sl...>>,
>         "sle...@li... <mailto:sle...@li...> users"
>         <sle...@li... <mailto:sle...@li...>>
> Message-ID: <171...@ac... <mailto:171...@ac...>>
> Content-Type: text/plain; charset="us-ascii"
> 
> It seems that the main reason people are running fiwalk at this point is so that they can run identify_filenames.py with bulk_extractor.  However, fiwalk is a bit of a mess and it doesn't fit in well with the sleuthkit tool suite. tsk_loaddb does a good job with most of what fiwalk does, but it doesn't support plugins and it doesn't export XML.
> 
> Options:
> 
> 1 - Modify tsk_loaddb to output DFXML.
> 2 - Have a Python script that takes a tsk_loaddb Sqlite3 database and outputs DFXML.
> 3 - Modify identify_filenames.py to read the Sqlite3 database produced by tsk_loaddb.
> 
> I think that #2 and #3 are the right options, in that order, provided that nobody is making use of the fiwalk plugins (or provided that they can migrate to something else). Does anyone on the list of a dependency on dfxml or fiwalk plug-ins?
> 
> Simson
> 
> There are two ways to move forward on this.
> 
> > On Mar 27, 2015, at 4:01 PM, Jeff Scarborough <jef...@gm... <mailto:jef...@gm...>> wrote:
> >
> > Thank you all for your reply.
> >
> > @Alex -- I believe you are correct in that fiwalk wants one file.  Fortunately, Jason Wright had a workable idea for that.
> >
> > @Brian Carrier -- Using tsk_gettimes on the image does seem to run through the data.  The process ran for several minutes before I stopped the program.  It seem the data would be more than what would be found in a single file.
> >
> > @Simson Garfinkel -- I have a few drive images that I am attempting to extract data using Bulk Extractor.  According to a presentation you had given on Bulk Extractor, I am using fiwalk to extract DFXML data and will then run identify_filesnames.py in hopes of linking the data with the files.
> >
> > @Jason Wright -- Thanks.  Using the affuse worked, once I had the commands down correctly.  Below are the commands I used for reference.
> >
> > affuse path/to/image.001 /mnt/combine
> > fiwalk -X report.xml /mnt/combine/image.001.raw
> >
> > Thanks again,
> >
> > Jeff Scarborough
> >
> > On Fri, Mar 27, 2015 at 1:58 PM, Simson Garfinkel <si...@ac... <mailto:si...@ac...> <mailto:si...@ac... <mailto:si...@ac...>>> wrote:
> > With the fiwalk rewrite, it's using standard Sleuthkit image processing.
> >
> > However, Jeff, what are you using fiwalk for? What's your interest in DFXML?
> >
> > Simson
> >
> >
> > > On Mar 27, 2015, at 2:07 PM, Brian Carrier <ca...@sl... <mailto:ca...@sl...> <mailto:ca...@sl... <mailto:ca...@sl...>>> wrote:
> > >
> > > TSK commands should find the remaining files if you give it just the ".001" file.   Not sure about fiwalk's usage.
> > >
> > > Jeff, if you run tsk_gettimes on the image, then does it find all of them?
> > >
> > >
> > >
> > > On Mar 27, 2015, at 1:27 PM, Jeff Scarborough <jef...@gm... <mailto:jef...@gm...> <mailto:jef...@gm... <mailto:jef...@gm...>>> wrote:
> > >
> > >> I am a new user to SleuthKit and I am attempting to run fiwalk on an image and output a dfxml file.  The image is, I believe called a split raw since it is in the form of filename.001, filename.002, filename.003 etc.  I am having an issue with the command line to output the file.
> > >>
> > >> The below command is the example i usually run across.
> > >>
> > >> fiwalk -X path/report.xml path/image.raw
> > >>
> > >>
> > >> I need to use fiwalk with split files.  I used the examples below with limited luck.
> > >>
> > >> fiwalk -X path/report.xml path/image.dd  -- this one said it had trouble opening the file
> > >>
> > >> fiwalk -X path/report.xml path/image.*  -- this one also has trouble
> > >>
> > >>
> > >> The command line below seems to start the process but as far as I can see only processes the first file in the list and none of the others.
> > >>
> > >> fiwalk -X path/report.xml path/image.001
> > >>
> > >>
> > >> Am I missing something in the command line that will process all of the files?
> > >>
> > >> I am using a virtual machine to run linux with SleuthKit installed and the image is on a USB drive.
> > >>
> > >> Thanks,
> > >> Jeff Scarborough
> > >> ------------------------------------------------------------------------------
> > >> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> > >> by Intel and developed in partnership with Slashdot Media, is your hub for all
> > >> things parallel software development, from weekly thought leadership blogs to
> > >> news, videos, case studies, tutorials and more. Take a look and join the
> > >> conversation now. http://goparallel.sourceforge.net/_______________________________________________ <http://goparallel.sourceforge.net/_______________________________________________> <http://goparallel.sourceforge.net/_______________________________________________ <http://goparallel.sourceforge.net/_______________________________________________>>
> > >> sleuthkit-users mailing list
> > >> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users> <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>>
> > >> http://www.sleuthkit.org <http://www.sleuthkit.org/> <http://www.sleuthkit.org/ <http://www.sleuthkit.org/>>
> > >
> > >
> > > ------------------------------------------------------------------------------
> > > Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> > > by Intel and developed in partnership with Slashdot Media, is your hub for all
> > > things parallel software development, from weekly thought leadership blogs to
> > > news, videos, case studies, tutorials and more. Take a look and join the
> > > conversation now. http://goparallel.sourceforge.net/ <http://goparallel.sourceforge.net/> <http://goparallel.sourceforge.net/ <http://goparallel.sourceforge.net/>>
> > > _______________________________________________
> > > sleuthkit-users mailing list
> > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users> <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>>
> > > http://www.sleuthkit.org <http://www.sleuthkit.org/> <http://www.sleuthkit.org/ <http://www.sleuthkit.org/>>
> >
> >
> 
> -------------- next part --------------
> An HTML attachment was scrubbed...
> 
> ------------------------------
> 
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/ <http://goparallel.sourceforge.net/>
> 
> ------------------------------
> 
> _______________________________________________
> sleuthkit-users mailing list
> sle...@li... <mailto:sle...@li...>
> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users <https://lists.sourceforge.net/lists/listinfo/sleuthkit-users>
> 
> 
> End of sleuthkit-users Digest, Vol 105, Issue 17
> ************************************************
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the 
> conversation now. http://goparallel.sourceforge.net/_______________________________________________
> sleuthkit-users mailing list
> https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> http://www.sleuthkit.org