Re: [sleuthkit-users] Tsk_loaddb generating an inconsistent file system tree

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi Brian,

I will gladly build and test the db_cache_uses_path branch and post results
here.

Thank you,
Luis

2016-05-01 21:32 GMT-03:00 Brian Carrier <ca...@sl...>:

> Hi Luis,
>
> I realized I forgot to circle back on this.  I committed a fix a while
> back to the db_cache_uses_path branch on github. It would be great if you
> could test that before we do another release. Can you build from github or
> should I send you and exe?
>
> thanks,
> brian
>
>
> > On Apr 12, 2016, at 1:14 PM, Luís Filipe Nassif <lfc...@gm...>
> wrote:
> >
> > Hi Brian,
> >
> > Yes, in my colleague's case, the deleted folder (dir2) name is "New
> Folder (2)" and the true parent folder has a user specific name, so I think
> some rename operation was done. But I am not sure if it was an unique
> rename that have caused the problem. In other cases it is not so clear to
> me, one has a Windows.old folder involved and looks like a system backup
> was done. In another case the false and true parent folders are at
> different levels in the FS tree.
> >
> > Thank you for your investigation,
> > Luis
> >
> > 2016-04-11 22:38 GMT-03:00 Brian Carrier <ca...@sl...>:
> > Hi Luis,
> >
> > I want to understand how we got into this state.  Based on the names of
> your folders, is it likely that the deleted folder (“dir2” in our example
> below) was moved to be “dir1”?    That is the easiest explanation of why a
> deleted name and an allocated name would point to the same entry and have
> the same sequence.
> >
> > It still means there is an issue in the caching code that you pointed
> out, I just want to make sure the solution makes sense for this general
> problem.
> >
> > thanks,
> > brian
> >
> > > On Mar 29, 2016, at 10:14 AM, Luís Filipe Nassif <lfc...@gm...>
> wrote:
> > >
> > > Hi Brian,
> > >
> > > I've attached the istat output for dir1. I do not know how to get
> istat result for dir2, because they have the same meta_addr. I searched the
> $MFT for dir2 filename and did not find it, I was able to find only dir1
> filename in $MFT exactly into the record at meta_addr offset.
> > >
> > > But I have found dir2 filename into $index_allocation ($I30) of its
> parent directory. It is attached too. Finally, dir2 is shown as (realloc)
> in fls output.
> > >
> > > Please let me know if you need any other info.
> > >
> > > Thank you very much,
> > > Luis
> > >
> > > 2016-03-28 22:59 GMT-03:00 Brian Carrier <ca...@sl...>:
> > > Hi Luis,
> > >
> > > Thanks for this info. Can you send me the ‘istat’ output for dir1 and
> dir2? I’m curious to see how they are different.
> > >
> > > thanks,
> > > brian
> > >
> > >
> > > > On Mar 24, 2016, at 9:29 AM, Luís Filipe Nassif <lfc...@gm...>
> wrote:
> > > >
> > > > Hi Brian,
> > > >
> > > > I have just confirmed fls shows the file path as /dir1/file1.txt
> based on your example.
> > > >
> > > > But in tsk_objects table the parent of file1.txt is dir2, which has
> the same meta_addr and meta_seq of dir1. I think the problem is in the
> parent cache map used in TskDbSqlite::storeObjId() and
> TskDbSqlite::findParObjId()
> > > >
> > > > I detected the issue in my cases searching for allocated files under
> unallocated folders, maybe you could find that in your test cases.
> > > >
> > > > Thank you very much for your time,
> > > > Luis
> > > >
> > > > 2016-03-24 0:11 GMT-03:00 Luís Filipe Nassif <lfc...@gm...>:
> > > > Hi Brian,
> > > >
> > > > Your example illustrates the issue, except that file1.txt and dir1
> are allocated and dir2 is unallocated.
> > > >
> > > > I will run fls on one image to check what it reports and post
> results.
> > > >
> > > > Thank you,
> > > > Luis
> > > >
> > > > Em 23 de mar de 2016 23:44, "Brian Carrier" <ca...@sl...>
> escreveu:
> > > > Hi Luis,
> > > >
> > > > Can you provide a sanitized example of what you are seeing?  Is it
> that:
> > > >
> > > > - You have an unallocated file.  If you look at its parent_path and
> file name in tsk_file, then you get something like /dir1/ and file1.txt
> (i.e. /dir1/file1.txt)
> > > > - When you use the DB / Autopsy, the file is actually shown as a
> child of another directory, dir2 for example.  (i.e. /dir2/file1.txt).
> > > > - dir1 and dir2 have the same meta_add and meta_seq values in the
> database.
> > > > - If you were to run ‘fls’ on the image, it would show the file as
> (/dir1/file1.txt).
> > > >
> > > > Is this correct?
> > > >
> > > > At some point in the past couple of years, we added a bunch of
> sequence checks because we were incorrectly mapping files to parent
> folders.  In theory, the sequence number is incremented each time an entry
> is reallocated, so it should be a reliable way of mapping it to the correct
> path.
> > > >
> > > > thanks,
> > > > brian
> > > >
> > > >
> > > > > On Mar 22, 2016, at 12:10 PM, Luís Filipe Nassif <
> lfc...@gm...> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > A colleague of mine have observed tsk_loaddb 4.2 (so I think
> Autopsy too) incorrectly decoding the file system tree of some NTFS images.
> I analysed one sqlite sent by him and tsk_loaddb is putting a number of
> (very important) files into a deleted directory with the same meta_addr and
> meta_seq of the true (not deleted) parent directory of those files,
> according to tsk_objects table. The parent_path of the child files are
> populated correctly into sqlite.
> > > > >
> > > > > Maybe the parent cache table logic used internally by tsk_loaddb
> should be updated to handle that situation (NTFS files with same meta_addr
> and meta_seq)?
> > > > >
> > > > > We will gladly provide any other information to help solve the
> problem.
> > > > >
> > > > > Thank you very much for your attention,
> > > > > Luis
> > > > >
> > > > >
> ------------------------------------------------------------------------------
> > > > > Transform Data into Opportunity.
> > > > > Accelerate data analysis in your applications with
> > > > > Intel Data Analytics Acceleration Library.
> > > > > Click to learn more.
> > > > >
> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140_______________________________________________
> > > > > sleuthkit-users mailing list
> > > > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> > > > > http://www.sleuthkit.org
> > > >
> > > >
> > > >
> ------------------------------------------------------------------------------
> > > > Transform Data into Opportunity.
> > > > Accelerate data analysis in your applications with
> > > > Intel Data Analytics Acceleration Library.
> > > > Click to learn more.
> > > >
> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140_______________________________________________
> > > > sleuthkit-users mailing list
> > > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> > > > http://www.sleuthkit.org
> > >
> > >
> > >
> <istat_of_dir1.txt><$I30_of_dir2_parent>------------------------------------------------------------------------------
> > > Transform Data into Opportunity.
> > > Accelerate data analysis in your applications with
> > > Intel Data Analytics Acceleration Library.
> > > Click to learn more.
> > >
> http://pubads.g.doubleclick.net/gampad/clk?id=278785471&iu=/4140_______________________________________________
> > > sleuthkit-users mailing list
> > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> > > http://www.sleuthkit.org
> >
> >
> >
> ------------------------------------------------------------------------------
> > Find and fix application performance issues faster with Applications
> Manager
> > Applications Manager provides deep performance insights into multiple
> tiers of
> > your business applications. It resolves application problems quickly and
> > reduces your MTTR. Get your free trial!
> >
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z_______________________________________________
> > sleuthkit-users mailing list
> > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
> > http://www.sleuthkit.org
>
>