Re: [sleuthkit-users] Tsk_loaddb generating an inconsistent file system tree

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi,

It is not related directly to this issue, but I discovered, based on
Microsoft docs (
https://msdn.microsoft.com/en-us/library/bb470124(v=vs.85).aspx), that the
sequence number is incremented when the file record is freed and not when
it is reused. I think that may affect the relationship between deleted
children and their parent, but I do not tried to confirm it.

Best,
Luis

2016-05-02 15:19 GMT-03:00 Luís Filipe Nassif <lfc...@gm...>:

> Hi,
>
> I applied the patch locally and it resolved the problem. Tested with one
> of my cases.
>
> Thank you very much for all the attention!
> Luis
>
>
> 2016-05-02 9:20 GMT-03:00 Luís Filipe Nassif <lfc...@gm...>:
>
>> Hi Brian,
>>
>> I will gladly build and test the db_cache_uses_path branch and post
>> results here.
>>
>> Thank you,
>> Luis
>>
>> 2016-05-01 21:32 GMT-03:00 Brian Carrier <ca...@sl...>:
>>
>>> Hi Luis,
>>>
>>> I realized I forgot to circle back on this.  I committed a fix a while
>>> back to the db_cache_uses_path branch on github. It would be great if you
>>> could test that before we do another release. Can you build from github or
>>> should I send you and exe?
>>>
>>> thanks,
>>> brian
>>>
>>>
>>> > On Apr 12, 2016, at 1:14 PM, Luís Filipe Nassif <lfc...@gm...>
>>> wrote:
>>> >
>>> > Hi Brian,
>>> >
>>> > Yes, in my colleague's case, the deleted folder (dir2) name is "New
>>> Folder (2)" and the true parent folder has a user specific name, so I think
>>> some rename operation was done. But I am not sure if it was an unique
>>> rename that have caused the problem. In other cases it is not so clear to
>>> me, one has a Windows.old folder involved and looks like a system backup
>>> was done. In another case the false and true parent folders are at
>>> different levels in the FS tree.
>>> >
>>> > Thank you for your investigation,
>>> > Luis
>>> >
>>> > 2016-04-11 22:38 GMT-03:00 Brian Carrier <ca...@sl...>:
>>> > Hi Luis,
>>> >
>>> > I want to understand how we got into this state.  Based on the names
>>> of your folders, is it likely that the deleted folder (“dir2” in our
>>> example below) was moved to be “dir1”?    That is the easiest explanation
>>> of why a deleted name and an allocated name would point to the same entry
>>> and have the same sequence.
>>> >
>>> > It still means there is an issue in the caching code that you pointed
>>> out, I just want to make sure the solution makes sense for this general
>>> problem.
>>> >
>>> > thanks,
>>> > brian
>>> >
>>> > > On Mar 29, 2016, at 10:14 AM, Luís Filipe Nassif <
>>> lfc...@gm...> wrote:
>>> > >
>>> > > Hi Brian,
>>> > >
>>> > > I've attached the istat output for dir1. I do not know how to get
>>> istat result for dir2, because they have the same meta_addr. I searched the
>>> $MFT for dir2 filename and did not find it, I was able to find only dir1
>>> filename in $MFT exactly into the record at meta_addr offset.
>>> > >
>>> > > But I have found dir2 filename into $index_allocation ($I30) of its
>>> parent directory. It is attached too. Finally, dir2 is shown as (realloc)
>>> in fls output.
>>> > >
>>> > > Please let me know if you need any other info.
>>> > >
>>> > > Thank you very much,
>>> > > Luis
>>> > >
>>> > > 2016-03-28 22:59 GMT-03:00 Brian Carrier <ca...@sl...>:
>>> > > Hi Luis,
>>> > >
>>> > > Thanks for this info. Can you send me the ‘istat’ output for dir1
>>> and dir2? I’m curious to see how they are different.
>>> > >
>>> > > thanks,
>>> > > brian
>>> > >
>>> > >
>>> > > > On Mar 24, 2016, at 9:29 AM, Luís Filipe Nassif <
>>> lfc...@gm...> wrote:
>>> > > >
>>> > > > Hi Brian,
>>> > > >
>>> > > > I have just confirmed fls shows the file path as /dir1/file1.txt
>>> based on your example.
>>> > > >
>>> > > > But in tsk_objects table the parent of file1.txt is dir2, which
>>> has the same meta_addr and meta_seq of dir1. I think the problem is in the
>>> parent cache map used in TskDbSqlite::storeObjId() and
>>> TskDbSqlite::findParObjId()
>>> > > >
>>> > > > I detected the issue in my cases searching for allocated files
>>> under unallocated folders, maybe you could find that in your test cases.
>>> > > >
>>> > > > Thank you very much for your time,
>>> > > > Luis
>>> > > >
>>> > > > 2016-03-24 0:11 GMT-03:00 Luís Filipe Nassif <lfc...@gm...
>>> >:
>>> > > > Hi Brian,
>>> > > >
>>> > > > Your example illustrates the issue, except that file1.txt and dir1
>>> are allocated and dir2 is unallocated.
>>> > > >
>>> > > > I will run fls on one image to check what it reports and post
>>> results.
>>> > > >
>>> > > > Thank you,
>>> > > > Luis
>>> > > >
>>> > > > Em 23 de mar de 2016 23:44, "Brian Carrier" <ca...@sl...>
>>> escreveu:
>>> > > > Hi Luis,
>>> > > >
>>> > > > Can you provide a sanitized example of what you are seeing?  Is it
>>> that:
>>> > > >
>>> > > > - You have an unallocated file.  If you look at its parent_path
>>> and file name in tsk_file, then you get something like /dir1/ and file1.txt
>>> (i.e. /dir1/file1.txt)
>>> > > > - When you use the DB / Autopsy, the file is actually shown as a
>>> child of another directory, dir2 for example.  (i.e. /dir2/file1.txt).
>>> > > > - dir1 and dir2 have the same meta_add and meta_seq values in the
>>> database.
>>> > > > - If you were to run ‘fls’ on the image, it would show the file as
>>> (/dir1/file1.txt).
>>> > > >
>>> > > > Is this correct?
>>> > > >
>>> > > > At some point in the past couple of years, we added a bunch of
>>> sequence checks because we were incorrectly mapping files to parent
>>> folders.  In theory, the sequence number is incremented each time an entry
>>> is reallocated, so it should be a reliable way of mapping it to the correct
>>> path.
>>> > > >
>>> > > > thanks,
>>> > > > brian
>>> > > >
>>> > > >
>>> > > > > On Mar 22, 2016, at 12:10 PM, Luís Filipe Nassif <
>>> lfc...@gm...> wrote:
>>> > > > >
>>> > > > > Hi,
>>> > > > >
>>> > > > > A colleague of mine have observed tsk_loaddb 4.2 (so I think
>>> Autopsy too) incorrectly decoding the file system tree of some NTFS images.
>>> I analysed one sqlite sent by him and tsk_loaddb is putting a number of
>>> (very important) files into a deleted directory with the same meta_addr and
>>> meta_seq of the true (not deleted) parent directory of those files,
>>> according to tsk_objects table. The parent_path of the child files are
>>> populated correctly into sqlite.
>>> > > > >
>>> > > > > Maybe the parent cache table logic used internally by tsk_loaddb
>>> should be updated to handle that situation (NTFS files with same meta_addr
>>> and meta_seq)?
>>> > > > >
>>> > > > > We will gladly provide any other information to help solve the
>>> problem.
>>> > > > >
>>> > > > > Thank you very much for your attention,
>>> > > > > Luis
>>> > > > >
>>> > > > >
>>> ------------------------------------------------------------------------------
>>> > > > > Transform Data into Opportunity.
>>> > > > > Accelerate data analysis in your applications with
>>> > > > > Intel Data Analytics Acceleration Library.
>>> > > > > Click to learn more.
>>> > > > >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140_______________________________________________
>>> > > > > sleuthkit-users mailing list
>>> > > > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
>>> > > > > http://www.sleuthkit.org
>>> > > >
>>> > > >
>>> > > >
>>> ------------------------------------------------------------------------------
>>> > > > Transform Data into Opportunity.
>>> > > > Accelerate data analysis in your applications with
>>> > > > Intel Data Analytics Acceleration Library.
>>> > > > Click to learn more.
>>> > > >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140_______________________________________________
>>> > > > sleuthkit-users mailing list
>>> > > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
>>> > > > http://www.sleuthkit.org
>>> > >
>>> > >
>>> > >
>>> <istat_of_dir1.txt><$I30_of_dir2_parent>------------------------------------------------------------------------------
>>> > > Transform Data into Opportunity.
>>> > > Accelerate data analysis in your applications with
>>> > > Intel Data Analytics Acceleration Library.
>>> > > Click to learn more.
>>> > >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785471&iu=/4140_______________________________________________
>>> > > sleuthkit-users mailing list
>>> > > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
>>> > > http://www.sleuthkit.org
>>> >
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > Find and fix application performance issues faster with Applications
>>> Manager
>>> > Applications Manager provides deep performance insights into multiple
>>> tiers of
>>> > your business applications. It resolves application problems quickly
>>> and
>>> > reduces your MTTR. Get your free trial!
>>> >
>>> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z_______________________________________________
>>> > sleuthkit-users mailing list
>>> > https://lists.sourceforge.net/lists/listinfo/sleuthkit-users
>>> > http://www.sleuthkit.org
>>>
>>>
>>
>