Re: [sleuthkit-users] [sleuthkit-developers] Millions of orphan files found with sleuthkit develop branch

[Luís F. N. <lfc...@gm...>]

Hi folks,

We are still having this issue with tsk-4.4. I have received one report
yesterday and one today about the processing hanging with thousands of
orphan files with ~4GB of size, which together result in 140TB of data in
one image!

Fortunately I have access to the image of the other report. Looking at it,
the orphans together resulted in ~500GB of data, they were recovered from a
FAT32 file system of only ~32GB of size! That 32GB FAT32 FS was recovered
from a 92,5 KB volume/partition! This 92,5KB partition recognized by
TSK-4.4 is shown by FTKImager as an unpartitioned space, but I think this
is another issue.

I have trimmed the image down from the original 320GB to 10GB (5GB in ewf
format) without the user data. Because of that, the orphan files together
now result in ~45GB (not the original 500GB). I will open a ticket at
github and post the link to the trimmed image there for reproducing the
problem.

PS: In the past I have received similar reports with thumbdrives. So looks
like the issue is with FAT and not with NTFS like I originally reported
below. Probably those ntfs images had fat32 partitions side by side. This
explains the upper limit of 4GB of size of those orphans.

Regards,
Luis Nassif


2015-09-07 12:12 GMT-03:00 Luís Filipe Nassif <lfc...@gm...>:

> Sorry for the long delay. I do not have the image with me, I will ask my
> colleague if trimming the image is possible... We worked around the problem
> by filtering out orphans with logical size greater than 10 MB before
> sending them to the processing engine.
>
> Thank you,
> Luis
>
> 2015-08-13 14:13 GMT-03:00 Stefan Petrea <ste...@gm...>:
>
>> Hi Luis,
>>
>> Could the NTFS image you're looking at be trimmed down and provided as
>> sample input to reproduce the problem ?
>>
>> Best Regards,
>> Stefan
>>
>> On Thu, Aug 13, 2015 at 8:05 PM, Luís Filipe Nassif <lfc...@gm...>
>> wrote:
>>
>>> This error have happened again with a colleague's NTFS image, using the
>>> develop branch compiled about 1 month ago. Thousands of huge corrupted
>>> orphans were added by loaddb, which caused our processing application (and
>>> probably Autopsy too) to process indefinitely the evidence.
>>>
>>> Any help will be appreciated.
>>>
>>> Regards,
>>> Luis Nassif
>>>
>>>
>>> 2014-09-30 21:00 GMT-03:00 Luís Filipe Nassif <lfc...@gm...>:
>>>
>>>> This problem still happens with 4.2.0 branch. If I can help with some
>>>> more information, please let me know.
>>>>
>>>> Thanks
>>>> Luis
>>>>
>>>> 2014-07-24 9:21 GMT-03:00 Luís Filipe Nassif <lfc...@gm...>:
>>>>
>>>>> Another information: the sum of the millions of file sizes resulted in
>>>>> 1,1 petabyte, while the image has only 250 GB.
>>>>>
>>>>>
>>>>> 2014-07-23 22:21 GMT-03:00 Luís Filipe Nassif <lfc...@gm...>:
>>>>>
>>>>>> We tested loaddb of both the released 4.1.3 version and the develop
>>>>>> branch of sleuthkit on a NTFS image of a hard disk with a lot of bad
>>>>>> blocks, many of them at the beginning of the disk.
>>>>>>
>>>>>> The 4.1.3 version found ~400.000 allocated files more ~100.000 orphan
>>>>>> files, about the same found by other forensic tools. The develop branch
>>>>>> found the same ~400.000 allocated files more ~2.500.000 orphan files! Most
>>>>>> of these millions of orphans have corrupted names or the name
>>>>>> OrphanFile-xxxxxxx and have lengths ranging from 0 to 4.294.967.296 bytes.
>>>>>> We think the recent changes to NTFS code are causing this large number of
>>>>>> corrupted orphans to be added to the case. Maybe it should be investigated
>>>>>> before the final 4.2 release.
>>>>>>
>>>>>> Luis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> ------------------------------------------------------------
>>> ------------------
>>>
>>> _______________________________________________
>>> sleuthkit-developers mailing list
>>> sle...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sleuthkit-developers
>>>
>>>
>>
>