Re: [sleuthkit-developers] Resident files and carvpaths (ocfa tskfs)

[Oscar V. <osc...@gm...>]

I think this all boils down to the question whether it is possible to
get an offset (relative to the start of the filesystem) to the $DATA
attribute of a resident file using the current TSK api.
Looking at the TSK_FS_ATTR struct, there is a size field. So, a
possible solution might be to iterate through all attributes of a file
(MFT record) adding the sizes as you go (assuming the size includes
the header of the attribute and the attributes are not alligned on
some boundary other than 1) until you hit the $DATA attribute.
The offset of the MFT record relative to the start of the filesystem
could be calculated from its "filenumber" and the MFT record size and
the starting address of the MFT itself.

This would probably give the info needed to construct a valid
carvpath. The question is whether there is a more direct way to get
this from the TSK api.

regards
Oscar

On Thu, Nov 11, 2010 at 12:52 PM, Rob Meijer <pi...@gm...> wrote:
> Basically a carvpath is a simple file-system-path compatible
> annotation of the offset/size and/or sparse fragments that make up  a
> file, partition or block within a larger entity (for example a disk
> image).
> The CarvPath library is used both by the forensic tool (in this case
> the tskfs OCFA module that uses libtsk) and the CarvFS pseudo
> filesystem.
>
> For example a CarvPath may that may look something like:
> "40960+512_S1024_41472+512/256+512" representing a single fragment
> entity within a second 3 fragment entity (that consists of an
> ofset/size fragment, a sparse fragment and a second offset/size
> fragment). LibCarvPath could flatten this carvpath to a shorter two
> fragment one level carvpath "41216+256_S256" (consisting of an
> offset/size fragment and a sparse fragment.
>
> Basically what now happens in OCFA is that:
>
> 1) A kicktree module appends a sparse version of a disk image to a
> large growing archive that is mounted using CarvFS and submits a
> carvpath representing that image into OCFA.
> 2) The image may get router to the carvpath aware (libtsk based) mmls
> module, that would extract the carvpaths of the partitions.
> 3) One of the partitions may, having an NTFS filesystem on it, as a
> carvpath get routed to the carvpath aware (libtsk based) tskfs module,
> that would extract carvpaths for all suitable (that is :
> non-compressed, non-encrypted, but currently also only non-resident)
> files.
> 4) One of the files may be an ISO image that as  a carvpath would also
> get routed to the carvpath aware (libtsk based) tskfs module, that
> would extract the files also as carvpaths.
> 5) The files from the ISO and other files from the NTFS will get
> routed to other ocfa modules for data and/or meta-data extraction.
> 6) The partition may after getting back from tskfs get routed to the
> carvpath aware (libtsk based) blkls module that will extract
> non-interrupted blocks of unallocated data from the NTFS partition and
> represent them as carvpaths.
> 7) The unallocated blocks may get routed to the carvpath aware
> (scalpel based) carver module that will extract carved files as
> carvpaths.
>
> You get things like "mnt/0/CarvFS/<partition
> annotation>/<iso-image-file-annotation>/<file-from-iso-annotation>.crv"
> or "/mnt/0/CarvFS/<partition
> annotation>/<unallocated-block-annotation>/<carved-file-annotation>.crv"
> anailable as a pseudo file to data/meta-data extraction tools like
> exiftags or antiword, without ever any tool needing to copy-out any
> data to other files on a real filesystem.
>
> I hope the above explanation is clear.
>
> The point now is that the above works fine for the most part and the
> libtsk library works great for most of the process. The only minor
> issue we have is with our tskfs module when it processes NTFS. In NTFS
> some of the files are resident, and I havn't been able to find my way
> in the libtsk API to get the tskfs module to gather information needed
> to represent these resident files as offset/size and/or sparse
> fragments needed to produce a carvpath for them.
>
> The code for tskfs is under OcfaModules/tree/tskfs in the OCFA source
> distribution on sourceforge (
> http://sourceforge.net/projects/ocfa/files/ ), with the relevant
> classes being TskFsInode and TskFsResidentDataAttribute and possibly
> TskFsCarvPathDataAttribute (with their code in equally named hpp and
> cpp files).
>
> Could you possibly have a quick look at the code and maybe propose
> changes to the code using the libtsk API that would allow me to
> implement TskFsResidentDataAttribute in a way not unlike
> TskFsCarvPathDataAttribute as to allow resident files to use the same
> libcarvpath based zero storage techniques with resident files that it
> uses with non resident files? I looked at the API over and over and
> can't figure out how I could make this work. Possibly the API simply
> doesn't allow this (yet?), but more likely I simply don't understand
> how to use the existing in order to gain access to the information I
> need.
>
> Rob
>
>
>
>
> 010/11/10 Brian Carrier <ca...@sl...>:
>> What is a "carvpath"?
>>
>> On Nov 4, 2010, at 1:39 AM, Rob Meijer wrote:
>>
>>> Yesterday the first release candidate for ocfa (open computer forensics architecture) was released. Jn this version of ocfa the main new thing is that the perl script calling sleuthkit tools was completely replaced with (carvfs aware) treegraph ocfa modules that use libtsk. This setup allows ocfa to use zero storage techniques for most of the extracted slethkit data. There are basically two reasons why copy out would still be needed at the moment. Compression and residentness. I believe howaver that resident files must be expressable as carvpath. If anyone on this list would want to have a quick look at the OcfaModules/tree/tskfs ocfa code for resident and non resident files, maybe we could find a way for resident files to be also represented as carvpaths. Tia,
>>>
>>> Rob
>>>
>>> ------------------------------------------------------------------------------
>>> The Next 800 Companies to Lead America's Growth: New Video Whitepaper
>>> David G. Thomson, author of the best-selling book "Blueprint to a
>>> Billion" shares his insights and actions to help propel your
>>> business during the next growth cycle. Listen Now!
>>> http://p.sf.net/sfu/SAP-dev2dev_______________________________________________
>>> sleuthkit-developers mailing list
>>> sle...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sleuthkit-developers
>>
>>
>
> ------------------------------------------------------------------------------
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> _______________________________________________
> sleuthkit-developers mailing list
> sle...@li...
> https://lists.sourceforge.net/lists/listinfo/sleuthkit-developers
>