[sleuthkit-developers] Resident files and carvpaths (ocfa tskfs)

[Rob M. <pi...@gm...>]

Basically a carvpath is a simple file-system-path compatible
annotation of the offset/size and/or sparse fragments that make up  a
file, partition or block within a larger entity (for example a disk
image).
The CarvPath library is used both by the forensic tool (in this case
the tskfs OCFA module that uses libtsk) and the CarvFS pseudo
filesystem.

For example a CarvPath may that may look something like:
"40960+512_S1024_41472+512/256+512" representing a single fragment
entity within a second 3 fragment entity (that consists of an
ofset/size fragment, a sparse fragment and a second offset/size
fragment). LibCarvPath could flatten this carvpath to a shorter two
fragment one level carvpath "41216+256_S256" (consisting of an
offset/size fragment and a sparse fragment.

Basically what now happens in OCFA is that:

1) A kicktree module appends a sparse version of a disk image to a
large growing archive that is mounted using CarvFS and submits a
carvpath representing that image into OCFA.
2) The image may get router to the carvpath aware (libtsk based) mmls
module, that would extract the carvpaths of the partitions.
3) One of the partitions may, having an NTFS filesystem on it, as a
carvpath get routed to the carvpath aware (libtsk based) tskfs module,
that would extract carvpaths for all suitable (that is :
non-compressed, non-encrypted, but currently also only non-resident)
files.
4) One of the files may be an ISO image that as  a carvpath would also
get routed to the carvpath aware (libtsk based) tskfs module, that
would extract the files also as carvpaths.
5) The files from the ISO and other files from the NTFS will get
routed to other ocfa modules for data and/or meta-data extraction.
6) The partition may after getting back from tskfs get routed to the
carvpath aware (libtsk based) blkls module that will extract
non-interrupted blocks of unallocated data from the NTFS partition and
represent them as carvpaths.
7) The unallocated blocks may get routed to the carvpath aware
(scalpel based) carver module that will extract carved files as
carvpaths.

You get things like "mnt/0/CarvFS/<partition
annotation>/<iso-image-file-annotation>/<file-from-iso-annotation>.crv"
or "/mnt/0/CarvFS/<partition
annotation>/<unallocated-block-annotation>/<carved-file-annotation>.crv"
anailable as a pseudo file to data/meta-data extraction tools like
exiftags or antiword, without ever any tool needing to copy-out any
data to other files on a real filesystem.

I hope the above explanation is clear.

The point now is that the above works fine for the most part and the
libtsk library works great for most of the process. The only minor
issue we have is with our tskfs module when it processes NTFS. In NTFS
some of the files are resident, and I havn't been able to find my way
in the libtsk API to get the tskfs module to gather information needed
to represent these resident files as offset/size and/or sparse
fragments needed to produce a carvpath for them.

The code for tskfs is under OcfaModules/tree/tskfs in the OCFA source
distribution on sourceforge (
http://sourceforge.net/projects/ocfa/files/ ), with the relevant
classes being TskFsInode and TskFsResidentDataAttribute and possibly
TskFsCarvPathDataAttribute (with their code in equally named hpp and
cpp files).

Could you possibly have a quick look at the code and maybe propose
changes to the code using the libtsk API that would allow me to
implement TskFsResidentDataAttribute in a way not unlike
TskFsCarvPathDataAttribute as to allow resident files to use the same
libcarvpath based zero storage techniques with resident files that it
uses with non resident files? I looked at the API over and over and
can't figure out how I could make this work. Possibly the API simply
doesn't allow this (yet?), but more likely I simply don't understand
how to use the existing in order to gain access to the information I
need.

Rob




010/11/10 Brian Carrier <ca...@sl...>:
> What is a "carvpath"?
>
> On Nov 4, 2010, at 1:39 AM, Rob Meijer wrote:
>
>> Yesterday the first release candidate for ocfa (open computer forensics architecture) was released. Jn this version of ocfa the main new thing is that the perl script calling sleuthkit tools was completely replaced with (carvfs aware) treegraph ocfa modules that use libtsk. This setup allows ocfa to use zero storage techniques for most of the extracted slethkit data. There are basically two reasons why copy out would still be needed at the moment. Compression and residentness. I believe howaver that resident files must be expressable as carvpath. If anyone on this list would want to have a quick look at the OcfaModules/tree/tskfs ocfa code for resident and non resident files, maybe we could find a way for resident files to be also represented as carvpaths. Tia,
>>
>> Rob
>>
>> ------------------------------------------------------------------------------
>> The Next 800 Companies to Lead America's Growth: New Video Whitepaper
>> David G. Thomson, author of the best-selling book "Blueprint to a
>> Billion" shares his insights and actions to help propel your
>> business during the next growth cycle. Listen Now!
>> http://p.sf.net/sfu/SAP-dev2dev_______________________________________________
>> sleuthkit-developers mailing list
>> sle...@li...
>> https://lists.sourceforge.net/lists/listinfo/sleuthkit-developers
>
>