[sleuthkit-developers] IO Subsystem patch for fstools

[Michael C. <mic...@ne...>]

Dear List,
    Please accept this patch to the sleuthkit to implement a pluggable IO 
subsystem for the fstools. (patch against 1.67, fstools directory).

    Background
    Quite often users are supplied with dd images that do not immediately work 
with sleuthkit. Two notable examples are:
	- when a dd image was taken of the hdd - in this case users have to use 
sfdisk to work out the partition offsets and then use dd with appropriate 
skip parameters to extract each partition, before being able to use the 
sleuthkit. This is because the sk expects to have a dd image of a partition 
(i.e. filesystem starts at offset 0 in the image file. This is not always the 
case).
	- Sometimes images are split into smaller sizes for example in order to burn 
to cd/dvd etc. This means that images need to be stuck together before 
analysis potentially wasting time and space.

  It would be nice if one could use the images directly - without needing to 
do creative dd manipulations.

   Solution
   This patch implements a modular io subsystem approach - all filesystem 
operations within the sk are made to use this subsystem, and the user can 
choose the subsystem they want. The subsystem is responsible to seeking into 
the file and extracting data out of the dd image - how that is implemented is 
completely abstracted from the point of view of the fstools.

  The user can choose the subsystem to be used by the -i (io subsystem) 
command line switch. Then a list of arguments can be passed to the subsystem 
to initialise it correctly. Once that is done, the regular sk calls can be 
made (e.g. fs_open etc). The io subsystem will take care of the specifics of 
implementation.

  This patch includes 2 subsystem modules: simple and advanced. The simple 
module is exactly the same as the old sk, while the advanced module allows 
for specifying offsets into the dd file, as well as multiple dd files in 
sequence.

   Example:
	As an example the fls and icat tools were modified to support the new sub 
system, more tools will be converted tomorrow once i get some sleep. Example 
of how to seek into a partition within a disk dd:

	fls -i advanced -o offset=524288 -f linux-ext2 test.dd

	This selects the advanced io subsystem and passes it the offset option 
specifying 1024 blocks of 512 bytes.

	Now we can split the dd image across multiple files (maybe using the split 
utility), and still analyse them at once:

	fls -i advanced -o offset=524288,file=xaa,file=xab,file=xac,file=xad -f 
linux-ext2 xae

	Note that xae (the last part of the image will be appened to the list of 
parts automatically). Also note that all the options in -o are passed as one 
parameter to the subsystem which then parses them into the relevant 
arguements.

    If the subsystems name is not found, the subsystem will list all known 
subsystems:

	bash# fls -i help -f linux-ext2 test.dd

Available Subsystems:
 
        standard - Standard Sleuthkit IO Subsystem
        advanced - Advanced Sleuthkit IO Subsystem
fls: Could not set io subsystem help

   To get more help about the options available, try setting an option which 
is not supported:

	bash# fls -i  advanced -o help -f linux-ext2 test.dd

option help not recognised
 
Advanced io subsystem options
 
        offset=bytes            Number of bytes to seek to in the image file. 
Useful if there is some extra data at the start of the dd image (e.g. 
partition table/other partitions
        file=filename           Filename to use for split files. If your dd 
image is split across many files, specify this parameter in the order 
required as many times as needed for seemless integration

	Future work:
	I am in the process of implementing a raid reassembly functionality. I.e. 
given a raid reconstruction map (a file telling sk the order in which raid 
blocks go together) and a list of dd images of individual drives, the io 
subsystem will transparently reassemble the logical data. I have a working 
prototype so i know its possible. The abstracted io subsystem concept will be 
very handy for that.