[sleuthkit-developers] Re: IO Subsystem patch for fstools

[Brian C. <ca...@sl...>]

On Feb 7, 2004, at 7:52 AM, Michael Cohen wrote:
> Although i would normally agree with you that configuration files can 
> contain
> much more information and therefore most parameters should go there, 
> in this
> case it would be a pain to have all these options passed in through a
> configuration file. This is primarily because fstools are most often 
> used as
> a backend to larger programs (like flag and autopsy), and so its much 
> easier
> to shell out to them even if they have 20 command line arguements, 
> than have
> to write a config file and then invoke them with it.

Yea, but autopsy or flag need to store all of those options as well in 
their own configuration file so that they can pass them to the fstools. 
  It would seem more flexible if the sleuth kit had a configuration file 
for the image, which could then be used by any GUI (including autopsy, 
flag, rex etc.)

If we are going to start discussing configuration files for the fstools 
(which we both agree are required for at least RAID), then I would 
rather make then general enough so that they can be used for other 
formats besides RAID.  I would even like to have these include
the file system type, mounting point, and hashes of each partition.   
Basically the stuff that the other tools include in some proprietary 
format with the image, we would put in a separate text file.


>> Oh ok.  I think that it will be very hard to create such a
>> configuration file.  To create the file, you will need to know which 
>> VM
>> / RAID system is being used.  I think it would be much easier to have 
>> a
>> subsystem for each VM / RAID type and then the only thing that needs 
>> to
>> be specified in the configuration file is the options for that type.
>> For example, if the Linux LVM were used, then you may need to only
>> specify the disk ordering and the block size.  When reading from the
>> image, the lvm-split-read() function would be used.
>
> The raid map is easily assembled using the gui in flag. This is not a 
> problem.
> If the investigator knows which raid implementation it is, they can 
> lookup a
> library of such maps and use those, but the io subsystem needs to 
> handle
> arbitrary reassembly maps.
>
> Maybe we can implement such a library in fs_io, so that you could 
> invoke the
> same raid subsystem, and pass it a parameter say "type" naming which 
> raid
> type it should use and it can look up the map by itself?

Sure.  For this to work with the Sleuth Kit though, there must be the 
ability to create the configurations in the sleuth kit.   If the only 
way to create the map is in flag, then that doesn't do autopsy any good 
or future interfaces and it doesn't make sense to replicate the stuff 
in each gui.

>> I have no problems changing all of the files.  If we are going to add
>> this functionality, I would rather do it right the first time.
> Great. What you propose is a much better way. We just need to resolve 
> the
> option issue because these options need to be present inside such an 
> IO_INFO
> struct that will be passed around the place. Since these options can be
> arbitrary and they only make sense for the subsystem itself, what do 
> you
> think about my original suggestion of having a single void *data field 
> in the
> IO_INFO struct and letting each subsystem put they stuff in there?

I'm still not convinced that we need so many options on the command 
line.  The only case that I can see where all of the command line 
options are beneficial is for a live analysis where you don't want to 
write to the disk.  But, in that case I don't see why you would need to 
use any of these complex image formats because you will have access to 
the raw device corresponding to the partition.

Is there a specific reason with flag that command line options are 
easier?

>> Actually, I guess we just need one io_open() function because fls.c 
>> and
>> similar files will not know if the file is a config file or an image
>> file.  io_open would have a char ** to list the  image files or config
>> file, a type field for the type of image format, and an offset value.
>> It would then fill in the IO_INFO structure and return it, which would
>> be passed to fs_open().
> Maybe it would make more sense to populate the IO_INFO structure 
> inside the
> FS_INFO structure?

I would rather not.  I would prefer to keep the file system code 
separate from the image format code.  In fact, I would even consider 
making all of this image stuff its own library,
imgtools maybe.  It seems much more logical to call the file system 
processing code with the filled in IO_INFO structure and let it read 
from it.  The file system code would never touch any of the file 
descriptors, it would just call the read functions.  This also allows 
the 'mm...' tools to use the image formats and any other future tools, 
such as memory images that are split or saved in another tool's 
proprietary format.

> 	Next we just need to agree on what to put in struct IO_INFO.

I would lean towards the way that FS_INFO is structured.  There would 
be a few basic items in IO_INFO, such as the function pointers and 
maybe the maximum size of the image.  Then there are image specific 
structures that have their needed values.  For example, the structure 
for split images may have an array of file descriptors and  a structure 
with the sizes of each split image. The normal image structure may just 
have one file descriptor.  Actually, maybe this whole thing is better 
called IMG_INFO instead of IO_INFO.

In the imgtools collection, we could actually  have a tool that 
converts the proprietary image formats to a raw image.

>> Compression would be a major pain.  Split, EnCase, and some of the 
>> RAID
>> systems seem much easier.
> Please see the patch i just submitted re compression.

Very cool. I had never seen sgzip before.  I guess it isn't as much of 
a pain as I thought :)

> Do you have any idea how you would read in encase files? I didnt get 
> the
> chance to ever use it so i dont know how complex the file format is 
> but there
> is nothing i can find on the net re the format.

Check out asrdata.com.  Somewhere on there is a link to the expert 
witness format.

I apologize if I am being a pain with some of these details, but after 
having to redesign autopsy because of a bad initial design, I want to 
make sure we add this new functionality the right way.

thanks,
brian