[sleuthkit-developers] Re: IO Subsystem patch for fstools

[Michael C. <mic...@ne...>]

Hi Brian,
> Ok.  I was under the impression that you wanted to have a configuration
> file for any of the more complex subsystems and therefore the options
> could be specified there.  The offset is the only variable in the
> process that may change between executions (i.e. accessing a different
> partition) and doesn't make sense to be in a config file.  Unless the
> config file allowed you to assign names to offsets.  For example,
> assign the name 'part1' to sector offset 63 and then you could use 'fls
> -f ntfs -o part1 image.dd'.
Although i would normally agree with you that configuration files can contain 
much more information and therefore most parameters should go there, in this 
case it would be a pain to have all these options passed in through a 
configuration file. This is primarily because fstools are most often used as 
a backend to larger programs (like flag and autopsy), and so its much easier 
to shell out to them even if they have 20 command line arguements, than have 
to write a config file and then invoke them with it.

> That is why I was assuming that a configuration file would be used for
> complex situations.  What does the blocksize value do for a split
> image?  It seems that only the RAID / VM configurations need complex
> options.  The split mode (or EnCase if that happens in the future) can
> be done w/out options.
To keep the system truely generic its useful to allow arbitrary options to be 
passed to the subsystem, rather than trying to incorporate within the current 
framework, its difficult to imagine what parameters will be required for some 
arbitrary subsystem someone might come up with in the future.

> I would rather force complex configurations to configuration files.
> The command line options for the sleuth kit are already too numerous
> and it will make using Autopsy easier if the config file can be
> referenced instead of having to load up the command line every time.
This makes sense if the config file was the same (not changing) for a single 
case for example, or a set of images. So the config file only gets written 
once and then continually referenced from invokation to invokation, rather 
than have to rewrite the file each time.

> Oh.  I was thinking that the configuration file would have an entry
> that identified which IO subsystem to use.  For example, a line that
> says:
>
> image_format = "split"
> or
> image_format = "lvm-splice"
There might be a misunderstanding with the raid stuff. What i am writing is a 
generic raid subsystem that reassembles any raid 5 implementation without 
having a clue about its type/brand etc. The only reason it prob should have a 
config file is because its so huge. (the raid map is big).

> Oh ok.  I think that it will be very hard to create such a
> configuration file.  To create the file, you will need to know which VM
> / RAID system is being used.  I think it would be much easier to have a
> subsystem for each VM / RAID type and then the only thing that needs to
> be specified in the configuration file is the options for that type.
> For example, if the Linux LVM were used, then you may need to only
> specify the disk ordering and the block size.  When reading from the
> image, the lvm-split-read() function would be used.

The raid map is easily assembled using the gui in flag. This is not a problem. 
If the investigator knows which raid implementation it is, they can lookup a 
library of such maps and use those, but the io subsystem needs to handle 
arbitrary reassembly maps.

Maybe we can implement such a library in fs_io, so that you could invoke the 
same raid subsystem, and pass it a parameter say "type" naming which raid 
type it should use and it can look up the map by itself?

> I have no problems changing all of the files.  If we are going to add
> this functionality, I would rather do it right the first time.
Great. What you propose is a much better way. We just need to resolve the 
option issue because these options need to be present inside such an IO_INFO 
struct that will be passed around the place. Since these options can be 
arbitrary and they only make sense for the subsystem itself, what do you 
think about my original suggestion of having a single void *data field in the 
IO_INFO struct and letting each subsystem put they stuff in there?

> Actually, I guess we just need one io_open() function because fls.c and
> similar files will not know if the file is a config file or an image
> file.  io_open would have a char ** to list the  image files or config
> file, a type field for the type of image format, and an offset value.
> It would then fill in the IO_INFO structure and return it, which would
> be passed to fs_open().
Maybe it would make more sense to populate the IO_INFO structure inside the 
FS_INFO structure? so instead of having a fs->fd, we have a fs->io_info. 
Makes more sense since with io_subsystems, the FS_INFO structure doesnt 
really deal with file descriptors anyway (the filesystem code never directly 
interacts with fds). So in the filesystem code you would have:
	if ((fs->io_info = io_open(name, O_RDONLY)) < 0)
		...

	and then, subsequently:
	fs_read_random(fs->io_info,(char *)ext2fs->fs,len,EXT2FS_SBOFF,"Checking for 
EXT2FS");
	....

	Next we just need to agree on what to put in struct IO_INFO.

> Compression would be a major pain.  Split, EnCase, and some of the RAID
> systems seem much easier.
Please see the patch i just submitted re compression.

Do you have any idea how you would read in encase files? I didnt get the 
chance to ever use it so i dont know how complex the file format is but there 
is nothing i can find on the net re the format.

> thanks,
> brian
Thanks,
Michael.