Re: [sleuthkit-users] libtsk vs automation framework

[Brian C. <ca...@sl...>]

On Nov 19, 2012, at 9:29 PM, Jon Stewart <jo...@li...> wrote:

> Hello,
> 
> I have two questions related to similar functionality being exposed in
> different ways between the core Sleuthkit library (libtsk) and the new
> Sleuthkit automation framework.
> 
> 1. TSK_FS_FILE vice TskFile. Libtsk has the TSK_FS_FILE struct which
> exposes a file's metadata and allows access to a file's content.
> TSK_FS_FILE has a split structure, where inode data is in a sub-struct
> and directory information is in another. The automation framework
> introduces the C++ class TskFile, which serves much the same purpose
> as TSK_FS_FILE but has a much different design.
> 
> TskFile provides an abstraction layer over the database (+1). However,
> as far as I can tell, it doesn't appear possible to get a TSK_FS_FILE
> from it, nor is it possible to construct one from a TSK_FS_FILE. This
> means that code written on top of libtsk cannot work in the automation
> framework and vice versa—obviously some code will want the extended
> persistence capabilities of the automation framework, but a great deal
> of tasks can be reduced to "do foo() on a file" and it would be good
> to reuse such code in either context. Is there some way of going
> between the two that I'm missing? If not, will it possible to unify
> these in the future?

We can provide a way to use both in the future.   A couple of comments on this topic:
- The automation framework was designed with the idea that you _could_ use a different file system extraction engine besides TSK. I.e. if there were a file system that TSK did not support and another tool did (or if you just wanted to use a different tool).  So, TskFile in the automation framework is decoupled from TSK_FS_FILE, which is all about TSK. Behind the scenes TskFile is just an interface that is implemented by TskFileTsk that is the TSK implementation of it. 
- TskFileTsk (the automation class) actually has a TSK_FS_FILE object behind the scenes.  To achieve your goal, the lowest common denominator is TSK_FS_FILE because it has less dependences than the automation framework.  We could add a method that allows you to get TSK_FS_FILE from a TskFile. We need to think about that though because if its part of the interface, then it would mean that all implementations of it (i.e. other file system parsers besides TSK) would need to provide that abstraction for it to be universally useful. And, it may impact our resource management so we'll need to figure that out too.
- To complicate things even more, there is the TskFsFile class in TSK core, but I'm not sure if anyone is actually using that at this point. That was added in 4.0 as a C++ wrapper around TSK_FS_FILE and it provides better protection in the case of multi-threaded apps. 


> 2. Similarly, the TskAuto class was introduced a while back as a
> convenient way of writing automation code with libtsk. The automation
> framework's module interface does not use TskAuto, however, using flat
> functions instead and requiring state to be stored in global/static
> variables, causing re-entrancy problems. Other than the problem noted
> above in 1, is there a reason why these interfaces can't be forged
> into one? A nice thing about TskAuto is that it encapsulates
> accumulated state between callbacks.

There's not a big difference, in theory, between the processAttribute() method in TskAuto and the file analysis modules in the automation framework.  And there isn't really an equivalent to the post-processing analysis modules from the framework in TskAuto.   At OSDF, we talked about passing in a context variable to keep state and we can certainly look to add something like that in the future. Is that all you're looking for, or do you really want to be able to use the same code for setups?  We could also provide a method to get the TSK_FS_ATTR object from a TskFile so that you could have a single method that analyzes a TSK_FS_ATTR and it could be called from processAttribute() and the file analysis module. 

brian