WAPI Files

__          __     _____ _____ 
\ \        / /\   |  __ \_   _|
 \ \  /\  / /  \  | |__) || |  
  \ \/  \/ / /\ \ |  ___/ | |  
   \  /\  / ____ \| |    _| |_ 
    \/  \/_/    \_\_|   |_____|
                   The WOMBAT API


This package contains all the essential code to start using the WAPI (WOMBAT
API). The WAPI represents an attempt to tackle two main challenges for
security data providers:

- Many of the data access primitives are not easily scriptable. Many data
sources provide web-based interfaces that, while easily accessible by human
operators, are not convenient for automated analysis.

- The interfaces for security datasets are very diverse in structure and
methodology. The analyst who wants to take advantage of multiple data sources
to perform correlations among them is thus forced to implement ad-hoc plugins
and parsers for each data feed. This process is not necessarily a simple task,
and requires the analyst to fully understand, for example, the schema of the
SQL database provided by the data owner.

The WAPI is a remote API based on SOAP that allows data consumers to retrieve
remote information from sources according to a given communication protocol.
WAPI facilitates the integration of information generated by multiple data
feeds and enables analysts to write programs that combine data from several
information sources through a uniform set of primitives. WAPI decouples the
structure and the characteristics of each dataset from the clients: dataset
maintainers can decide what they are eager to share, in which format and to
whom, and can dynamically refine their dataset structure (e.g. add new
information types to the existing datasets) without any need to update the
querying clients, that discover the dataset modifications at runtime.

You can find more information and details on WAPI on its sourceforge page:

http://wombat-api.sf.net

=== Software requirements

* python 2.6.x or more recent
* python-openssl
* twisted python (http://twistedmatrix.com/trac/)
* a recent version of SOAPpy (http://pypi.python.org/pypi/SOAPpy)

For running the WAPI client in interactive mode:

* ipython (http://ipython.scipy.org) 

=== Installing on UBUNTU Linux

On a Ubuntu Linux distribution, these dependencies can be met with the
following commands:

> apt-get install python-twisted
> apt-get install python-openssl
> apt-get install python-soappy
> apt-get install ipython 

=== Migration from previous versions of WAPI

The new WAPI implementation has significant differences in terms of
dependencies with older versions of the library. In order to avoid
compatibility problems, you need to make sure of removing the older versions
of SOAPpy (older codebases required a modified version of SOAPpy) from your
system python path. M2Crypto is no longer used for the SSL authentication (it
was replaced by python-openssl) but its presence on the system will not affect
the execution of the new library.

=== Compatibility with previous versions

While the new client is backwards-compatible with previous versions of the
WAPI server, older versions of the WAPI client are not compatible with the new
version of the server. To reduce problems with long-lived connections, the new
WAPI server and client avoid keeping a persistent connection throughout the
client operation. Old versions of the WAPI client are not able to recover from
dropped connections with the server, and are thus unable to deal with the new
server behavior.

=== Scripting the WAPI

The new implementation is based on twisted python. Twisted python leverages a
special interaction model that has important consequences on the client
operation. Twisted is an asynchronous framework, and as such its operation
must be decoupled from the (synchronous) WAPI client interaction. This is
achieved through the wapi.client.WSession, that runs the twisted reactor in
the main thread and delegates the interaction with WAPI objects to a child
thread. The child thread is thus allowed to block and wait for the synchronous
events happening in the main (twisted) thread.

wapi.client.WSession is not suggested for scripted operation: the reactor is
not restartable, and running multiple WSessions in the same code would have
side effects. The class wapi.client.WScript uses multiprocessing to run the
reactor in a separate child process, allowing the execution of the reactor
multiple times without consequences on the parent process.

=== Installing the library

The installation is straightforward:

  cd src/
  python setup.py build 
  sudo python setup.py install


=== Package contents:

* src/                The WAPI source code
* doc/                Information on the WAPI and its principles
* ssl_tools/	      Tools to generate SSL certificates when running a
                      WAPI dataset.