Re: [Razorbacktm-users] Dispatcher calling Nugget multiple times

[Patrick M. <pm...@so...>]

Frank,

Here's what's happening:

You have four SOs that are being loaded by rzbNugget.  All four SOs
process the same Data Type but have different App Types.

In the routing table, what you would end up with is something similar
to the following --

Data Type (EXE)
   App Type 1 (EXE parser 1)
      rzbNugget -- EXE parser 1 module
   App Type 2 (EXE parser 2)
      rzbNugget -- EXE parser 2 module
   App Type 3  (EXE parser 3)
      rzbNugget -- EXE parser 3 module
   App Type 4 (EXE parser 4)
      rzbNugget -- EXE parser 4 module

An important thing to note in this setup is that in each of those four
App Type N handlers, the same rzbNugget is receiving the data to hand
off to the parser.  Also important to note is that when rzbNugget
receives data, it simply goes through its list of shared objects and
hands that data off to the appropriate shared object(s).

What all this means is that when data of type EXE in our example is
received, the Dispatcher checks its routing table for handlers for the
EXE data type.  Then it sends a copy of the data to each App Entry for
that Data Type.  This is where you get the Dispatcher sending four
copies of the data, as expected.  The unexpected behavior is that when
rzbNugget receives the data (remember, it's ultimately the listener
for each of these App Types), it then gives a copy of that data to
each shared object that handles that data.  Since there are four
shared objects that handle this type of data, it sends a copy of that
data to each of the four nuggets.  Every time it receives that data,
which will be four times because of the four App Type entries.  This
is why the data gets processed N^2 times.

The immediate and easy fix is to have one handler for a given Data
Type per rzbNugget, which means put separate copy of rzbNugget in
individual directories with a single shared object (per Data Type) to
load.  This means that each handler will only get the input data once
and it will run as expected.

We are working on a more intelligent way of doing this routing that
should fix this issue, but in my opinion rzbNugget was never really
meant to be a nugget workhorse.  It was a debugging and convenience
tool for loading up multiple shared objects to make getting razorback
up and running faster and easier.  Ideally, a separate process would
be used for each handler.  But then again, once this routing fix gets
in, having a single handler like rzbNugget handle multiple processors
of a single data type would be a nice way to reduce network traffic
when getting copies of input data to the individual data handlers of
the same Data Type.

Let me know if this solves your issues.  I know I already sent this to
you privately, but also note that the snort 2.9 version of the SaaC is
on Sourceforge in an easy to download tarball (it was already in SVN).


Thanks,

~Patrick


On Wed, Feb 2, 2011 at 10:01 PM, Mannix, Frank - 0668 - MITLL
<fra...@ll...> wrote:
> Environment: Ubuntu 9.10, Razorback-trunk-Oct30, SaaC (Snort 2.8.6).
>
> I wrote 4 detection nuggets, each with their own uuid, to perform different types of detection work on one data type.
> I modified the rzb_http-server.c file to send all downloaded files to the dispatcher to be handled by my new nuggets.
> For each time the collector sends data to the dispatcher, the detection nuggets were dispatched 4 times.   If there were
> only two nuggets for that data type, each nugget would get dispatched twice.  So, it seems that the nuggets are fired
> off the same number of times that there are nuggets supporting a specific data type.
>
> You've probably already seen this but I thought I'd pass it along any way.
>
> Frank