#3 non-atomic, out of order insertions

[David Wilburn]

Insertions of new events into the MySQL database are
not atomic, and occur in an illogical order. For
instance, when a TCP event is inserted, the event data
gets stored in TCPHDR and DATA before it is inserted
into EVENT, and the insertions are not part of a
single, atomic transaction. This means that if
communications suddenly die or there is otherwise some
sort of a problem in the middle of an insertion, data
for an event will often be inserted into the auxilliary
tables before it is inserted into EVENT. When the
system components or communications are restored, the
components first seek out the next sid,cid key value
from EVENT. This value will not be correct if the
insertion error situation outlined above takes place,
and a primary key error will be returned during
subsequent insertion attempts. Barnyard immediately
exits in this situation.

The workaround is the following:
Shut down all SGUIL components.
delete from <auxilliary table> where sid = <sid> and
cid = <cid>;
This must be done for each auxilliary table where the
insertion took place (e.g., TCPHDR, DATA, UDPHDR, etc.,
in the place of <auxilliary table>). The value for
<sid> should be the sensor that attempted the failed
insert. The value of <cid> should be the offending cid
value for that sensor's failed insert. I tend to know
these things because I log my data collection error
messages, but this could be very difficult to debug
otherwise.

There are a few possibilities for a fix that pop to mind:
1) Make all insertions for any given event a single
transaction. This may have too much overhead to be
practical.
2) Change the order of the insertions so that EVENT
gets inserted first. If some of the auxilliary tables
don't get inserted, I don't believe that it would
produce any kind of critical error in the system, you
just won't have that data is all.

[Steve Halligan]

Logged In: YES
user_id=688905

Fixed in 0.3.0