[Jnetstream-users] Update - progress on Filters and Dispatcher

[Mark B. <voy...@ya...>]

My back is getting better but has really put me out of action for nearly 3
weeks since I came back from Europe. I'm able to work fulltime on jNetStream
once again :)
 
Anyway, almost done with the filters. I had to rewrite a lot of the filter
API and there are so many dependencies all over. I also added a couple of
small variations to jNetPcap to allow faster filter processing in Pcap
native BPF virtual machine and basically made jNetPcap BPFProgram compatible
with jNetPcap BpfProgram without having to convert between the 2. The
jNetPcap API is fully backward compatible with jNetPcap 1.1.
 
I needed to make the necessary changes to filtering API now, since this is
low level stuff, before heading over to NPL compiler. I should be done with
the filter by tomorrow. CaptureDispatcher with filtering mechanism is
already working. Also the ground work for Analyzers is also done. So
basically I'm just closing things up and will release a daily build as soon
as it all jUnit tests correctly.
 
Then move on to NPL compiler and get those about 60 NPL protocols compiled
:)
 
For those interested in a little more here is how things work now in
relation to filters and dispatcher.
 
There are many options for filters now. The defaults provide the fastest
possible combination which can be overridden. There are 3 types of enum
tables which influence how filters are applied:
 
1) FilterExpression.Type = provides constants for the type of syntax of the
filter expression Type.PCAP, Type.NPL, Type.WIRESHARK, Type.SNORT, etc...
 
2) Filter.Type = The type of filter this is. Type.BPF_KERNEL,
Type.BPF_NATIVE, Type.BPF_JAVA, Type.JAVA_VM and others which are "file"
record related.
 
3) Filter.Strategy = Allows the user to suggest a filtering strategy.
Strategy.DEFAULT, Strategy.KERNEL, Strategy.KERNEL_IF_POSSIBLE,
Strategy.NATIVE, NATIVE_IF_POSSIBLE, INTERPRETED
 
With pretty good defaults chosen. But the user has control over how filters
are applied both on each network interface (CaptureDevice) for each capture
session and for dispatcher filters which are applied as packets are
dispatched to listeners to filter out only required packets.
 
The dispatcher is part of the capture session (main Capture interface) and
allows listeners to register with any capture session. There are various
events for which a listener can be registered as well as an efficient
mechanism for dispatching packets. This second mechanism can completely
replace the foreground iterator based approach for fetching packets using
iterator's hasNext() and next() methods where now packets can be dispatched
to listeners. In addition, each listener can have a packet filter applied
before packets are dispatched. This allows different listeners to have
different packets delivered to them efficiently from the same capture
session. 
 
The dispatcher can dispatch raw byte buffers or packet objects, depending on
what is needed by each Analyzer. Raw byte buffers are the most efficient
means of acquiring packets as they can potentially avoid any overhead with
decoding packets into object state especially when an analyzer knows exactly
what its looking for within a raw buffer.
 
The capture session also provides Capture.loopForeground() and
Capture.loopBackground() methods which dispatch packets via a dispatcher
loop (threaded and non-threaded or blocking/nonblocking). Analyzers can now
efficiently hook onto a Capture session, specify exactly what they are
interested in and do their own processing in separate threads. 
 
Writing analyzers is the fun part and I will squeeze in between next
releases at minimum the Flow, TCP and Statistics analyzers to provide
capture and protocol statistics, reconstruct tcp streams, analyze for
errors/warnings and group packets into network flows.
 
There have been too many delays over this summer with my progress, but I
hope to make all this up quickly over the next few weeks.
 
Cheers,
mark....
"The definition for insanity is doing the same thing over and over again and
expecting a different result" - Albert Einstein