[Jnetstream-users] Working on BitBuffer and filters

[Mark B. <voy...@ya...>]

I'm somewhat back in action :)
 
I'm looking into the BitBuffer issue reported and also working on filters.
As soon as I have something concrete written or fixed, I'll make an
immediate daily build.
 
The live-capture module will be used to compile libpcap expression using
Pcap.compileNoPcap() call found in jNetPcap library. Then compiled filters
will be appropriately set depending on the type of capture. If its a live
capture (a live network interface), a normal Pcap.setFilter() will be used,
if the capture is a jNS file type such as SnoopFile or PcapFile then
appropriate BPF Virtual Machine will be chosen. Remember jNS doesn't utilize
jNetPcap's Pcap.openOffline() call to read in libpcap files, but supplies
its own implementation in java which includes full editing capabilities. 
 
On Win32 systems the native libpcap BPF VM will be used using
WinPcap.offlineFilter() call, and on all other platforms the jNS provided
pure java BPF VM implementation. Since the compiled BPF bytecodes are
fairely short, there isn't any noticable difference in speed between the
native BPF VM and the java BPF VM jNS provides. Except when the filter is
set in the kernel and packets are filtered close to the NIC, that can only
be implemented by jNetPcap anyway.
 
I'm also begining work on analyzers starting with the Flow and TCP
analyzers. That is why the filters have to implemented first as analyzers
need a very efficient way of selecting a set of packets from all the
incoming packets. The goal is so that raw buffers can be delivered to an
analyzer, then the analyzer may choose to instantiate a packet or not,
depending on what its' function is. This way we avoid the overhead of
creating a packet to be passed to an analyzer that may decide that the
packet is not needed, or that it only needs a certain payload from the
buffer and discards everything else. Filters work at the ByteBuffer level,
not the packet object level, so we may be able to avoid creating packets
until the user requests them. But if the user also doesn't need packet
objects and can work with raw buffers, then no packets will ever have to be
instantiated yet flows and tcp streams will be collected.
 
For those interested, a little about how the filters were designed in jNS.
There are 3 components to a filter. 
 
1) Filter VM type = currently only BPF VM is supported. In the future,
direct compilation to java byte code will be provided as well for non live
capture filters.
 
2) Filter Expression = Pcap and NPL expressions are supported, although NPL
doesn't work yet (until the NPL compiler is finished)
 
3) FilterTarget = ProtocolTarget and FileRecordTarget. ProtocolTarget tells
jNS to scan the contents of the packet, while FileRecordTarget tells jNS to
scan the record headers of capture files and leave the packet contents
alone. When you use any of of the File.seek() operations for example you are
using a set of builtin FileRecordTarget targets. 
 
So FilterExpression takes a string and compiles it based on the subclass
that implements that interface. PcapExpression for example uses a libpcap
compiler to compile the expression, while NplExpression uses a NPL compiler
to compile and generate the neccessary BPF byte code program. 
 
I'm going to slightly alter the API and add a new factory method
Captures.setDefaultFilterExpressionType(FilterExpressionType type) which
will allow the user to override the default FilterExpression type when using
any of the capture factory methods that take a filter as a string. Those are
convenient methods such as Capture.openLive(String filter_expression), but
they do not allow the user currently to override how the expression is to be
interpreted. So you'll be able to do as follows:
 
Captures.setDefaultFilterExpressionType(FilterExpressionType.PCAP);
Captures.setDefaultFilterType(FilterType.BPF);
Captures.openLive("host 192.168.1.1");
 
The alternative is to explicitely set the expression type in the open call:
Captures.openLive(new PcapBpfFilter("host 192.168.1.1");
 
Or using the capture.setFilter(new PcapBpfFilter("host 192.168.1.1");
 
Other combinations that will be possible are FilterExpressionType.NPL and
FilterType.JASM. The defaults are PCAP and BPF for expression and filter
type.
 
This type of filter architecture provides great flexibility. Expression
syntax is separated from actual filter implementation. Other filter types
could be propriatory filter byte codes that are implemented on certain NICs
by vendors. Could even create filters for custom capture file formats that
utilize for example a database of information about packets stored in that
file for super efficiency. (That is something that jNetStream's NAP file
format provides support for.) 
 
Cheers,
mark...
-----------------------------